85. Small-Group Discussions

Small-group discussion activities provide all students, even the quiet ones, with an opportunity to actively engage with course material. In this episode, Dr. Dakin Burdick joins us to explore a variety of small-group discussion activities that can be productively integrated into our classes. Dakin is the Director of the Institute for College Teaching at SUNY Cortland. He has been active in professional development for almost 20 years, and has served on the Board of Directors for both the Professional and Organizational Development Network in higher education (the POD network) and the New England Faculty Development Consortium, where he was a president for four years.

Show Notes

  • Burdick, Dakin (2019). Small Group Discussion Protocols
  • Joan Middendorf — Teaching Resource Center Director at Indiana University
  • Middendorf, J., & Shopkow, L. (2017). Overcoming Student Learning Bottlenecks: Decode the Critical Thinking of Your Discipline. Stylus Publishing.
  • IUPUI — Indiana University – Purdue University Indianapolis
  • Angelo, T. A., & Cross, K. P. (2012). Classroom Assessment Techniques. Jossey Bass Wiley.
  • 84. Barriers to Active LearningTea for Teaching podcast (with Lindsay Wheeler and Hannah Sturtevant)
  • Larry Michaelsen — Professor of Management at the University of Central Missouri, pioneer of Team-Based Learning
  • Michaelsen, L., Knight, A., & Fink, L. (2005). Team-Based Learning: A Transformative Use of Small Groups in College Teaching. Stylus Publishing.
  • Team-Based Learning Collaborative
  • Elliot Aronson — Inventor of the Jigsaw classroom technique
  • Aronson, E. (1978). The Jigsaw Classroom. Sage. Chicago.
  • ZoomiOS, Android
  • Eric Mazur — Balkanski Professor of Physics and Applied Physics and Area Chair of Applied Physics at Harvard University
  • Teaching Professor Conference
  • David Pace — Professor Emeritus of History at Indiana University, Bloomington.
  • Pace, D. (2017). The Decoding the Disciplines Paradigm: Seven Steps to Increased Student Learning. Indiana University Press.
  • Pace, D. and Middendorf, J. (2004). Decoding the Disciplines: Helping Students Learn Disciplinary Ways of Thinking. Jossey-Bass.

Transcript

John: Small-group discussion activities provide all students, even the quiet ones, with an opportunity to actively engage with course material. In this episode, we explore how a variety of small-group discussion activities can be productively integrated into our classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Dakin Burdick, the Director of the Institute for College Teaching at SUNY Cortland. Dakin has been active in professional development for almost 20 years, and has served on the Board of Directors for both the Professional and Organizational Development Network in Higher Education (the POD network) and the New England Faculty Development Consortium, where he was President for four years. Welcome.

Dakin: Hi. Good to be here.

Rebecca: Today’s teas are…

Dakin: Yes, I am drinking Sleepytime Vanilla today.

Rebecca: That sounds yummy.

John: And a great way to start the day. [LAUGHTER] And I have Ginger tea.

Rebecca: And I have something different today. I have Strawberry Grapefruit Xue Long Flavored Green tea.

John: Okay.

Dakin: Nice.

John: We’ve invited you here to talk about effective ways of engaging in small-group discussions. You’ve done quite a few workshops on that. Could you tell us is about your shift to small group discussion protocols in your own classes and how you get started with using small group discussions?

Dakin: Sure. My method of instruction was lecture primarily. I started off in history and I did a lot of lectures. Some of them were good—and I’m proud of those few—but I was teaching a class at Indiana University Purdue University in Indianapolis at Columbus and it was a U.S. history survey, first-year survey. It was once a week, three hours long, and some of the students had traveled for more than an hour to reach the class after a full day of work. Some of them came in with their dinners, and I knew that lecture class was not going to get me through that class. Nobody would survive it; not even me. [LAUGHTER] So I worked with the Teaching Resource Center Director at Indiana University, Joan Middendorf. I selected several small group discussion methods: jigsaw discussions, role playing. I modified the debate system to create a evidence-based debate protocol. I used the just-in-time teaching method from IUPUI, and the classroom assessment techniques of Angelo and Cross, which I kind of regard as my Bible. The combination of those worked really well. The students remained active throughout the class, we were often surprised to find the class was over, everybody was still energized, I still had a protocol or two to go. Everybody learned each other’s names because we did random groups, and the class as a whole was tremendously successful. I was really happy with the results and I’ve used it ever since.

John: Whole-class discussions are often used but what are the advantages of small-group discussions relative to a whole-group discussion?

Dakin: I actually advocate both of those. I advocate lecture, small-group, and whole-group discussions for different purposes. The large group for me is one where I would often find faculty having the usual suspects were the only ones talking. You had three or four students in a class of 20 to 30, no one else was talking, and the faculty member would usually come to my office and ask, “Well, how can I change this? How can I get more students involved?” and often, that’s where the diagnostics began. What I found was, first of all, they needed to have preparatory homework, the students needed to do it, it needed to be graded, and if it was graded and it’s frequent assessment in order to reduce faculty load, they had to grade this lightly, and place the effort onto the students and not onto themselves. And if they did that, usually things improved. The advantage of the small groups is that if they’ve done all that work, then the students wants to talk. If you get to a large-group conversation and you’re not talking, it’s pretty boring for the students that aren’t talking. For the faculty member, it sounds like it’s a really good conversation because they’re the center of the wagon wheel, they’re the center of the hub, and so they’re constantly talking with those three or four students, but they’re wondering what’s happening with the rest of them. And that’s a good thing to wonder about, frankly. If you use small groups, you have the advantage that more people are talking at the same time. So instead of having one person talk at a time, you can have six people in the room talking at a time, so there’s a lot more conversation taking place and hopefully, more change in learning, which is important. Students in small groups feel more free to talk, there’s less risk in a small group, they can gain confidence from that talking, and they’re more active in the classroom at the same time because there’s more people talking. It also gives them the chance to practice disciplinary skills that the faculty member has put into that assignment. So the assignment shouldn’t be about declarative knowledge or facts, it should be about how can you do something in the field? How can you emulate the skills of an expert? The other piece of this is that the small groups have been demonstrated to be effective. Students in the 1920s said they preferred this sort of discussion—at least large-group discussion at that point—and then by the 30s and 40s, there was research and social psychology showing that small groups were more effective in promoting change and student learning. And from there it went on and since the 50s and 60s, a lot of different types of protocols have been invented and developed. There’s just a lot of advantages.

John: Could you tell us a little bit more about how you ensure that students come prepared to actively participate in those small group discussions?

Dakin: The main thing is grade it. The rule is that students—according to the Carnegie unit—are supposed to study for two hours outside the class for every one hour in. Well, the Study Study at Indiana University showed they weren’t doing that. They were studying one hour outside of class for every hour in, and it was usually on Sundays. That means that we kind of have to take that knowledge and make use of it. We know they’re going to do their work on Sundays. So okay, the assignments going to be due Sunday night, 11 o’clock, 11:55. But you make sure that they work harder. You don’t feel guilty about putting more work on them because more work means more practice. All of them could use more work reading, all of them could use more work writing, so that’s what I have them do. And then I make sure they turn their assignments in on Sunday night, I grade them Monday morning—which is a principle of just-in-time teaching—it’s preparatory homework, I read that, and then I modify my class based on what the students bring to it. And so I can see—first of all—their weaknesses. I can see their misconceptions. I also can see their strengths. Occasionally they have real strengths they bring to the class that would be totally invisible if I hadn’t done this work. And my example for that is I had a class where I was teaching the My Lai Massacre—about Vietnam—and I had in the class two very strong students. One was a G.I. who had fought in Vietnam, and one was a First Lieutenant who had taught the rules of land warfare at Fort Benning for three years. Both of these guys were A, smart; B, aggressive; and C, constantly fighting with each other because it was enlisted versus officer. So they dominated the class very often. And when I got back their feedback, they told me their knowledge and I hadn’t seen that knowledge before. I did not know that these were their backgrounds. So I spent an extra two to three hours reading about the rules of land warfare, came to class prepared, and instead of a class that would be a trainwreck for me, what happened is they came in and the Lieutenant said, “Everybody knows the rules of land warfare. So they’re all guilty, and they are all responsible,” and the G.I. says, “No G.I. is going to read a 100-page field manual on the rules of land warfare.” And the Lieutenant says, “Well, there is no 100-page manual on the rules of land warfare,” and I said, “Well, actually, there’s three. [LAUGHTER] There’s the 1956, 1965 and the 1973 (revised on the basis of My Lai).” Okay, so that stops that conversation. Then I turned to the GI and I say, “Okay, but every G.I. has those little plastic helmet liners, right? …with the 10 rules of land warfare on them.” “Yeah.” “Okay. So we’re agreed; they knew the rules of land warfare. Some follow them, some didn’t. Now, let’s talk about why,” and at that point, the conversation became really useful. First of all, all the other students could participate, because they now had the background, and the two people that were real experts in the room could help us kind of determine why people followed or did not follow those rules. And again, if I had not done that preparation—just two extra hours—that class would have been ruination.

Rebecca: What are the kinds of questions that you have students respond to that maybe elicited some of the information that helped you? What are the keys to asking good questions for that preparatory work?

Dakin: I think the keys to that are knowing your subject. So, everybody that is a content master—every faculty member —has their own expertise, and it’s pretty impossible for me to name the prompts that they might use effectively. But they probably know them, they’ve probably seen them in their graduate work. They know that these are the elements that made up their dissertation exams, their qualifying exams, and they’re probably pretty smart about what are those major issues in their field that need to be discussed, and to be prepared, and the students need to be prepared. The big thing is making sure that we’re talking at a high level of cognition. So in Bloom’s taxonomy, talking about analysis, talking about evaluation, those are the levels you want to get at. And those are almost impossible to get at with multiple-choice questions or tests, so that means there has to be conversation, there has to be writing. Those are elements that are important.

John: Earlier you mentioned that you use random assignments for small-group activities. Do you do change the groups on a daily basis or do you have more persistent groups?

Dakin: Occasionally, if I’m doing long-term group work, I will do some sort of pre-test, find out what the strengths and skills of the students are, and then place them mindfully into those groups, so that they construct useful groups because they’re going to be in those for half a semester. I’ll do a swap halfway through, but that’s a long time to be in a single group. For the random groups, I definitely do that and I do that on a daily basis. The students originally complain about that, but they get used to it pretty quickly and they’re ready for it. And the advantage is that they get to meet everybody in the classroom. They get to be in a group with everybody else and that builds trust and it builds community. And that allows them, by halftime through the semester, they know everybody, they’re comfortable with everybody, they trust that other people have their best intentions at heart, and then the conversation just escalates from there because everybody’s now willing to talk.

Rebecca: In our previous episode that we just released last week, we discussed some of the issues that can come up when you’re using evidence-based practices for the first time. A lot of people know or buy into the idea of small-group discussions and might just go for it without necessarily having a good plan in place, and things might go awry. Can you talk a little bit about ways to be prepared for trying something new, the kinds of things that might go wrong, and how we might adjust ourselves a bit as faculty members as we’re trying new things?

Dakin: The thing that I find usually is that people just don’t give the new techniques a chance. It’s scary. Now there was a study back in the late 90s out at Brigham Young and they asked faculty two questions. They asked them first, “What do you think are the most effective teaching methods?” and then “What do you do?” and they were diametrically opposed. [LAUGHTER] And the reason was time management, people are very busy and the stuff that’s effective takes a lot of time to do—or they think so. So I view my job as an instructional designer when I’m helping them to reduce that amount of time and make sure that they can do that. So first, make it time manageable, so that you can do the task and you can feel comfortable. Secondly, trust the system. Trust the change you’ve made. You made this change for a reason, trust it. And third, trust your students. Your students want to succeed, they want to learn. Trust that and have them help you make this successful. Tell them what it is you’re doing in the classroom, why you’re making this change, why you think this is going to help them learn better, and then also use feedback from them to get it. So I typically will use something called a stop, start, continue—What do you want me to stop doing? What do you want me to start doing? What do you want me to continue doing?—and use that student feedback to then modify the class. So it’s kind of like a mid-semester evaluation, but I feel like doing it whenever I do… it is just fine.

John: Now earlier you mentioned that whole-group discussions have a place. In what sort of sequence might you use or in what combinations would you use small-group discussions and then whole-group activities?

Dakin: My process is basically four-part. One, preparatory homework. There has to be preparatory homework and it has to be graded—lightly graded—and it should be moderately challenging. Next, they come to class, there’s a brief lecture and the lecture introduces the material, frames the questions we’re going to talk about today, maybe corrects some of the errors that were made in that preparatory homework, also celebrate successes from that preparatory homework. Once that lecture is done, maybe 10, 15 minutes, then move them into small group work. Small group work can be anywhere from one to two minutes in a lecture hall to 40, 50 minutes—and you might do a whole session on the rest of that piece, maybe a debate or some large-scale exercise—usually though, about 10 to 15 minutes in small group. Then when you hear the sounds rising, that means they’re talking about things they enjoy, which means their social life, [LAUGHTER] and so it’s time to stop them. You’ll also see sometimes that there will be a student—maybe all the A students somehow got at the same table—and they’re done three to four minutes before everybody else. Well, the point of putting them into small groups is to build energy and confidence and you don’t want your A students to be bored. So if you have a group that’s done first, you appraise how much of everybody else got through, “Can I stop this now?”—usually you can—and you bring them back to the large-group discussion. And in that large-group discussion of 10 to 15 minutes, you do debriefing and you find out what they think they know, maybe use a classroom assessment techniques from Angelo and Cross, and you evaluate and you build feedback that you can use later. And then once you’ve got that, then you move back to the lecture, and you clean up the misconceptions, you explain and reframe the next issue, and then it’s just a cycle. So it’s lecture, small-group discussion, large-group discussion, and continually like that.

Rebecca: What are some strategies that you use in small groups to make sure that everyone participates or is engaged and stays on task?

Dakin: First, make sure they’ve done the homework. Secondly, randomize so that I’ve got some good students and some poorer students in the same groups, so that we have people that can interact—also, so that people can learn about each other. To keep them on task and walk the room: first of all, be engaged with them. Listen to what they’re saying and if it’s on task, you just congratulate them and move on, if it’s off task, okay, now start working with them—and there’s going to be one group that’s off task, certainly. Other pieces are… that you might encounter a small group where there’s a number of dominant individuals. So there’s a couple of people that are really assertive, and they’re talking all the time, and they are just dominating the whole piece, and the other people aren’t getting heard. And so in that point, then you start introducing other discussion protocols that will allow more inclusivity: so things like expense account, talking stick, things where other people’s voices are valued. Another one would be Larry Michaelsen’s Team-Based Learning that also does that.

Rebecca: Can you talk through each of those, for those that aren’t familiar with each of those?

Dakin: Sure, let’s start with talking stick. Talking stick is very simple. You have an object—usually a pencil or something—and one person gets that stick, and is able to talk for one minute without being interrupted, or any comments from anybody else. And then you pass it to the next person, for one minute they get to talk, and it goes around the room that way. And then once it’s gone around once, everybody can talk at once and kind of work out what it was that they heard said, but everybody’s voice is listened to and heard during that time. That’s a rather formal way. Another less formal way is expense account, which is maybe you give them three—or however many pennies you want—three tokens. And they pass those tokens in each time they talk. So the assertive ones are going to spend their pennies very quickly. [LAUGHTER] And the less assertives are going to then have a chance to spend their pennies. And when everybody has spent their pennies, you all get your pennies back and now you can start again. But again, that’s a way to give people a chance to speak. But people can choose when they want to speak, rather than having this turn where it’s coming around. And it’s very set. Larry Michaelsen’s Team-Based Learning is much more complex. Larry started this in a lecture hall. And so he has basically an IRAT and GRAT. And the IRAT is an Individual Readiness Assessment Test and that GRAT is a Group Readiness Assessment Test. So he has them take an IRAT first, and as an individual give their answers, turn that in—that’s a grade—and then he has them do the GRAT. So as a group, they now turn in their group grade. And he also uses the scratch off cards, the if/at cards, that sort of thing. Initially, the assertive ones—again—are giving the answers. But as they discover that they don’t have the right answers all the time, then the quieter ones in the group suddenly become more important to the group because their grade is dependent on this. So they’ll start asking, “What did you get for this? You seem to get A’s all the time. Can you please help us?” And so that’s his method of doing that inclusivity.

John: And in that approach though, I believe he recommends persistent teams over the course of the semester, so that they develop that sort of team dynamic.

Dakin: Yeah, very much so.

John: Are there any other small group activities that you like to use?

Dakin: I have a lot of protocols that I’ve gathered over the years—probably got 40, 50 protocols—and the ones that I select are the ones that are low risk. So I’ve kind of classed them as low risk, medium risk, and high risk in terms of how much risk does the student feel when they’re in the classroom doing these… and I like low-risk things which are usually small group where they’re by themselves and they’re talking, and it’s not in front of the faculty member, and they are not having to answer to the whole class in front of a large group. So, some of the pieces I like are jigsaw—which comes out of Elliot Aronson’s work in the 70s—the idea that you break up an assignment into five pieces, and each of the students in a group will do one of those five pieces, and then they will talk in class and share out what they’ve learned from each of those five pieces. So it’s a great way to synthesize a lot of data that maybe you don’t want all students doing. So when I was teaching my Middle Eastern history class, each student was responsible for a different country and they had to do a lot of reading on that country. But if I had all the students do all that reading, it would have been far too much. So instead, I can have these various countries sit at a table and then have a conversation, and the student representing Israel can talk about Israel’s point of view, and the student doing Jordan can talk about Jordan’s point of view, that sort of thing. So that’s one method I love and I even do a double jigsaw. But I only do a double jigsaw maybe twice a semester and they’re at moments where there’s so much content, that there’s absolutely no way we can cover it. And the best example of that is U.S. history survey, first day, which is the dawn of time to 1492, which I think is horribly disrespectful to everybody that was in North America before 1492. So we do a double jigsaw, which is where you have a jigsaw that creates experts at each table and then those experts then are now experts in five different topics. And those people then go off to create super jigsaws. And that works well, but it takes a lot of time. The other one I love is role playing. Role playing… just because it’s my age… I grew up with role playing, but I’ve done a lot of different styles of role play. The one that I think I use the most in first-year history is Articles of Confederation. Everybody takes a representative to the Confederation and talks about what it was that person was like, and why they voted the way they did, and what were their goals. And then we skip ahead to the Constitutional Convention and we talk about who’s still there, who’s not there, why are they not there? If they’re still there, do they still have the same opinions? Are they still voting the same way? Why are the results different at the Constitutional Convention as opposed to with the Articles of Confederation? So that’s a good one. Other classics are the Oregon Trail… everybody loves the Oregon Trail. And unlike the computer game, what you learn is not many people died on the Oregon Trail. People who died most were the Native Americans who are along the trail, everybody else pretty much made where they were going, but that had to do with who those people were. The other one we did was Cuban Missile Crisis. Did the Cuban Missile Crisis and role played the various operatives in the Cuban Missile Crisis. And then next week we talked about Watergate and again looked at those same operatives and where they were now in the Watergate plumbers. So that was also useful …those kind of things. There’s lots of different ways to use that though. I’ve seen people use that with theorists. So in psychology, different theorists are represented by the students and they argue their different theories and try to figure out how these things go together. Role playing is obviously one I love.

Rebecca: Can you talk a little bit about how you set the role play up? You mentioned what some of the topics are and when you use it, but can you talk a little bit about the logistics of setting that up and how you have students prepare for that?

Dakin: And those vary a lot. When we’re doing the Articles of Confederation, I just have a list of representatives and I asked them to choose one and then we go from there. So my prep on that is zero, they are the ones responsible for that prep. On some of the others, there’s a lot of prep. With Oregon Trail I worked out, I took a K-12 game that had been done for Oregon Trail, and then I made it much more complex and they had to purchase their gear so I had a full list of gear, I had a list of where they were going to stop, how they were going to stop, and what the mortality rates were. So I basically created this whole game around it and then they played through that. With the Watergate and Cuban Missile Crisis, it was kind of halfway between there. I made cards with each of the people they would role play and on the back—like a Clue card—it tells you who this person is and what their role is and then I gave those to them. And then from there, they again generated most of the data.

John: You mentioned you have these organized by levels of risk. How would you recommend using the different levels? Would it make sense to start with low-risk activities, and then as more trust is built, build the higher ones, or would that be affected by the level of the class that you’re teaching, whether it’s introductory or more advanced?

Dakin: Yeah, it definitely depends upon the purpose of the class. I tend to teach introductory classes so I build a lot of trust, I use a lot of low-risk pieces, and I’ll move to maybe medium risk by the middle of the semester—or maybe I won’t ever use medium risk, it’ll all be low risk—because I’m trying to get them to get used to college and figure out what that’s involved in. If I am teaching a class that’s kind of a gateway or portal class that’s going to lead on and it’s supposed to cull out people, it’s supposed to find out who the best people are, then maybe it makes sense to start doing some of those high-risk pieces, but I probably wouldn’t do that until at least the third year. Build a lot of confidence, a lot of trust, and there’s a lot of learning that has to take place before that, before you get to that point. Traditionally of course, if you look at law school or medical school, they have a lot a lot of high-risk protocols, because there’s a lot at stake and people have to do well. And I remember when I started working with the med school that I read academic medicine, and one of the articles was, “We should abuse our students less.” [LAUGHTER] Not, “We shouldn’t abuse our students,” but, “We should abuse them less.” So, that kind of gave you a sense of what we were dealing with.

Rebecca: Can you talk a little bit about some of the small-group techniques that you use in online environments?

Dakin: Sure. Online environments—actually, the small groups—I usually leave that up to the groups themselves. So if there are groups of students that I’m having work on particular topics, I’ll have those students organize, say, a synchronous conference where they get together on Zoom, and they talk about these things. So they get to pick whatever tool they want—I’ll recommend Zoom because it’s free and you can have up to 40 minutes free, and you can record it. It’s very simple—but they will do that work by themselves. If we’re dealing with, say, a discussion forum, what I’ll generally deal with is ways that students can interact kind of more of at a large-group level, because there’s really no need for a small group when you’re on the discussion forum. But what I do want to do is get rid of the old “post one, reply two” because after you’ve taken two or three online courses, you’re pretty bored with that protocol. And so giving them different ways to think about it and moving the jigsaw into it, moving a debate into it, moving role playing into it, those are all really useful.

Rebecca: Can you pick one of those more complex ways of using a discussion board and talk through how you set that up or organize that?

Dakin: Sure. I think the main thing I do is really—it’s not so much about the organization of the board when I’m doing it—but building student activity. I do a big sales job in terms of talking about what is the value you get from an online course. Now, if it’s just teacher to student, I think that’s a really limited amount of value because there’s a lot of good books out, you can read, you can train, you can look at YouTube, there’s all sorts of great ways to learn. But a real value from an online course for me is who is in that class with you and finding out what their strengths are and what they can bring to it, and that’s where a large part of the education comes. So I don’t use this “post one, reply two” but I do want them to make sure that they are responding weekly to their colleagues, but at a level they feel is appropriate. So don’t say something if you think it’s totally pointless. But if you have a comment and you feel it’s worthwhile, say it, because we need to hear it. That’s the largest part of this. In terms of the organization, the only pieces I’ve done in terms of organization have been very slight. So, with an assignment, you turn in your first post Wednesday, and you turn in your final post on the piece, on Friday, Saturday, or Sunday. So the initial post is what your response is—out of a think-pair-share, this would be the think part—that you’re doing your initial writing. And then the second part of the week is simply the sharing part. So now you’re responding to those students. Now, if you’re in Canvas, you can do this with setting up the initial one as your due date, and the second piece as your until date, so you can do it within one assignment. Unfortunately with Blackboard, you can’t do that, you have to have two separate assignments. But that’s the only real difference.

John: What about larger classes? What techniques do you recommend there?

Dakin: In large classes, I often talk about Eric Mazur and his peer instruction, simply because I can send them to the videos he’s got on YouTube, and he’s got a lot of videos there. He’s got a lot of publications. So that’s great, I have a lot of resources I can send them to that they can start working on. But Eric’s technique is largely one that applies to an interactive lecture. It’s not really small-group work per se, it’s a way to maintain activity by the students and also makes sure that you’re getting feedback back on what they’re understanding. But since Eric uses multiple-choice questions, he’s really not getting beyond that understanding- or application-level question. So, the issue really with small-group work and large classes is really not about the size of the class, it’s about the furniture in the class. So, you can do small-group work with a very large group as long as you’ve got movable tables and chairs. So, I did this at the Teaching Professor Conference a few years ago. I had 110 people in the room and we did value line, and we did jigsaw, and we did all these different things, and it’s very easy to do as long as you’ve got the furniture that allows you to do it. The hard part about a “large” class is really it’s about the lecture hall and the furniture in it. So if you’ve got furniture that’s fixed, if you’ve got a table that’s fixed, if you’ve got chairs that are fixed, it’s hard to have more than two to three people working together at any one time because they can’t turn around—they can’t do anything else. Also, since you’re in a large lecture hall, there’s a lot of noise. So again, you don’t want to get more than three people because you won’t be able to hear the others. If in a lecture where students can turn around, then you can have a larger group of say four to six. So you have two to three in the front row, two or three in the back row, and they’re talking together in that small group. And I’ve seen small-group work in lecture halls with as many as 160 people, so I know it works. It does take some effort in terms of arranging it. Usually they don’t do random small groups every day—because that would be chaos—but they do long-term teamwork and the faculty member who did this was David Pace at Indiana University—he was very good at this, he’s the one who taught me how to do much of this—he does a pretest, he organizes the students, he puts them into these long-term teams, and then in those teams in the lecture hall, they have the seating arrangement where they’re sitting. And then when he wants to do small-group work, he’ll do his lecture, and he’ll do small-group work, then he’ll do a debrief, same sort of pattern.

Rebecca: What do you find your role is, as an instructor during small-group work? You want to put a lot of the onus on the students, but what’s your role during all of that and how does that scale up to a big class?

Dakin: My role is—as an instructor small-group work—is essentially challenging, adding to, and supporting. Making sure that they know they’re encouraged and they’re doing a great job and going around doing that sort of thing as I walk the room. A lot of the work I do is really the preparation. Making sure that those things are well thought out, that I have a lot of idea of which directions they can go, and to, after the class, make sure I’ve done my reflection: I’ve written down all the weird places they went so that I know that those are possibilities and I can be ready for those, or maybe I just work towards those. Maybe those were better ideas than the ones I came up with—which is actually one of the big advantages of small-group work because you are paying more attention to the students, the students have a bigger role in the class, and your life isn’t as boring. If I was doing the same lecture 20, 30 years later, I would be bored to tears. But as it is, since I’m using these, every semester is different because every group of students is different. So my life is constantly interesting. And it’s almost like doing improv, really, in a way. You have to be a little brave about it, you give them opportunities, but there’s a lot of trust, you trust the students are there to help you. And everything goes well, even with apathetic classes that when I’ve walked in, the class has just been dead, they don’t want to do anything. After a week or two of this, they start getting into it. And by the end of the semester, they’re the same as every other class, and it’s going very well. So it’s highly enjoyable. So I think that’s it, make the class fun, get them to trust and encourage them to do their best work.

John: And in large classes if you have TAs, you could have TAs going around and doing the same thing, just so that you get more of the room covered.

Dakin: That, or if you’ve got a tight space to deal with, you could also have a backchannel going, so people in the groups are reporting out and the TAs are looking at that backchannel through Twitter or something else and kind of getting those ideas and feeding those back to either the students or the instructor.

John: Earlier you mentioned that light grading be used. Could you elaborate on that a little bit?

Dakin: I think that’s the hardest thing for faculty to do is light grading. Faculty members really want to mark everything. If they see something wrong, they will mark it. And I must admit, myself, when I’m posting to Facebook and somebody writes something and spells it incorrectly, I have to respond. It’s annoying, but I have to do it, and it’s the same way with grading. People will try to grade everything and they will eat up their lives giving these huge responses back that the students really aren’t going to listen to. Nobody has time to make all those corrections. So the smartest guy I ever saw was Bob Ferrell, who was a professor of history. And Bob was highly published—he had 50 plus books—and he still had a line out the door of students that he talked to every day and that was highly admirable as far as I was concerned. And so I wanted to find out how he did this, and what he did is… I took a readings class with him and I handed in a paper a week, and we worked through that. And every paper, the first time he got it, he marked it up pretty heavily to show, “You need to work on your grammar and I’m watching you.” But after that, every week, it was three things. He’d mark… circle one, flip a couple pages, circle another, flip a couple pages, circle another, “There you go,” out the door, you’re done. And so for me it felt like, “Oh, I only have three things to change. This is great, I’m really close to getting that top grade.” And next week it would be another three. And next week it would be another three, and so on. So, he was doing light grading, he was giving me feedback—feedback that was useful to me—feedback that was moderately challenging. I didn’t feel at sea, I felt I could do it. Great. And so I would do it. And that’s the way I come to this. The way I implement it is, say if I’m in a freshman class, I will have the students writing say 1000 words response every week, which for a freshman class seems like a lot, but I want them to work and I want to hear their voice. I will tell them not to use any quotations, I want to hear their voice, I don’t want to hear somebody else’s. I want to hear them thinking, and if they don’t agree with the text, argue with it—that’s fine. If you don’t agree with me, argue. That’s what you should be doing. You’re trying to construct your ability to speak and write. So, when they do that, they then turn these pieces in, and I grade them but I grade them lightly, which means I’ve got now 40,000 word essays I’m supposed to be grading, that will take me about 40 minutes. I spend about a minute on each. I just kind of flip through it, I can tell if somebody’s done the reading or not, I can tell if there’s a major issue or not, and then I write down my responses but I don’t give them to the students. I just give the students grades. And when I get to the class, I’ll do a group grade. So at the beginning of the class, I will then do a couple things. One, I will celebrate some people, and I’ll talk about that in a minute, but I also make corrections. I’ll do grammatical corrections, will say “Here’s the five grammatical errors of the week,” and by about mid semester, I’m still showing some of those grammatical errors up on the board and the girl who’s done it says, “Oh my God, it’s me again!” So they get it and they’re trying to reduce them and that’s fine. The other thing is I talk about misconceptions. Say somebody has a misconception about a particular piece, I’ll say, “A couple people had a misconception about X.” Now it’s not a couple people, it’s Joe. It’s always Joe. Joe’s in the back room, Joe never does the reading, Joe’s having trouble. But Joe knows he’s not doing the reading and Joe knows he’s not getting a good grade, he doesn’t need to have his name called out in front of the class. So I say, “A couple people had this issue.” I talk about that and address it. And then the next part, the celebration. So, in order to make them feel better about what just happened, I then say “Now then, I wanted to talk about some of the great things that were done this week. So first of all, Jenny had this fantastic response, it was just so meaningful. I want to share it with you, because I think it’s really worthwhile listening to. And Bob said something that no one has ever said in this class before and so I think it’s important to address that.” And then maybe I talked about Jim, “Jim really did a very deep reading of the text, he brought up some serious issues that I did not bring up myself, and I think we need to explore those today. So that’s part of our discussion today, it will be based on what Jim has talked about.” So that’s the celebration, but every week, it’s a different three. It’s never the same three, it’s never always the A students. Over the course of semester, I find a way to celebrate each and every student in that class, including Joe. And Joe is hard. Joe’s really hard. So I’m always watching every week to see what Joe is talking about and if Joe says something good it’s like, “Yes, I can now celebrate Joe! Good. Check.” I’m celebrating. And that’s the way light grading works for me, it allows me to spend more time interacting with students, less time interacting with their work.

Rebecca: I’m sure we can all take advice on reducing grading, right? [LAUGHTER]

John: When we talk to faculty about using group discussions in class, one thing they often raise is a question of when students are teaching each other—in general with peer discussion or peer activities—there’s a concern that perhaps it may reinforce misinformation. How can you be sure that that doesn’t happen with small-group activities?

Dakin: Well, I don’t think you can be sure it doesn’t happen, but you can certainly set up a system to check for that and make sure it’s not happening or that if it’s happening, you’re correcting it. So the way to do that would be use some ungraded assessments, those classroom assessment techniques from Angelo and Cross. In the large group during the debriefing, some of those may come to light again, and you can then use lecture to correct those misconceptions once they become apparent. I agree that small-group work builds confidence, but it can really be like the blind leading the blind sometimes, especially if the students haven’t been doing the preparatory homework, and especially if the groups aren’t randomized.

John: And if you do that sort of pretesting, where you’re getting the initial feedback, you can tell what those misperceptions are, so that you can be prepared to address them during the class, which should help reduce that issue.

Dakin: Sometimes… yeah. I have to know which questions to ask, and often I don’t. So it’s that ungraded assessment where they toss back an answer that completely takes me by surprise. Oh, I am so surprised. Now I know what your misconception is. But I couldn’t have guessed at it.

Rebecca: I think that’s important to remember too, that [when] you’ve been teaching for a long time, the misperceptions that you might have come across five years ago are really different than the misperceptions that you might experience this year, because the experiences of our students change and the group of students change, and all of that influences prior knowledge and prior experience that influences how they might interpret material.

Dakin: That is so important. Over the 30 some odd years I’ve been teaching, my students have changed a lot, not only in their content knowledge and what they know and what they’ve experienced, but also how they think and how they behave. And again, that’s the strength of using small group work, because you get to see how they think and how they behave. And they’re not just sitting there in rows in front of you and you imagine that’s the same class you were teaching in 1987. [LAUGHTER]

Rebecca: So we always wrap up by asking, what’s next?

Dakin: I’m collecting all these protocols for my own use and also to help others, so it’d be great if people listening to this podcast could send me some new ideas, send me some more protocols, so I can share those out. The way to do that is to send them at teaching@cortland.edu. That’s our email address. In the meantime, we’ve got a new Institute for College Teaching down here, we finished up a faculty needs survey. We’ve got our advisory committee in place, and we’re just about to start setting up priorities for next year. So, there’s a lot happening, I just don’t know what it is yet.

John: Because you’ve just taken over that position fairly recently, right?

Dakin: Two months ago.

Rebecca: Oh, the surprises you might find, right? [LAUGHTER]

Dakin: I have been very pleasantly surprised so far. I have found a lot of really skilled and dedicated faculty, and I’ve just really been enjoying talking to them. I know I enjoy this because it’s a challenge, and I love a challenge. And they are so well-educated already. It makes me work very hard.

Rebecca: Which means you’ll never be bored, right?

Dakin: Exactly, and that that’s why it’s so important to me.

Rebecca: Well thank you so much for joining us today. It’s been really interesting, and I think will help faculty as they plan for their next teaching adventure.

John: Thank you and we will share some of the resources that you’ve provided in the show notes as well.

Dakin: Thank you. I’ve really enjoyed our conversation. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

84. Barriers to Active Learning

Despite research demonstrating the efficacy of active learning approaches, observations of classroom instruction show limited use. In this episode, Lindsay Wheeler and Hannah Sturtevant join us to explore potential interventions to overcome the barriers to the adoption of effective teaching practices.

Lindsay is the Assistant Director of STEM education initiatives at the UVA Center for Teaching Excellence and an assistant professor. Lindsay’s background is in chemistry and she has a PhD in science education. Hannah’s a postdoctoral research associate at the center. Her PhD is in chemistry with an emphasis on chemical education.

Show Notes

  • Henderson, C., & Dancy, M. H. (2007). Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics-Physics Education Research, 3(2), 020102.
  • University of Virginia programs
  • Teach Better Podcast Episode 80
  • Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE Life Sciences Education, 12(4), 618–627.
  • POGIL- Process Oriented Guided Inquiry Learning
  • PODLive! Webinar
  • Meghan Bathgate — Postdoctoral associate at Yale University
  • Emily Walter — Assistant professor of Biology at California State University, Fresno

Transcript

Rebecca: Despite research demonstrating the efficacy of active learning approaches, observations of classroom instruction show limited use. In this episode, we explore potential interventions to overcome the barriers to the adoption of effective teaching practices.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guests today are Doctors Lindsay Wheeler and Hannah Sturtevant. Lindsay is the Assistant Director of STEM Education Initiatives at the UVA Center for Teaching Excellence and an assistant professor. Lindsay’s background is in chemistry and she has a PhD in science education. Hannah’s a postdoctoral research associate at the center. Her PhD is in chemistry with an emphasis in chemical education. Welcome, Lindsay and Hannah.

John: Welcome.

Hannah: Thank you.

Lindsay: Thank you.

John:   Our teas today are…

Hannah:  I have a lemon filled Earl Grey tea. [LAUGHTER]

Lindsay: I have my water.

Rebecca: I’m drinking English Afternoon.

John: And I have Blueberry Green tea. We’ve invited you here to discuss the study you’ve done on why STEM faculty are reluctant to try new teaching techniques. What prompted the study?

Lindsay: One of the big things that we try to focus on in our center is how we use local data to drive faculty development to help improve teaching and learning on our campus. As part of that back in 2016, 2017, we did a large-scale observation project where we observed over 200 STEM undergraduate courses. And we wanted to look for differences in the different instructional practices that our faculty were using based on whether they were engaging in our center or whether they were not. And this was sort of the beginning piece of driving everything that we’ve done since then, because we did see differences in their instructional practices between faculty who have and have not engaged in our center, but we didn’t see as much as we thought we would. And so we really wanted to further explore that and understand what things were hindering faculty from doing what they wanted, using evidence-based practices, particularly those that had gone through our Course Design Institute and had done other programs with us. And these are things that we heard anecdotally but we really wanted to better systematically measure this. That’s where Hannah comes in as a postdoctoral research associate and I’ll let her talk about what we did to further explore this idea of what the barriers were.

Hannah: So I came into the project when Lindsay was wanting to develop this barrier survey of some kind. And so I started by going through the literature and I found a lot of work that was of a qualitative nature that people had done in various fields, looking at barriers to implementing evidence-based practices and research-based practices. A lot of different terms are used so you have to know which ones to search depending on which field… in which journal… you’re in, so I got introduced to that, which was a bit of a challenge, but was able to kind of sort out all these different areas and find work that had been done both in DBER in specific fields and then more of the faculty development field. So I pulled on all of those different sources, but I did not find any survey instrument that was of a quantitative nature that delineated all of these different barriers that had been found in the qualitative papers. I found a couple surveys that had little sections of barriers and then I found a survey that looked at institutional climate, but I didn’t find any that delineated lots and lots of barriers that I’d seen in the qualitative work. So I drew on all that qualitative work to develop a survey instrument that we then piloted, so that’s kind of where all that came from.

Lindsay: And to add into that, there are benefits for doing interviews and qualitative work, but we wanted to really be able to find a way to quickly but systematically capture these barriers. Because as I mentioned, we are really interested in using that locally driven data and there’s only so many people in our center that can be able to do that work. That was part of the driving force behind developing the survey itself.

John: Just backing up a step, Hannah mentioned DBER. For our listeners, could you define that just so there is some clarity there?

Hannah: Yes, DBER is Discipline-Based Education Research. So I am a chemical educator, I’m a DBER researcher. Biology educators, astronomy educators, those are all DBER researchers.

John: What did you find in the survey?

Hannah: That survey instrument was not just barriers, but also some related ideas, so it included a section on teaching-research identity because that was something that came out of looking at the literature and seeing that this tension between teacher-researcher identity seemed to be something that might be a part of the barriers. So we added a section on that because we’d also not found any survey instruments that delineated those in a quantitative way. So moving on to the study. We piloted with 86 and that was a subset of the 150 instructors that were observed in the study that Lindsay mentioned earlier, that was kind of a rationale for the current research. So we were able to get 86 complete datasets out of that from the 150 that we sent it to. So first of all, we had 46 Likert scale questions— different statements about barriers that faculty participants could rate on a scale of one-to-five of, “This is not at all a barrier for me,” to “This is a barrier for me all the time.” And when we looked at the results of all of those Likert questions, the top five were number one—and that’s 65 percent said this was at least a moderate barrier for them so they rated that at three-out-of five at least—was lack of time. The second was tenure and promotion guidelines. The third was fixed seats or infrastructure constraints at 61 percent of faculty mentioning that. Number four was that students don’t come prepared at 59 percent. And then five was that too much prep time in particular was required to implement these evidence-based practices, that was at 50 percent of people mentioning. And we investigated those also qualitatively and the qualitative question that we asked—the open-ended question that we asked—was simply, “What barriers are most significant to you in your own teaching and why?” That question was a bit different. So we had all 46 of those Likert scale statements that faculty rated, but this one was getting at, “Okay, so now thinking about your work, what is the most significant barrier for you?” so it was a slightly different question than what we asked to the quantitative, and it produced some very interesting results. So what these 86 respondents said is, number one, aligned with the quantitative at lack of time, but that was only 57 percent that were saying that. Second was classroom space and lack of needed technology at 22 percent. Third was the lack of institutional support, so there’s a lot wrapped up into that question. And then number four was a variety of student-related issues and student resistance and not doing what they’re needing to do at 12 percent. And then finally, the lack of TA support and classes being too large coming in at nine and eight percent. So that gave us a greater understanding of what’s the number one issue for our particular faculty, as well as the overall landscape of all of these different Likert scale barriers. So that was interesting and drove what we were doing in our research. So one of the other results that came out of this work had to do with satisfaction and dissatisfaction with evidence-based practices. We asked the faculty who responded to the survey to go through a list of evidence-based practices and say which ones that they used. And looking at one of those practices—for instance—collaborative learning, we asked them if they were satisfied or dissatisfied with that practice—or both—and that was the practice that people were most dissatisfied with. And when we looked at that, and we compared it with their barriers results, we found descriptively that those faculty had higher barriers across all of the different barrier groupings on the survey. The ones that were dissatisfied with collaborative learning had higher barriers across all the different barrier groupings and we ended up grouping those into five. They had higher barriers across the board and we had been investigating, “What does that mean?” and as we’ve been expanding the study, wanting to get more data to really understand that and look into the policy responses on why they’re dissatisfied… things like that. But what came out of that was what Lindsay referred to, was the need to support faculty, not just before they implement an evidence-based practice, but when they’re implementing it. And we found this excellent study from Henderson and Dancy back in 2007. They did a qualitative study of physics faculty looking at supporting them and what they found is for those faculty that weren’t supported, once they came across these, what they called “situation barriers,” when they were implementing a practice, that made them stop using the practice. And so we think that our results really back up what Henderson and Dancy found and the need to support faculty once they start using a practice, helping them understand what barriers are going to be when they implement that practice and then supporting them throughout the time that they’re implementing. Because otherwise, if they’re not aware of the barriers that they’re going to face, then they may stop using that practice altogether. So that was one of the tentative results that came out of this pilot study was showing us… demonstrating the need to support those faculty.

Rebecca: I was also going to say that a lot of times faculty don’t give themselves a break. The first time you do something, you’re not perfect at it, just like our students, they’re not perfect at it the first time. You have to practice and do it over and over again to get good at it. So I think reminding faculty when they’re doing something new that will also happen for them, doesn’t hurt. [LAUGHTER]

Hannah: Exactly. There was a study that came out recently, it was over five years of implementation. And the first year went horribly, and they adjusted. It wasn’t until like the third implementation that things started to go much better, student resistance started to go down, and just recognizing the first time you implement, there will be a lot of barriers… there will be a lot of problems and that’s okay… to keep going, that this is a normal thing.

Lindsay: I think that’s part of, really, the importance of this. Other people are struggling too. Helping to normalize the fact that when you try something new in the classroom, and it doesn’t go well, it’s par for the course and that other faculty are going through that as well.

Rebecca: Those are some interesting results, but not entirely surprising. I think those are some similar things that we’ve heard and seen in other research. But interesting that it’s at your specific institution from your specific faculty, and that the qualitative and quantitative pieces somewhat align. So what have you been doing with that data?

Lindsay: We have a few different programs that we are working on refining, aligning, expanding to what we’ve found systematically in these surveys with our faculty. Some of these include our Ignite program. Our Ignite program is something that we’ve been running with new faculty for the last few years. This is a program meant to support faculty as they implement a newly redesigned course. So these new faculty go through a week-long Course Design Institute with us and then they spend the next semester whenever they implement their new course, either in Fall or Spring, they meet biweekly with one of our faculty developers, and anywhere from five to 10 other new faculty in a learning community and they build on some of the things that they’ve been learning about course design and implementation. So they’re really getting that support throughout the semester. And one of the things that came out of our barriers survey was that the other work that we’ve been doing—particularly around these observations—is that the implementation is really important and that we really need to support faculty through that. We have some studies that, particularly around Ignite and new faculty, that demonstrate how important this learning community is, not just for the implementation, but the success of students. And so now we are expanding our Ignite program to all faculty, not just new faculty at our institution. We’re doing that for the first time this Fall semester. So that’s one of the programs that we have refined based on some of the data that we’ve been finding.

John: I think one of the benefits of that is if one of the barriers is departmental culture, that prevents people from trying new techniques, bringing in more senior faculty might break that down.

Lindsay: Yes, and one of the places that we’re beginning to expand to as well are learning communities, particularly for mid-career faculty. Many of our Ignite faculty are now moving into being tenured and so they are now becoming leaders in their departments and how do we foster and continue to help support them around teaching and learning?

Rebecca: Does your Ignite program come with course releases or does it come with time?

Lindsay: That’s a good question. We do not have course release at our institution, but they do receive a $1500 professional development fund, which helps support them in being able to continue to develop, they may be able to go to conferences, they do get supported in that way. Another one of our programs that we are developing and as Hannah mentioned one of the barriers are around class size and TAs. And so we have developed over the last few years a program called Spark. Spark is intended to be a program to support teaching assistants in the STEM departments. And over the last three years, we have had over 250 TAs enroll in our one-credit teaching methods course where they actually learn about different pedagogical techniques, learning theory, and they’re able to apply that every week as they are TAs in lab courses, discussion sessions, and even in co-instructor type roles. And that has been a really important piece to help support transformation in the STEM departments because our TAs are really the primary point person in many of our first- year courses and so providing them the support has been really transformative. One of the third things that we are doing in the center is around curriculum redesign. So one of the things that we found in the study that I think you alluded to was the differences between departments and the importance of the departmental culture and departmental support in helping faculty be able to utilize and implement evidence-based practices. And so we are actually working with departments to think about not just individual courses, but what is the curriculum look like for an actual major? What do we want our students to be able to know, value, and do at the end of four years—or five years—within different departments? And so we’re really working to develop this. This is something that we’re doing this year and really working to refine our programming around curriculum development and redesign.

Rebecca: One of the themes of all three programs is curriculum development. What are some things specifically that you’ve implemented or changed consistently to help with some of the issues that you’ve identified?

Lindsay: As part of the redesign process, we don’t necessarily recommend a single type of redesign or curriculum. We really strive to use evidence-based practices, whether that’s at the course level or curriculum level, to allow faculty to think about what best aligns with what they want students to be getting out of the course, or the curriculum. For example, if one of their learning objectives has to do with being able to collaborate and communicate, we might recommend some sort of collaborative learning design as implemented in their course. If they’re more interested in students engaging with the community, that might look a little different in terms of the actual design of the course. So I don’t know if that answers your question, but we don’t necessarily recommend one particular approach.

Rebecca: If faculty are resistant to evidence-based practices, and you were already introducing faculty to evidence-based practices in these programs, is there a different way that you’re presenting this information now to faculty to get them to buy in more to these practices, especially considering time concerns and student resistance and that kind of thing?

Lindsay: Interestingly enough, there are a handful of faculty that I think are resistant to the idea of active learning. The way that we’ve set up at least our Course Design Institute is in such a way that we attend to motivation first, and so we really get very little resistance to the idea of active learning or evidence-based practices. They want to do it. Some of them do do it. They either feel like they can’t do it as much as they want to or they do it and they’re not satisfied with their practice. We really don’t run up into the barrier of, “I don’t believe in active learning,” with the exception of a handful of faculty.

John: And there’s probably not much you can do with those. But I would think working with entire departments might help reduce some of the resistance because when you have that sort of collaboration with the department, it becomes part of the department culture, I would think. How has that been working?

Lindsay: We have had a cohort of faculty within a department go through our Course Design Institute and then another program that paralleled Ignite that was specifically for STEM faculty. And this department really has transformed, so this is about five years ago that they went through as a cohort. The department itself, the culture there is focused on teaching and learning, they continue to engage with our center, we have a recently started SoTL Scholars Program, so Scholarship of Teaching and Learning. We had five of the faculty from that department actually go through this together this past year. They’ve started their own reading group. We’ve been looking at data from the department and we see that student failure rates are going down in their department, particularly for underrepresented students. So working with the departments I think are really, really important and we’re seeing the fruits of that.

John: Earlier Hannah mentioned something about looking at issues of identity in terms of teachers and scholars and so forth. And I would think that perhaps the work you’re doing with SoTL might help unify that. Could you tell us a little bit more about the results you found and how you’ve been addressing those?

Hannah: What we expected to find was that there would be a correlation between teaching and research identities and that if you were high in teaching, you might be lower in research. If you were high in research, you might be lower in teaching. And what we found was that there was no correlation, that you could have both. You could be both an excellent teacher and researcher, you can be really strong in both of those identities, or you could not be. It was all over the place. And part of that is the sample size, and we have since expanded and haven’t analyzed that data yet, but we’re looking into that more.

Lindsay: And to add on to that, so the way that we looked at identity was the idea of how connected you feel with that particular profession. So if you feel connected to the teaching community versus feeling connected to the research community. And we also had a third aspect to that, which was the work identity… so how connected do they feel to the university? What we found was that faculty who had a strong work identity—meaning that they felt connected to the institution—they felt that the department was less of a barrier for implementing evidence-based practices, and they didn’t perceive that they had barriers related to supports. So things like having TAs, classroom space, and things like that.

John: Going back a little bit, you mentioned that one of the barriers that some faculty mentioned was the size of their classes. How have you helped faculty get past that?

Lindsay: We’ve actually had conflicting results around that. So faculty perceived class size as being a barrier to implementing evidence-based practices. But when we look at the actual observations of those faculty teaching, we see that faculty who have engaged in our center use more evidence-based practices, even when controlling for class size. And so what we need to further investigate is how our center plays a role in reducing barriers for faculty. The sample size that we have with our survey results is much smaller, and we can’t really disaggregate. There is something interesting that has to do with class size, and we’re not exactly sure what it is. Whether it’s a perceived barrier or an actual barrier, we’re not quite sure. But I might guess it’s a perceived barrier because we do see more active learning even when classes are large. So faculty are able to do these things, but sometimes they may not think they can.

Rebecca: Or they might not know what practices work at a large scale, because there’s different ways to implement… and so the more we expose them…

Hannah: Exactly. Yeah, because they’re trying to use approaches that require a studio. You can’t do that with a 500-student lecture. So obviously, that particular evidence-based practice is not going to be useful in that case. You can bring in some of these perhaps smaller practices but that are still powerful to get students actively working and collaborating with one another. Think-pair-shares, things like that, that you can still do and then there’s all sorts of work—great work—that’s going on now talking about what you can do with large classes.

Lindsay: And those are the things that we talk about in our Course Design Institute. How do you design your course, knowing that you have particular limitations because of things like class size? Or maybe it’s a required entry-level course, or maybe it’s an upper-level course, or a graduate course. All of those things are really important in thinking about the design.

Rebecca: Or the chairs don’t move.

Lindsay: But we do talk with them about how to deal with that. So in the lecture hall that the seats are fixed and you want to do group work, we have recommended to faculty—if they have space—leave every third row empty, and that way you can actually access students and students can turn backwards to work with people behind that. So we definitely try to help them think about ways to go beyond what they think are perceived barriers.

Rebecca: How to hack your classroom 101.

Lindsay: How to hack your classroom, I like it.

John: And actually, let me put a plug in for one of the Teach Better podcast episodes, which came out in April on the importance of classroom design. We’ll include a link to that in the show notes. The research they were citing finds that active learning helps, but classroom design helps even controlling for the use of active learning. So some of that flexibility is useful. This has been implemented in STEM fields—I think many of those topics that you found would work in other disciplines. Has the teaching center more broadly started to roll out some of these techniques throughout the institution?

Lindsay: I’m going to answer this from a much more broad perspective, thinking about what we’re doing in terms of our programming and supporting faculty. And I think Hannah can talk about the more specific piece around what we’re doing to better gather data around faculty barriers beyond our STEM faculty. So one of the biggest things that I think I mentioned at the beginning that we really are striving to do is use our own local data in addition to the literature to really drive what we do. For us, this goes beyond just doing a needs assessment. This is really doing research around teaching and learning at our institution. One of the pieces of evidence that we found around our prior work is the ways that students engage with each other in class, and how the instructor sets up that group work in class is really important to student success. And so what we are now doing this past year, we are collecting data to better understand not just faculty perceptions of how they design group work, how they assign students to groups, what do they do to assess group work, but we’re also looking at the student perspective. We are actually following students that are working in groups over time, having them reflect on their practice, share audio files and share working documents, to better understand what’s going on in group work. All of that data now we’re using to develop a advanced collaborative institute for faculty that’s going to use not just the literature that’s already published around group work, but also locally derived data that’s both STEM and non-STEM faculty in classrooms. And it’s been interesting because we think about our disciplines being very distinct in terms of  “Oh well,  STEM classrooms are very special, and they need to do these particular things.” As we’ve interviewed faculty, the reasons why they use group work—regardless of their discipline—is very similar. They want students to develop professional skills. I think it’s really important to gather that data to understand this perspective so that when we develop these programs and supports for faculty, we can actually talk about what the faculty are saying and how we use that to improve. So that’s just one example of how we’re broadening this idea of data-driven faculty professional development.

Rebecca: How are you gathering that data about group work?

Lindsay: In our center, I am 50 percent research and assessment and so a lot of my work is around being able to assess our programs, but also be able to gather the data to drive programming. As we said in the beginning, my PhD is in science education. So this is my formal training, being able to do this type of work. So I actually have a group of three graduate students—as well as Hannah and another postdoc—that helps support the research and assessment and center. So for example, as part of that group work study, I had one graduate student who over the course of two weeks, interviewed 19 faculty and over 1000 minutes of interviews that had to be transcribed. I really have a committed group of graduate students and postdocs that help support this work, because they’re really interested in helping make the improvements as well. I don’t think if this was something that was very abstract and not related to helping improve instruction that we would have such buy-in from the people that are helping support this work. So we’re doing interviews with faculty, students are submitting reflections, audio files and documents. So those are the data sources we have right now. We also have syllabi and course documents that the faculty have developed that articulate how they are setting up these group work or group projects.

John: That’s a great resource, I think, for all teaching centers because most of us don’t do that, and it’s nice to see this sort of research. We often talk to faculty about the importance of doing SoTL research, how the classes are working, but teaching centers don’t always do quite as much assessment of how their programs work, and how things are working on their own campuses in this way. So it’s a nice example, I think.

Hannah: Right, and I can talk to the real specific research that we’re trying to do to expand from STEM into non-STEM fields to kind of get more of that research across the university going. So the survey that I developed that has the barriers, that has the identity, that has some qualitative background questions to try to understand where their beliefs come from, all of that. I have been working with STEM faculty and non-STEM faculty now, to expand into the humanities, the arts, the social sciences. And what we’ve been doing is working with humanities faculty at the Center and then I had a focus group this week with several scholars in those areas to talk about the language that we use in the survey. So what I quickly found when we were trying to expand the survey across the university, is that the language that you use is really important. Now STEM faculty, they are fine with the use of the term evidence-based practices. And discipline-based faculty and researchers, we want to see the evidence. We want to know if something works, we want to know that there was a rigorous study that backs up that particular practice, and once we see that, we’re ready to kind of go for it. But when you try to expand that wording into the humanities, that’s not so much a crucial thing for them, they’re wanting to see that things work. The type of research that they do is very different and when we use the term evidence-based practices, the way that they think about that is very different from STEM faculty. So we had to change the wording, we’re modifying the survey, how the questions are asked, the types of words that we use, the assumptions that we’re making. So that’s been my job the past few weeks and will continue because it’s been proven it can be quite challenging to make sure that we’re not alienating a lot of the people that are taking the survey to the point where they see certain words and are like, “This doesn’t apply to me, I don’t want to take the survey anymore.” So that’s been the challenge with this, expanding this from STEM, is the language can be a barrier to people taking the survey and then we don’t get the data that we need. I’ve been working to figure out, “How do we talk about this in a way that we can compare across all of these groups, but still get useful data and not alienate groups within those different departments.”

Rebecca: I think sharing a summary of that information would actually be useful for a lot of centers and researchers too because teaching and learning centers probably also suffer from their advertisements and stuff, perhaps alienating groups of people and not realizing it for the same reason, potentially.

Hannah: Definitely, definitely. And one of the humanities faculty members here at the center and I have been talking about that and may be coming out with a paper once we gather more data on this, on the language that we use. What is useful and what is not useful by discipline?

John: That’s something I wouldn’t have thought of, because we use a lot of evidence-based practices here all the time.

Hannah: Yeah, I didn’t think of it either, and so I was in for quite the shock when I started talking with humanities faculty.

Lindsay: And I think another thing to add in terms of how we’re broadening this work, one of the places that I’ve begun to explore is how do we set up infrastructure at our institution so that we can actually systematically gather data, connect data sources, and then help faculty use that individually to improve instruction. It doesn’t do anybody any good if we gather evidence or research—we do research on our own—and then we don’t do anything with it. And so, we’re developing as I mentioned, our SoTL Scholars program so we can help faculty learn how to do this research on their own. So we are developing a set of tools that we can use to, for example, go out and observe faculty teaching in their classrooms and then from that data, create some sort of visualization that can be used in a consultation. We have a consultation program—many institutions in our centers have consultation programs—but what we really want to begin to do is gather that data in a way that we can begin to represent it on some sort of timeline, where the faculty can see, “Okay, the first 15 minutes of class we did lecture, I asked a few questions here and there, students didn’t answer those questions,” or, “I answered them myself or I moved on too quickly,” and so really honing in on some of those small details that can really help them make tweaks and improvements to their own instruction. So we’re really working at that infrastructural-level now to think about how do we create these tools and set up databases so that we can gather data and share that with faculty.

Rebecca: A follow up question to the qualitative research that you did at the very beginning… What kind of observations you were making for that qualitative research and what you were focused on? What you were looking for specifically…

Lindsay: Good questions. So the original observation study that we did a few years ago, we ended up using COPUS, the Classroom Observation Protocol for Undergraduate STEM. If you’re not familiar with that, COPUS measures the presence or absence of various different types of student and instructor behaviors over two-minute time increments. I was able to train 35 undergrads on how to use COPUS reliably and we were able to gather… for each individual course we observed twice. And we were able to then calculate the percent of time the instructor spent lecturing, or spent doing quicker questions, group work, administrative tasks. And we were recently co-authors on a science publication where the COPUS data were then transformed into profiles and so we were able to then categorize these different classes as primarily lecture—which was greater than 80-percent lecture using COPUS—interactive lecture—which was lecture but it had some clicker questions or some other group work interspersed throughout— and then the third set of categories was around student-centered instruction, so it could be POGIL type classes—so Process Oriented Guided Inquiry Learning type classes—or primarily group work, working on worksheets, doing problem solving, or a variety of different group activities. And so of those, we had 239 classes that we observed. Of those, we were able to classify those classes into those three categories—lecture, interactive lecture, and student-centered—and then we took those classes and organized them based on the intervention that the faculty have gone through. So whether they’ve engaged in our Course Design Institute, whether they’ve done our Ignite program, and we actually had a fair amount of faculty that we observed that have never engaged in our center at all. And so that’s where we were beginning to see differences… that our Ignite faculty, we saw much more student-centered instruction than faculty who had never engaged our center. We also gathered grade data on those classes. Do you want to know about that?

Rebecca: Yeah.

John: Sure.

Lindsay: This is actually a paper that’s currently in review, but the grade data was the thing that was really interesting to me. What we ended up doing is we calculated a DFW rate. That’s D, F, and withdrawal. So basically, failure rate for students in those classes that we observed, those 239 classes. We also were able to calculate failure rates for underrepresented minority students. So those were black, African-American students, and Hispanic students combined together compared to white students in the class. And even when we gathered observations of 239 courses, when you started to look at the courses taught by faculty at the different types of interventions—so that was Ignite, Course Design Institute—and then when you broke it down even further by, “Let’s look at those courses taught by Ignite faculty that did active learning, or lecture, or interactive lecture,” the numbers got very small very quickly. But one of the most interesting pieces that we found descriptively was, when you looked at just courses that were categorized as having student-centered instruction—so active learning, group work, those types of things—the faculty that have gone through things like our Ignite program, and another program called Nucleus—which is similar for STEM faculty—the failure rates between white students and underrepresented students were nonexistent. When you looked at student-centered courses where the instructors had not gone through our Course Design Institute or gone through any of our communities, the failure rates for underrepresented minority students were four times that of white students. Now this is descriptive, this is not anything that’s inferential, but that was one of the driving forces for me that made me realize that we need to look more at group work and what was going on in group work because it’s suggested that when you implement group work or student-centered instruction in your courses and you’re not supported in doing so, you are doing a disservice to your students, and that seems to differentially impact underrepresented students more so than white students. And that was really disturbing to me that we saw those differences on average. This was not the max, this was a mean value. And so that was so important for us to further explore, and we would not have known not had we not done such a large-scale study, and had we not used our own data.

Rebecca: That’s really interesting.

Lindsay: Thank you.

John: You’re making a big difference there, clearly.

Lindsay: We are, and it’s so exciting.

Rebecca: Yeah, I think sometimes we don’t always realize those other kinds of impacts. Or that there could be a difference in the kind of impact that one makes. So I think that’s a really interesting initial discovery to explore, so I’m really interested to see what else you find out.

Lindsay: So we wouldn’t have been able to make those findings had we not been able to connect to institutional data, and so that’s another reason why this developing infrastructure is so, so important, that we’re not going to be able to find meaning if we’re not connecting all of the pieces.

Rebecca: I think one of the things that is really interesting is that you’ve been able to do such robust research at your own institution and have the support to do that. Even how you structured that and how you’ve gathered that would be of interest to many other centers, I think. Sometimes the details of how you arrange that and organize it and how one thing led to another can help other organizations do something similar.

Lindsay: Thank you. I will put a plug in. So in terms of helping other centers be able to do this type of systematic research assessment work, we had a PODLive! webinar on Friday, April 26. If you’re a POD member, you should be able to access this through their website to see what we talked about and what questions we ask ourselves as we go through the process of thinking through measuring impact.

Rebecca: Great. We will make sure we link to that in the show notes and let people know how to access that.

John: We always end with the question, what are you doing next? You’ve already described some things, but we’ll still ask anyway.

Lindsay: So if you can’t tell already, I’m really passionate about data… using data to help drive what we do to improve teaching and learning. And so the two sort of big things that are next for me are really trying to build the infrastructure so that we can liberate data and be able to use data meaningfully, respectfully, and purposefully to help improve instruction. And also being able to help empower our faculty to be able to do research on teaching and learning in their classrooms… so trying to expand our SoTL Scholars Program, and developing further supports in that area. So that’s what’s next for me.

Hannah: And for me, I am working on a couple projects related to the barriers work. So we talked earlier about the humanities expansion, so developing a survey instrument that can be given across departments. So I’m continuing to work on that, work on the language that we’re using, making it relevant to them. And then we’ve got a national study that we’re trying to work on. So we have implemented the pilot—which is what we talked about today, the results of that—and then we implemented a second one also at UVA, but much larger. And then we’re wanting to now expand this and do a national study because the real beauty of this instrument is that it’s not just for us at UVA, it is meant to be a tool for any university, any department to be able to use. And one of the findings that came out of our study was that the barriers are different by department. The barriers, the use of evidence-based practices differs by department… it’s not just the university being different from another university. It’s the department being different from another department at a different university. And so this tool allows any department, any university, to give this to their faculty and see contextually, what are the barriers for these faculty? Now you look across the board, time is usually the highest barrier, but what comes after that differs by department. If there’s particular issues with one department, one university with the teaching-research balance at a given university, all of that’s going to be different. And so the beauty of this instrument is let’s look at a variety of types of universities, types of departments, let’s try to understand what is useful, what are supports, what are barriers across different institutions, across different departments. Try to look for where are there trends and where are there not trends. Where is it just entirely dependent on a given context and where do we see maybe some trends in tenure-track faculty versus non-tenure-track faculty, general faculty, things like that. So we’re really hoping to dig into a much larger sample in the coming year and investigate this further, and I will say that there are a couple of other researchers who are also working on this. So this is an up-and-coming area of research that you’ve got Megan Bathgate at Yale, you’ve got Emily Walter at Cal State Fresno, they’re both doing studies along this idea of barriers and supports for faculty using evidence-based practices. So, I just wanted to put a plug in that we’re not the only researchers doing this. There’s a lot of great work that’s going on and I think this is an up-and-coming area to really help support moving higher education forward and transforming higher education, ultimately, by understanding how can we help our faculty implement more of these practices that we know are going to support our students better?

Rebecca: Great, sounds like a lot of exciting things coming down the road for us to take in soon.

Hannah: Definitely.

Lindsay: Yes.

John: Thank you for joining us. This was a really interesting discussion, and I think many of us will reflect on it in our teaching centers.

Rebecca: Yeah, thank you so much.

Lindsay: Well, thank you, appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

83. ACUE

Faculty are often excited after attending professional development workshops and plan to implement new techniques, but often don’t follow through. In this episode Dr. Penny MacCormack joins us to talk about one program that provides scaffolding and structure to help faculty improve their teaching using evidence-based practices.

Penny is the Chief Academic Officer of the Association of College and University Educators (ACUE). Before joining ACUE, Penny had served as the Chief Academic Officer for the New Jersey State Department of Education and as an adjunct professor at Southern Connecticut State University, and Montclair State University. She began her career in education as a science teacher.

Show Notes

Transcript

Rebecca: Faculty are often excited after attending professional development workshops and plan to implement new techniques, but often don’t follow through. In this episode we talk about one program that provides scaffolding and structure to help faculty improve their teaching using evidence-based practices.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Penny MacCormack, the Chief Academic Officer of the Association of College and University Educators, or ACUE. Before joining ACUE, Penny had served as the Chief Academic Officer for the New Jersey State Department of Education and as an adjunct professor at Southern Connecticut State University, and Montclair State University. She began her career in education as a science teacher. Welcome, Penny.

Penny: Thank you. I’m happy to be here.

Rebecca: Today’s teas are…

Penny: Green tea.

John: I have Bing Cherry Black tea.

Rebecca: And I have Lady Grey.

John: We’ve invited you here to join us to discuss ACUE’s effective practice framework and the associated professional development program. How did this program come about?

Penny: So I think, like many ideas, initially with a conversation among leaders in higher education, some very respected leaders, talking about some of the challenges and changes happening in higher ed. An increasingly diverse student body, certainly more attention being paid to retention and graduation rates, and increasing contingent faculty, as well as the public starting to question the quality and the value of a degree in higher ed. And as we looked at the student success agenda, with many strategies that made good sense, really paying attention to maybe more nuanced financial supports, guided pathways with better advisement, data analytics, instructional supports, et cetera. We felt that there was a missing element and we felt like that element was more foundational than just one of the strategies that folks should be thinking of. For example, guided pathways or advisement make really good sense to us…that a student would have a clear path to a meaningful degree. But what we thought attention needed to be paid to was the quality of instruction in those courses along the pathway, and then across an entire institution, the quality of teaching. And we were very aware of the fact that faculty—including contingent faculty—are experts in their discipline, in their subject area, and they’re experts in the research processes. But most have little—sometimes no—training in evidence-based teaching practices in teaching. So we felt like that missing foundation needed to be addressed and set about to develop a comprehensive…we wanted something that would give folks a foundational base of the evidence-based teaching practices we know to be effective in the college classroom. So we wanted to be comprehensive, we wanted it to be research based, we wanted it to be high quality, and we wanted to be scalable. Recognizing that while it’s important for small groups of instructors to become better teachers, the reality is, all of our students, and all of our faculty deserve to be interacting with the evidence-based teaching practices we know actually improve engagement and deepen learning. So we set about to do that.

Rebecca: It’s a pretty big undertaking. It sounds like you probably had a lot of people involved in that process. Can you talk a little bit about how did the design of the program happen and who was involved?

Penny: So you’ll notice here one of the things I said was comprehensive, that we wanted faculty to gain a foundation in evidence-based practices. And so we needed to identify, what are the core set of knowledge and skills you need to be effective in the college classroom? And to be very honest, we had hoped perhaps that already existed somewhere. [LAUGHTER] But lo and behold, that was not the case. And so we reached out to scholars in teaching and learning across the country and worked with them, did a deep dive into the literature, and worked through an iterative process to identify that core set of knowledge and skills. And once we had that, we also worked with the American Council on Education, to endorse our courses and our framework. And they brought to bear their own set of experts across the country in the Scholarship of Teaching and Learning to review the framework. And then eventually, ACE endorsed the framework and so we feel pretty confident at this point through the processes we used and ACE used to say that our framework and effective practice does outline the core set of knowledge and skills you need to be effective in the college classroom. So in that case, the folks who really informed that work are the experts in the scholarship of teaching and learning across the country, folks like Linda Nilson, Tom Angelo, Elizabeth Barkley, Saundra McGuire, really making sure again, to involve those folks that teaching centers across the country know really have done the majority of scholarship in that area.

Rebecca: Of course, once you came up with the framework and that comprehensive knowledge, you had to figure out how to deliver it. Can you talk a little bit about how that decision was made?

Penny: Absolutely. You point out something that is quite important. It’s one thing to develop a list, right? “Here’s the core set of knowledge and skills.” It’s yet another thing to do that those other three describers, right? Research based—that was kind of easy, because the list was research based—but high quality. And for me, when I’m talking with folks, high quality really means that faculty will love it. Because if faculty are not going to be engaged in this course and engaged enough to actually change the practices that they’re using in the classroom, then we’re not going to realize that student level impact that is our mission. So in order to design the course now—to your point, got to do that part—we did a couple of things. So one, we paid a lot of attention to the research on how people learn, how does the brain work, and specifically, how do adults learn. The course needed to be scalable. It needed to be offered online, so a lot of attention to online practices. But then we did something really important. And that was to talk to faculty focus groups across the country and do a couple of things. One, put some materials in front of them. Some questions, some video, some text, and ask them to critique, which they did happily, because faculty are quite good at critiquing. [LAUGHTER] The second thing we did was we asked them, “What would you need to consider changing the practices you use in the classroom?” And so they were crystal clear. One, they wanted to see those evidence-based practices in action, in authentic classrooms, by their peers…peers teaching…people that they could see would be instructors in the classroom. Two, they wanted to hear from those instructors why they were using those practices. Icing on the cake would be to hear from students as well, how those practices were working for them. Three, they wanted to hear from researchers. They wanted to hear from the folks who demonstrated that these practices are effective in the classroom. Makes sense, they’re higher ed folks, they want to hear from the folks that did the research. And four, they wanted opportunities to learn, discuss with their colleagues as they were learning, to learn with and from their colleagues. And so just as we paid attention to the research on how people learn, how adults learn, online practices, we paid really careful attention to what faculty asked for, and we delivered it. We made sure that those four things that I heard over and over and over again—from faculty across the country—we delivered on. We listened to them.

John: Maybe it would help if you sketch out the process of a typical module, because it incorporates all those things. And we’re new to ACUE, but our faculty so far have really been enjoying it and they really appreciate the design of the program. But it might help for our listeners who aren’t as familiar to know how a typical module is structured.

Penny: I’m happy to discuss the learning design because we spend a lot of time and a lot of attention to it. Each module includes 12 components. I can divide those 12 components into four groups of three. So the first three components are really designed to pique somebody’s interest and to activate prior knowledge. So we show an introduction video, where that includes clips from our classroom demonstration, kind of like how 60 Minutes gets you interested in the rest of the show, we’re showing little clips to get folks interested in the topic. We outline very clearly the learning objectives and the rationale for the module, so we connect the practices that they’re going to learn to the research that demonstrates it does impact students, and then we offer a group of questions to activate that prior knowledge because what we know about that is if you activate prior knowledge, you’re more ready for new knowledge. So that’s the first three components. The second three are designed to build that foundational knowledge. We decided to show before tell first. And so we have a classroom demonstration video, where you see faculty utilizing the evidence-based practices being recommended in that module. You hear from those faculty why they’re using those practices and you hear from students about how those practices are impacting their learning. Next component, you hear from the researchers about the research behind that component. We actually utilize speed drawing there, so that it’s not just a talking head, but there’s a little bit more interaction going on and then finally, we offer resources to faculty so that when they implement any one of the practices that they’ve just seen in that classroom demo, they have all the resources they would need to implement. The next three components are about deepening learning, and allowing for that collaboration to happen with their colleagues. And so the first component is some text. We wanted faculty to read a little bit deeper about the practices and the way we do that is to address some of the common misconceptions, common challenges that faculty might think of, and we address those with the research. And so a common challenge or a common misconception will include a couple of paragraphs from the research about why that’s a challenge and how to overcome it or why that misconception exists in the information that kind of helps you see it differently. We follow that by two sections of what we call observe and analyze. Up to this point in any module, faculty would be able to do all of those components on their own online when it’s most convenient for them. With the observe and analyze, oftentimes faculty will schedule a particular day that they’re all going to engage in watching these videos, and the videos are of what I call developing practice. So you’ll remember that faculty would have seen effective practice, they would have heard from the researchers, but now we show them developing practice—somebody doing some things well and some things that could be adjusted some—and that is the conversation that faculty have. So they watch this video, and then they engage in an online conversation—some of our partners will sometimes bring folks together face to face—but they engage in a rich conversation about what that person is doing well, and what they might adjust or tweak.

John: We should note that no actual students were harmed during these demonstration component videos.

Penny: Absolutely. As a matter of fact, during the demonstration videos where we were doing developing practice, students knew what we were doing, and it’s completely scripted. So I think what was interesting about students is they understood when a practice was really effective, because remember, it’s developing. So it’s not like a train wreck, it’s some things being done well, and some things that could be tweaked. And when you think about it, the faculty watching the video are in the same shoes as the person trying it for the first time. So they’re watching somebody try something for the first time making some mistakes but doing some things that are quite good. And they’re able, they have that opportunity, before they’re asked to implement one of those practices in their classroom so it’s a really rich learning opportunity that they get to do with their cohort to collaborate with their colleagues. The last set of components, faculty are asked to practice and reflect and then we do a closing video. So we indicate to faculty, “Here are the learning objectives for the module and here are the practices.” And there’s always between five to 10 practices offered in every single module. And we say to faculty, “Choose one,” and that’s important. In adult learning you don’t want to say, “This is the one thing you have to do and you have to do it now,” because faculty are teaching different classes, have different students that they’re working with, we want to give them a choice. So they choose one of those practices and they implement it in their classroom. And then what we require is they reflect on that experience in writing. And that written reflection is submitted to us to be scored. We do present to faculty a rubric for how we’re going to score that reflection. So those requirements are up front, we try to practice what we preach, as far as teaching and learning goes. Faculty submit the reflection, we have national readers that score it using the rubric, and if a faculty’s reflection isn’t quite up to our meets category, we get it back to them with specific feedback and they can resubmit. Now we finish every module with a closing summary—again, practicing what we preach, good teaching and learning—close with a summary of the learning objectives and some more commentary from the researchers.

John: A lot of our faculty have commented how they appreciate the fact that the course itself uses all the practices that are implemented—as you mentioned—and they really enjoy the skeletal outlines, they like the ability to go in and critique these demonstrations. And one of the things that we as working with our teaching center appreciate is that we’ve done workshops on many of these topics and some people have attended them two or three years in a row without actually implementing them. And what we really appreciate is the fact that now people have to get past that barrier of actually trying it in the classroom. And a lot of people who have been coming to our gatherings have said they did this for the course and now they’re doing it in every class. So it’s already making some big changes in people’s teaching practice. So it’s been working really well.

Rebecca: I think another real strength is the external reviewers is really important so that as teaching and learning center staff, we can support our colleagues and not feel like there’s some sort of punitive relationship where we’re judging.

Penny: Yeah, we are a learning organization and so actually when we first piloted a smaller number of the modules, we had the facilitators—our course facilitators, often folks from an institution’s teaching and learning center—scoring their reflections, and they were crystal clear with us that that didn’t feel right. And so we took that on, so that they could really be the coaches that we want them to be with the cohorts.

Rebecca: I think that works really well and I think that really encourages faculty to follow through and to do them and to actually take the actions in the classroom. So I think we really benefited from that particular feature.

Penny: Yeah. I know our mission has been to realize student outcomes— better retention, graduation rates, better learning— through quality instruction. And so in order to impact students, we knew faculty had to go beyond learning these evidence-based practices, but actually using them and so the requirement to complete a module became the implementing of one of the practices. And then what we know to be true in professional development is reflection is such a strong way to not only implement but actually to continue thinking about what went well, what didn’t go well, what might I refine, et cetera. That’s really putting you on the trajectory to becoming a better and better instructor.

Rebecca: I think one of the other interesting advantages of this particular online course is that a lot of our faculty may never have taken an online course but may be asked to teach online courses, so having the experience of a well designed online course is an important experience, especially as faculty move more and more into teaching online and having an idea of how to implement some of these practices, not just in face-to-face situations, but also in online or hybrid situations.

John: And we should also note that in each module, the options that people have could be either for a face-to-face class, or there’s a set of options for people who are teaching online, so it facilitates both types of instruction directly for people with different teaching schedules.

Penny: And we have actually even brought that to a more sophisticated level. So we will be offering our course in online essentials coming up in the next few months, where if we had a cohort of online instructors, they would be doing an observe and analyze about online instruction versus face-to-face so that they would really have that full experience of, “How do I do this core set of skills needed to be an effective instructor online?” So we’ve gone beyond just offering the online resources, to making sure we offer some real high quality learning experiences for them.

Rebecca: That’s great.

John: You mentioned the goal of improving instruction and improving all these outcomes. I know that there’s been some research that has been done at some campuses in terms of what sort of impact this has had. Could you tell us a little bit about what’s been found in terms of the effectiveness of this program in improving student outcomes?

Penny: Absolutely. We’re really, really proud of the work that we’ve done with regards to efficacy. And I think it’s important to recognize that when we partner with any institution, we partner to assist and support implementation. So when you partner with ACUE, we don’t say, “You can click on here and get to our courses, and good luck!” [LAUGHTER] When we partner, every institution has an academic director who will work with the campus lead—oftentimes the teaching and learning center folks as well—to design the course sequence and cadence and make sure that it makes sense for that particular group of faculty. And then in addition to assisting with implementation, we actually study efficacy. And we are very proud of multiple studies now demonstrating student impact. But I always like to indicate that the first set of data that we collected was around faculty, because as I was mentioning before, if faculty aren’t engaged with the course, faculty aren’t learning, and faculty aren’t changing their practices, then you have no hopes of seeing student impact. And we’re particularly proud of what we have with regards to faculty data across over 2,000 faculty members. Ninety-seven percent on average report that the course is relevant. On average, faculty report learning 55 new practices and learning more about 71. And then on average, faculty report implementing 28 new practices as they engage with the 25 modules and a plan to implement 28 more. So we’ve got that faculty data that says to us, “Hey, you know what, you’ll likely have student impact data,” because again, all of the practices in the course are evidence based, they’re already research based. And we’re, again, really proud to share some of the findings we have at Delta State, we have a study where we were able to show an increase in A’s, B’s and C’s, and a decrease in DFW’s. At Miami Dade College, we were able to show an all of these results are statistically significant. In fact, I invite anyone to go on our website, look at the impact page, if they’re particularly interested in the statistical analyses. At Miami Dade, we saw increased student engagement, comparing faculty to themselves before and after they engaged in the course as well as to a matched cohort. We saw an increase in grades. At Texas Women’s University we saw an elimination of course completion gap, a rracial course completion gap. And at Broward, we actually gave students surveys where they indicated that they had engaged regularly in evidence-based teaching practices. And we’ve got a number of studies currently going on so we have been able to show and realize the student level impact that you might expect as faculty start to regularly use evidence-based teaching practices. It’s really, pretty quite amazing.

John: How many schools have participated in this program?

Penny: So currently, we are partnering with over 100 colleges and universities across 37 states. And again, as we partner with any university, we work with them to design the course offering for that particular set of faculty at that particular institution.

John: We appreciated the fact that since we started in late January that the structure was able to accommodate teaching schedules of our faculty, so that people were doing things that were relevant at that portion of the year.

Penny: Yeah, I am particularly proud of the fact that this is not just some lockstep set of courses we ask you to follow, but rather thoughtfully sequenced, dependent on when faculty are starting to engage in the course, and we sequence in a way so that faculty pretty early on—as they implement in their classrooms—start to have some positive feedback from students because that itself is pretty motivating.

Rebecca: Yeah, I think one thing to point out is that we often think about when you teach someone how to teach, you start with the syllabus or you start at the beginning, and we started in the middle, because we were in the middle of the semester, and it made perfect sense for our faculty. I think that it was really effective and I think that the faculty really appreciated that they were able to do stuff right away and not plan things for a semester out.

Penny: Yeah, what we found essentially is as much as I love to think about learning outcomes, and aligning my assessments and aligning my activities, that’s not what everybody enjoys doing. And it’s best to put that towards the end of a sequence. So that faculty really can utilize practices that connect with their students, motivate their students, really embrace the diversity in their classroom, and have those kinds of interactions and then get to, “Okay, so how do I structure this? How do I write a learning outcome that really helps students learn more? How do I make sure my assessments are aligned,” et cetera. That’s work that’s best after they’ve had some of those other experiences.

John: And after the toolkits have been developed, so they have activities they can plug into those learning objectives.

Penny: I do think that when an institution feels like, “Gosh, we need to do something about courses,” they’ll often go to course design as their strategy and leave out the how the course is taught all together and just think the redesign is going to do it, but it really is the combination.

Rebecca: So we always wrap up by asking what’s next?

John: For either you or ACUE?

Penny: For both me and ACUE—I’m happy to say—as I described before, we’re a learning organization. So we are constantly listening to our partners, seeing what’s happening in higher ed where we think we might be able to have some positive impact. But one of the key areas—no surprise—is continuing education. So, we’re helping faculty have this strong foundation, but we know it takes a lifetime to become an effective instructor. And so we want to support faculty in continuing to build on that strong foundation. As well as looking at what are some other areas in higher education where we might be able to offer some courses and some learning that would assist with, again, realizing student success.

John: We’ve really enjoyed talking to you and we’re really enjoying the program here.

Rebecca: Yeah, thank you so much for joining us today.

Penny: I’m so happy that folks are enjoying the program. When we hear from faculty and we hear the kinds of appreciation and even as they talk about how their students are more engaged or learning at deeper levels, there’s simply nothing better than that, and so we’re excited to be working with you folks and with folks across the country.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

82. Geeky Pedagogy

When they were students, most faculty members were not the “average student.” They generally enjoyed learning and were willing to spend long hours independently studying topics that others may not care much about. In this episode, Dr. Jessamyn Neuhaus joins us to examine how geeks and nerds can successfully teach our more “normal” students.

Jessamyn is a professor in the history department at SUNY Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessamyn is the recipient of the State University of New York’s Chancellor’s Award for Teaching Excellence. She’s also the author of Geeky Pedagogy: A Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers, which is scheduled for release in September 2019.

Show Notes

Transcript

John: When they were students, most faculty members were not the “average student.” They generally enjoyed learning and were willing to spend long hours independently studying topics that others may not care much about. In this episode, we examine how geeks and nerds can successfully teach our more “normal” students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Jessamyn Neuhaus. Jessamyn is a professor in the history department at SUNY Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessamyn is the recipient of the State University of New York’s Chancellor’s Award for Teaching Excellence. She’s also the author of Geeky Pedagogy: A Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers, which is scheduled for release in September 2019. Welcome, Jessamyn.

Jessamyn: Thank you. Nice to be here.

John: Welcome. Our teas today are…

Jessamyn: I am drinking Lemon Zinger in my book nerd mug. [LAUGHTER]

Rebecca: That sounds perfect.

John: You’ll have to bring that to book signings too. [LAUGHTER]

Rebecca: I’m drinking Rose Garden today.

John: I’m drinking Twinings Enchanted Forest Fruits Black tea, which I picked up in Epcot last year.

Rebecca: Yeah, my Rose Garden’s from there too.

John: I was there for the OLC conference and you were there actually for a vacation.

Rebecca: Yep. [LAUGHTER]

Jessamyn: Enchanted garden sounds super nerdy. [LAUGHTER]

John: It is, and it tastes very good, too. We’ve invited you here to talk about your new book. What inspired you to write this?

Jessamyn: For as long as I could remember, I’ve loved to read and write and think. I’ve always been an introvert, I need a lot of solitude in order to function. I’ve always done well academically. My son, on the other hand, he’s an off-the-charts extrovert and a different student—let’s call it that, different—watching him grow up and go through the education system made me realize how differently he and I experienced school. And like I say in the acknowledgments section of my book, living with him and with his father—my significant other—is a master class in the difference between nerds and normals. [LAUGHTER] They are the normals, I’m the nerd. This was a big part of the inspiration for the book. Nobody was saying what I think is pretty self-evident: that people who earn advanced degrees—by and large—are pretty nerdy, which is as it should be—we’re the experts. Introverts are also disproportionately represented in academia, we take pretty easily to those long hours of isolated study that’s required to earn an advanced degree. Another inspiration was I really wanted to inspire teaching self-efficacy and helping faculty become effective teachers. And I use that word really, really deliberately—I understand why SoTL folks and professional developers use terms like best teachers, excellent teachers, even good teachers—but I think those terms really feed into some disempowering myths about teaching, myths like “good teachers are born, not made,” or myths like “only the most astounding super teachers affect student learning.” And those highly idealized impossible standards, I think, can really undermine teaching self-efficacy… feed into doubts and insecurities… So that was another inspiration. And similarly, throughout the book, I use us and we, when I’m talking about teaching, trying to create a sense of shared undertaking. Like we do in our classes when we talk about our class, our learning, our discussion, trying to help students become aware of their own responsibilities for their learning. And similarly, a lot of SoTL authors who I know want to invite readers to join the Teaching Commons, inadvertently undermining this goal by handing down these rigid dictates from above. “You should do this,” “Don’t forget to do that,” “You do this… that…” as if the person writing is not also in the teaching trenches trying to learn and relearn how to be an effective teacher. And I guess along with that, I also really wanted to bridge the major gulf between SoTL converts and faculty who are new to—or even resistant to— professional development. I see so much conversation about college teaching that is really divided along these two extreme positions. On one side: pro-student SoTL experts, they’re practically perfect, they never get frustrated by students… [LAUGHTER]… they’re 100% compassionate, and they’re totally on board with professional development. And on the other extreme, faculty who are totally burnt out, or completely cynical, and they’re always sniping at each other like, “You should be more compassionate to students.” “No, students are always terrible.” [LAUGHTER] And I want something in the middle saying “We can learn how to be effective teachers, we can be compassionate, we can be understanding, but also, sometimes students are irritating. It’s frustrating.” And I think finally, the most important inspiration for me was, I saw a need for a teaching book that strongly and repeatedly acknowledges the importance of our individual teaching context. And what works for one instructor just plain might not work for another. I mean, even what works for you in one class may not work in another class. And this isn’t like a brand new concept—it’s widely acknowledged—but I don’t think it’s acknowledged consistently enough. I think, especially for new instructors, I think you can read a lot of SoTL that seems to be suggesting, “If you just do this, you’ll be an effective teacher,” and that’s not nuanced enough.

Rebecca: I know that I was really excited to hear about your book, because it includes words like introvert and geeky in the title [LAUGHTER] and I identify that way. And I know that the first chapter in your book is on identity. Can you talk a little bit about why it’s important to think about identity and why that’s not often included in professional development?

Jessamyn: Sure. I’m so glad you asked that question, and the short answer to why I included it is because embodied identity is an important reality in human interactions. And I’m not sure why it’s not more fully acknowledged in professional development. It does seem like a lot of advice about teaching and scholarship on teaching and learning seems to imagine that we’re teaching in some sort of enchanted bubble that’s floating above the dreary workaday world, this wondrous place of true equality. There’s no racism or sexism and students and teachers are purely intellectual beings, and we gather—totally free of our biases—just to learn together every morning [LAUGHTER] the sky is full rainbows, and we skip down lollipop lane to another glorious day of tenure, but…

Rebecca: Right after we walk by the unicorns, right? [LAUGHTER]

Jessamyn: That’s right, yeah. When we enter the classroom, we don’t close the door, and presto change-o, there’s no race and ethnicity and gender expression, and speaking voice and physical abilities, sexual identity, they don’t exist. We bring all those assumptions, and stereotypes, and biases, and unconscious biases with us. But a lot of otherwise excellent scholarship on teaching and learning just does not fully acknowledge this. There’s a widespread assumption about what a professor looks like, and it’s a white guy, probably wearing a tweed jacket with elbow patches, and he’s lecturing so brilliantly in front of these mesmerized students that they learn without effort. And to be clear, I’m not suggesting that white guys don’t have to work hard to be effective teachers—they do—and I’m also not suggesting anyone who’s not a white guy can’t teach effectively—of course that’s not true—but any teaching advice is not going to apply in exactly the same way in every classroom and embodied identity is one—it’s only one—but it is one important aspect of our individual and unique teaching context. So, just to give an example, it’s pretty clear from the scholarship that effective teachers build rapport and demonstrate immediacy with students. But what I have to do to achieve that as a white gender-normative woman is different than what my white gender-normative male colleague would have to do. And it’s different from what all faculty of color have to do, especially because their expertise is not assumed in the same way. It will be challenged in a different way than many white faculty members. To take an even more specific example, I’ve seen teaching advice that talks about how professors need to be friendly and approachable, and that that would include smiling to students. But, you know that saying you should smile more means something different to women than it does to men, and we will hear it differently. It doesn’t mean we shouldn’t smile with students, but it means we have to utilize that scholarship in a different way. I think in every aspect of our world, white male privilege is often the default center of SoTL, and I would add maybe tenure too—this kind of default privileging—and I think we just have to start more regularly acknowledging that effective teaching and learning is shaped by embodied identity just like everything else we do as human beings. And one last point on this, it was very, very important to me that I not reinforce gendered and racialized stereotypes about geeks and nerds. Historically, those terms have been gendered male and raced as white. That’s changing, and you can see it in some contemporary popular representations. You can see it in people’s lived experiences. Those stereotypes aren’t gone—they still have an impact on people’s lives—and in fact, geek gatekeeping where white male geeks say, “You can’t play this, you can’t do that,” is still a factor, especially in fan cultures and gaming communities, it’s still a problem. But there’s so many of us nerds and geeks who fully embrace and celebrate Spock’s view of the universe, [LAUGHTER] as a place of infinite diversity in infinite combinations.

John: You also in that first chapter include a section called “Learning is Hard” and that’s probably a useful thing for faculty to remember because many of us have either found it to be easy along the way or it was so long ago that we were at that position that we’ve forgotten how difficult it is to learn new things. Could you address that just a little bit?

Jessamyn: That section draws on all the great science out there about the brain and learning. And you’re absolutely right. It’s especially important for us nerdy experts because we know our topic so well—our brains are so skilled and practiced at it— that’s the expert blind spot that gets in our way. It’s something that we do so automatically, it’s hard to remember what it’s like for a new learner. It’s hard to remember how long it takes to learn. It’s hard to remember how emotional learning can be, especially when—like I was saying with my son—we’ve had really different academic experiences than most people. Even our most brilliant students at our most elite institutions, most of them don’t take to academia the way we did. Most people want to be done with school. We said, “No, I like school so much, I’m going to stay in school forever.” So the science of learning is an important way for us to keep reminding ourselves that learning is hard and we have to do it over and over… and that does apply to us too. One of my main points in this book is that learning how to be an effective teacher never stops. We are always learning and relearning because students change, we change, curriculum changes, we’re always having to relearn. And yet faculty will often throw up their hands at the first obstacle they get to and say, “I’m not a good teacher.” I’ve lectured my students, I don’t know how many times on,“You got to have a growth mindset. Don’t tell yourself you’re bad at something.” But then when I was trying to teach myself how to do Twitter, after two weeks I was like, “Oh, I’m terrible at this. Everybody else is so good. I just can’t do it.” It’s really hard to learn, and it’s easy to forget in our subjects because we’re so skilled at them.

Rebecca: Can you talk a little bit about what you mean by geeky pedagogy in general?

Jessamyn: Sure. So this is the first college teaching guide addressed to geeks, introverts, and nerds, which is pretty amazing because, damn. Like, look around you, pal. [LAUGHTER] I mean, academia is jam packed with us but it’s more than a gimmick. So at the heart of my argument is that geeks, introverts, and nerds as a group—and I’m generalizing, this isn’t every single person—but as a group, we face certain obstacles to effective teaching and learning. Obstacles like effective communication, building rapport, productive professional and social interaction. We’re highly successful academically and most of us who have not taught much before somehow believe that that’s going to magically translate into helping other people be successful academically, but it doesn’t, it’s not necessarily. That being said though, geeks, introverts, and nerds as a group bring important and necessary skills to learning and relearning how to teach effectively. We’re passionate about our subjects and we can draw on that passion to create what I call a geek culture of sharing pedagogy. So that is creating classrooms and instruction that invites everyone into the study of whatever crazy, arcane, esoteric topic we love with all our dorky hearts. Instead of acting as a geek gatekeeper… keep people out, prove how smart we are… geek culture of sharing says, “Come in! This cool thing we are learning about… I can’t wait to share with you.”

John: You have a chapter in your book on preparing for class, could you tell us some of the things you focus on in there?

Jessamyn: Okay, so my first recommendation for preparation is you’re going to read Geeky Pedagogy from cover to cover. [LAUGHTER] You’re going to follow me on Twitter @geekypedagogy. You’re going to visit my website, geekypedagogy.com, going live July 1st. But seriously, the first thing I would emphasize—my top recommendation for preparation—the thing I most want to share with my tribe of nerdy eggheads—is to think about teaching effectively as an intellectual activity. We have to use our big fat brains for effective teaching. All those geeky study skills we have, we have to apply it to teaching preparation. So, we have to do some research on pedagogical content knowledge, how to teach your subject. We have to think carefully about our syllabi, and prepare them in a timely way—do not procrastinate. That’s my one nag, [LAUGHTER] don’t procrastinate—we have to think through our experts’ blind spots, we have to read up on the science of the brain, how people learn, and we have to take into account that over and over again in our individual and unique teaching context, we’re going to be learning and relearning about effective teaching. Now in the book, it’s a narrative guide. So I don’t offer extensive checklists or step-by-step… do this do that, and then, “Tada! You’re an effective teacher.” Those can be helpful at times, I think maybe especially if it’s your very first class if you’re really, really nervous, but I want faculty drawing on—a lot of this may be specific advice I have in the book—but I want faculty to feel empowered to research their own specific teaching context to become what Stephen Brookfield called experts on our own teaching. Keep figuring out what worked, what didn’t work, reflect on it, repeat.

John: One of the things you mentioned is preparing for confrontation and conflict. What advice do you give faculty? or what types of confrontation and conflict do you address?

Jessamyn: We were just talking about the emotional aspects of teaching and learning. Teaching and learning include a lot of extreme emotions. I’m not the first. Stephen Brookfield, Maryellen Weimer both mentioned that teaching is a roller coaster, there’s a lot of ups and downs. Same is true for learning, there’s a lot of ups and downs. For different kinds of assessment and creating there can be conflict. Student incivility, which is a very polite term for things that sometimes are pretty egregious, it can happen. And this is all—it’s a tiny part of teaching—but it happens and it sucks up a disproportionate amount of our time and energy. Another example might be plagiarism. One issue I talked about is academic entitlement, a new and growing issue. The most important thing—and this is across all teaching contexts, even taking into account what Roxanna Harlow called disparate teaching reality—so even taking those into account, preparation is the one thing that can help mitigate any kind of conflict. Clearly conveying and communicating—not easy for introverts and nerds—but clearly conveying your expectations, being as transparent as possible, that’s the number one thing, preparing for those. And then the other thing I guess in that section, the most important point is that understanding as introverts… and not every introvert and nerd is socially awkward…. I am. I’m definitely socially awkward, smarty pants, that’s my persona. [LAUGHTER] Understanding that, for me, high-levels of emotion are hard to handle. And it’s definitely not easy for anybody in any workplace to deal with conflict and anger and strong emotion. But I think it can be especially daunting for introverts, for people who aren’t extra socially skilled. It’s hard to be right there in the face of extreme student emotion. Preparing for it with some scripts in your mind, not like endlessly rehearsing, “I’m going to say this to so-and-so,” but just having a kind of standard for plagiarism, this is kind of my standard script. For someone upset about their grade, this is some steps I do. Being as mindful and as present as possible in that moment tends to help as well, which is actually a strength for introverts is listening. So the research seems to be suggesting that the best way to defuse any kind of student conflict is for students to feel like they’ve been listened to, and that seems to matter more than what you actually do to resolve the situation. So there was one study that showed, for example, students might view the offer to do makeup work for something as either a positive or a negative resolution, fully depending on if they felt like the professor had been listening to their concerns. That’s great news for introverts and nerds, because it means it has everything to do with our communication skills, which we can do.

Rebecca: You mentioned reflective practice a little bit ago, can you talk a little bit about some techniques or ways that we can build that into our practice and then actually use the time that we reflect effectively?

Jessamyn: Right. Well, I’m glad you mentioned, those are two very different things, and neither one of them is easy but the second one is definitely harder. So applying the knowledge you’ve gained from reflection to your actual classroom practice is a lot harder. The number one thing to do and not do is don’t limit your pedagogical reflection to student evaluations. Too often, that’s the only feedback, and the only reflection faculty do about their teaching, and it’s insufficient. It can tell us some important things, but it’s not adequate on its own. I have some specific suggestions in the book. Things as simple as keeping ongoing notes throughout the term on your syllabus about things that are working or not working well. But I would say generally, I really want to encourage people to find reflective practices that engage you as part of your actual work of teaching and make sense for you. I was really aware that for some of us, mindfulness practices, yoga, we’re onboard. And then for some of us, even anything slightly new agey is not going to work at all. So my main recommendation is find reflective practices that help you reflect on what’s working, what’s not working, and then apply to your practice. An important part of that is thinking about reflection as something we do as individuals, but also part of a community of practice. We have to talk to other people about teaching. And sorry, introverts, you can’t do it just on your own. Academia doesn’t encourage it at all, you’re often going against the grain to try to talk about teaching, especially if you want to talk about our teaching mistakes—which are the most important way we learn just like our students—it’s the most important way we learn. But there’s so few opportunities, we have to really go out of our way to make those opportunities happen. And the one last plug I put in is for adding a gratitude practice to pedagogical reflection. And here I’m drawing on Kerry Howells’ book. It’s called Gratitude and Education: A Radical View. I want to emphasize this is not just positive thinking, it doesn’t mean ignoring the toxic aspects of your workplace—not that academia has any toxic aspects whatsoever, right? [LAUGHTER] Or injustice, or inequality, or anything going wrong—that’s not what gratitude practice means. It does mean being fully aware of and paying attention to every aspect of your teaching context. And Howells argues that our teaching context in her gift paradigm of education… as opposed to the consumerization model. In the gift paradigm, our teaching context always includes gifts, things we get no strings attached from students, from colleagues, and staff. I know people listening might be thinking, “Gifts? Give me a break. Like, I get jack squat every day from my frustrating students.” But I would counter with—pardon me while I super nerd out the wise words of Thorin Oakenshield from the Hobbit—he said, “There is nothing like looking if you want to find something, you certainly usually find something if you look, but it’s not always quite this something you were after.” So gratitude practice by opening up our view of our teaching context, we will find a gift.

John: Early in the book, you start off with, “Learning is hard,” and at the end, you conclude with, “Teaching is hard.” Could you tell us a little bit about that?

Jessamyn: Sure. So the last chapter is the shortest one. It’s called “Practice.” It’s the best news and the worst news you’re ever going to hear about teaching, is that you can get better with practice. So that’s great news, because it means we’re always learning how to do it better. It’s bad news because nothing can replace it. Fellow bookworms, there’s nothing you can read that will replace it. And if you don’t have employment security—like most of us teaching college, the majority of us teaching college are doing so on a contingent basis—if you don’t have employment security, that’s hard news. And it’s a vicious irony, that teaching effectively is so key to our employment and yet, the thing that will help us the most—being able to do it over and over—is dependent on our employment status. That very last section in chapter five is just an acknowledgement that when you’re working hard to be an effective teacher, it’s tiring, it can be daunting. There’s some real highs and lows. And to guard against burnout—to be aware of what you can and can’t do, and to really—I circle back and say again—fight that super teacher myth. Get that damn Robin Williams in Dead Poet’s Society, “Captain, my captain.” Get out of my head, get out of my students’ head. That’s not how teaching goes. Every once in a while you have a magical moment, but there’s a lot of grinding hard work. Most of us become effective teachers the same way our students learn how to do something, which is just slogging away at it day after ever loving day. That’s how you get better. That’s how you become effective. And it’s hard.

Rebecca: Earlier you talked about inviting students into our geeky spaces. Can you talk a little bit about how you invite students into your geeky spaces?

Jessamyn: Well, I’ll never forget the time—I described this in chapter five—I just stumbled on it. I made a joke in passing about my own geekiness about an article we were reading and I got this big student laugh. So student laughs, always good. But later, reflecting on it, I realized what made that so pedagogically effective. So first, there’s a lot of debate and discussion about what exactly a nerd is, what exactly a geek is, but the one thing we all agree on is they’re super smart. So when I said, “I’m a big nerd,” to my students, it was joking and yet it was also reinforcing my expertise and my knowledge. I know a lot about this, I am a big nerd. It also, I think one thing I talked about in the book is the importance of enthusiasm and that’s a difficult term for someone like me who’s pretty reserved—in many ways, an introvert—I’m not going to be a cheerleader. I’m not extra warm and fuzzy, I’m pretty intellectual. But I am passionate and I love the things I’m teaching, and when I position myself as the big geek in the room who can’t get enough of this topic, it helps me convey that enthusiasm to students. One of the studies that I cite in the book says a massive survey of students who said they perceived a teacher as authentic when the teacher was happy that class begins. That’s a tough one for introverts because part of me is always going to be back alone in my office doing my research or my scholarship, whatever gets me going as a scholar. But by embracing my nerdy love for my subject, I’m able to convey to students, “I am happy when class begins.”

Rebecca: Before we started recording, you were talking about zombies in your class. [LAUGHTER]

Jessamyn: Yeah.

Rebecca: Could you talk a little bit about some of the things you’d like to nerd out on?

Jessamyn: Sure.

John: Or about that class in general.

Jessamyn: I would say—let’s see—I’m going to put it in the teaching context. So the thing that I most love about teaching my subjects is getting students to rethink something they thought they knew about popular culture. And everybody arrives in my classes, knowing a lot about popular culture in many ways. They know how to watch a movie, they know how to play a video game. And so getting them to rethink those things is what is most rewarding to me as a teacher, I sort of joke about it but sometimes when students will say, “Professor Neuhaus, you ruined such and such for me,” like some movie or some TV show like, “Now I can’t stop thinking about it.” That’s so rewarding for me as a professor. Being able to have students apply some pretty abstract cultural studies work to their real lives, that’s what makes me happiest as a professor. Every once in a while a student will say, “That documentary we saw, it was so interesting, I forced my roommate to watch it,” or, “We looked it up online, I wanted my dad to see it.” That’s like a microphone drop for a professor. I figure if they’re talking about it for no reason except they were interested outside of class, then I’ve definitely done my job. For me personally nerding out, a big chunk of it has been on the history of gender and prescriptive gender norms, prescriptive literature. So my first book was about cookbooks and gender, my second book was about advertising and housework. And I’ve written articles about sex-manuals, classroom films, instructional films, high-school instructional films, and I could talk about those things for hours and hours.

Rebecca: Will that be geeky pedagogy as well, right?

Jessamyn: Yes, that’s right. [LAUGHTER]

John: What strategies do you recommend for faculty trying to improve their teaching and where can they find assistance?

Jessamyn: You got to nerd out about teaching and learning. We already know how to nerd out about our topics, ask anyone teaching a college class “Tell me about this subject” and watch their little faces light up, hear their voice get animated, no matter how arcane or obscure. Donald Glover said “Strange, specific stuff; That’s what makes a nerd a nerd.” And that’s what makes academics, academics. And I’ll just use as an example, my good friend and colleague—he’s a historian of industrialization, Jeff Hornibrook, at SUNY Plattsburgh—he spent almost 20 years studying a single coal mine in China. A hole. A hole in the ground. But as he explains in his book, A Great Undertaking, this hole can tell you so many interesting things. That’s an academic nerd for you. We can apply that same focus and ability to studying, teaching, and learning. And just like we’re always learning about our topic, we’re always going to keep learning about teaching. So resources for that definitely SoTL—scholarship on teaching and learning—keeping in mind anything you read you have to apply in your specific, unique, individual context. I also think, probably the most immediately effective resource is your campus teaching and learning center. I’ve yet to speak to a single person who’s ever had anything negative to say about their experiences at a teaching and learning center. If you have access to one, if there’s one on your campus, it’s the very first step you should do for any kind of support and for resources. I’ve also learned a ton about teaching from teaching conferences, which I think are a totally different world than academic conferences. Academic conferences, you’re supposedly there to share knowledge but really it’s about proving you’re smarter than other smart people, in my humble opinion. Teaching conferences, I really do see people trying to share knowledge, and like in the book I say is like, “The mothership come home,” you’re surrounded by other nerdy people who want to learn about teaching. And I’m also going to put—going to say—academic Twitter. I’m going to say that. I only joined Twitter for recently. Thanks, marketing team West Virginia University Press, [LAUGHTER] they really said, “You should think about doing this,” and I did, hoping to get the word out about my book. But, just very surprisingly, I found that it has significantly expanded my pedagogical community of practice. Of course, it has significant limitations. 280 characters, that doesn’t leave a lot of room for citations as citations nerd like me. But it does provide a key component of reflection and improvement for teaching practice, which is talking to other people about teaching. And especially for me—as an introvert—I’ve always been terrible at networking. And I teach at a very small, very rural, and isolated university, and Twitter has really expanded my ability to hear what other people are doing in the classroom. And also, it really does, in many ways, work to de-center privilege in discussions about teaching and learning. It really is a platform for voices from across different employment status, teaching contacts, identity, and so on.

John: One of the barriers I think that a lot of people have with academic Twitter is when they first sign up for it, it doesn’t seem to offer much benefit until they start following enough group of people. So it takes time to develop that personal learning network there to make it more useful. And it’s worth taking the time to do that, but it doesn’t have that immediate feedback that many other types of social networks perhaps may.

Rebecca: But you can take your time to compose your interactions and not be caught on the spot. So in that way, it’s really wonderful. [LAUGHTER]

Jessamyn: Yeah, it is. It’s kind of a boon for introverts.

Rebecca: What are some ways that your book differs from other books on teaching and learning?

Jessamyn: Well, it’s fun to read. [LAUGHTER] That already sets it apart. I would call it SoTL with a side of snark. So in other words, it’s real scholarship. I mean, it’s so packed with citations, I had to cut out a bunch and I’m going to have to make that available as supplemental bibliographies online. So it’s real research, there’s all kinds of scholarly resources in there. But I’m allergic to jargon, and pomposity, and I really like to make people laugh so the book is highly readable. It’s multidisciplinary. I’m a multidisciplinary scholar so it was easy for me to move beyond the rigid dictates of studying history—that’s my discipline—and it takes into mind the real variety of faculty who are looking to become effective teachers with all kinds of different individual teaching contexts, employment status, embodied identity. It’s highly readable in a narrative style and it’s written by someone who who doesn’t take herself unduly seriously, and someone who can acknowledge the roller coaster of teaching and learning.

Rebecca: I’m really looking forward to checking out when it comes out in September, right?

Jessamyn: Yes, September 1st.

John: And we will include links to everything we’ve referred to here in the show notes.

Jessamyn: Great, thank you.

John: We always end our podcast with the question. What are you doing next?

Jessamyn: I guess you mean after I’m done with my worldwide speaking tour…

John: Right.

Rebecca: Clearly.

Jessamyn: ….when Geeky Pedagogy becomes an international bestseller. [LAUGHTER]

Rebecca: So it’ll be years from now.

Jessamyn: Yeah, years from now. After I convince all intellectuals everywhere to nerd out about teaching and learning. What I’d like to next focus on is de-centering privilege in SoTL. I’m not sure what form this will take yet. I’m considering maybe like an edited collection. I’m thinking specifically of practical pedagogical strategies for underrepresented and marginalized faculty. There are a lot of excellent books and articles, anthologies, scholarship, and reflection about disparate teaching realities—that’s Roxanna Harlow’s term—but what’s needed now, I think, is building on that for some practical suggestions and guidance for increasing pedagogical content knowledge when you don’t look like a professor.

Rebecca: I look forward to seeing what form that takes, it sounds really interesting.

Jessamyn: I’ll be back to talk about it on Tea for Teaching.

Rebecca: Yeah. [LAUGHTER]

John: Well, thank you very much.We’re both very much looking forward to your book and hearing more about it.

Jessamyn: Thank you. Thanks for having me.

Rebecca: Yeah, we appreciate the time you spent with us today.

Jessamyn: Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson. Brittany Jones, and Gabriella Perez.

81. Intentional Tech

Some faculty try to use each new educational technology tool they find. Others are reluctant to try any new tools. In this episode, Dr. Derek Bruff joins us to examine how to productively choose educational technology that will support and enhance student learning.

Derek is the director of the Vanderbilt University Center for Teaching and a principal senior lecturer at Vanderbilt Department of Mathematics. He’s the author of Teaching with Classroom Response Systems: Creating Active Learning Environments. His new book Intentional Tech: Principles to Guide the Use of Educational Technology in College Teaching will be available from West Virginia University Press in November 2019. Derek is also a host of the Leading Lines podcast.

Show Notes

Transcript

John: Some faculty try to use each new educational technology tool they find. Others are reluctant to try any new tools. In this episode, we examine how to productively choose educational technology that will support and enhance student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Derek Bruff. Derek is the Director of the Vanderbilt University Center for Teaching and a principal senior lecturer at Vanderbilt Department of Mathematics. He’s the author of Teaching with Classroom Response Systems: Creating Active Learning Environments. His new book Intentional Tech: Principles to Guide the Use of Educational Technology in College Teaching will be available from West Virginia University Press in November 2019. Derek is also a host of the Leading Lines podcast. Welcome, Derek.

Derek: Hi, I’m happy to be here.

John: We’re happy to have you here. Our teas today are…

Rebecca: Do you have anything that you’re drinking Derek?

Derek: So I do, I have some coffee here. [LAUGHTER] I’m not a tea drinker. But there’s a bit of a story. I’m drinking a coffee called Kaldi’s Dog from a local coffee vendor called Bongo Java. And a couple years ago—I work here at the teaching center—we had been serving Folgers coffee in our coffee machine in the break room for several years. And some of us claimed that it was terrible and others of us claimed that people can’t actually tell the difference between coffee brands. And so we actually had a taste test at one of our staff meetings, a blind taste test. [LAUGHTER] From Folgers and several other kind s of fancy coffees and I have to say, I was justified actually. It was very clear that that some coffees were more alike than others. And this was actually the winner, Kaldi’s Dog… the winner of our taste test.

John: So there was no p-hacking or anything going on there? [LAUGHTER]

Derek: No. No.

Rebecca: The nice thing about our tea selection is that we just make hot water and then you can have any of the many varieties that we have in our office.

Derek: That’s true. That’s true.

Rebecca: So speaking of which, what do you have John?

John: Ginger Peach Black tea.

Rebecca: And I have Gold Monkey still.

John: Okay. Mine is nearly empty. [LAUGHTER]

John: We’ve invited you here to talk about your new book. Could you tell us a little bit about the new book and what prompted you to write that?

Derek: Sure. So my work at Vanderbilt involves working with a lot of faculty around their teaching—much like your work—helping them kind of think through the choices they have as teachers, what kind of objectives they have as teachers, and what are some teaching strategies, activities, tools that they could use to try to kind of reach those objectives with their students. It’s a really great job, I get to work with faculty all across the campus, lots of different disciplines. And, in recent years, it’s taken me to other universities as well—and colleges—to kind of talk to faculty there. And so my area of expertise and kind of specialty is around educational technology. And I kind of feel like a lot of faculty come at technology in their teaching from kind of three different areas. Some faculty are told by their administrators that they need to use more technology. And they’re not always sure why. [LAUGHTER] Like what’s it good for? Why do I need this? How can it be helpful? And then kind of at the other end of the spectrum, we have all these faculty who are easily distracted by shiny objects and they see a new technology, and they’re like, “Oh, Pokemon Go, how can I use this in my teaching,” right? [LAUGHTER] And they’re great, these folks are great to work with, they’re all great to work with. But there are also a lot of faculty kind of in the middle who just want to teach well, right? They want to connect with their students, they want their students to succeed, and they want some sensible tools to help them get there. For all three faculty, sometimes they struggle with figuring out how to match technology with learning goals and teaching principles. They know kind of what they want to accomplish, but they’re not sure how to select or use the technology that helps them get there. And so the example I often give in my talks is, I’ll say that my favorite teaching technology is actually wheels on chairs. [LAUGHTER] When I walk into a classroom, right, I have stuff I want to do with my students, I have learning experiences I’ve constructed for them, and I want the furniture in the room to be flexible enough to support what we need to do. Maybe it’s small group activities today, maybe we circle up and have a whole class discussion, maybe we use a debate. I want the technology in the room to support my teaching choices. And so it’s pedagogy first, but then we find tools that help us accomplish those pedagogies. As I’ve talked with faculty at Vanderbilt and elsewhere, I see a lot of patterns in how they use technology and what I’ve done is I’ve tried to distill these patterns down into seven teaching principles, because it’s a book and you have to have seven principles, right? That’s the rule.[LAUGHTER] So seven teaching principles that kind of give you a reason for using technology. And so the intent is to guide faculty to say, “Oh, here’s why I would use technology,” and then each chapter explores one of those principles and has lots of examples of actual teaching practice from faculty in a variety of disciplines. What does it look like in English to use technology to accomplish this goal? What does it look like in biology? What does it look like in communication studies? That kind of thing. I love telling stories and one of the reasons I’m excited to be a part of the Teaching in Higher Education series of West Virginia University Press, it’s edited by Jim Lang—who is a fantastic writer—and he takes this kind of storytelling first-person personal approach to his writing and I was really excited to be a part of this series, because that’s how I like to write too.

John: And you start your book with a chapter on a time for telling, speaking of narratives. Could you tell us a little bit about what the focus of that is?

Derek: This is a little counterintuitive, sometimes for faculty, but it’s really one of the most useful principles I found when working with faculty around designing especially—I mean, to some degree, it works at all scales—but it’s really helpful in kind of a lesson-plan scale, like one day in the classroom. And so I think sometimes there’s this impulse that faculty have to explain the thing, and then have the students do something with it, right? Here’s what it is, here’s the background, here’s the context, here’s the theory, and then let’s have the students do something with that. But the example I give actually comes from my daughter’s preschool. This was 10 years ago now, her preschool had science day and they asked the parents to come in and do sciencey things. And so I was the dad who brought the Mentos and Diet Coke. [LAUGHTER] So, you’ve seen this, right? You take a Diet Coke two-liter and you put some Mentos breath mints in there and then half a second later, you get this huge geyser of Diet Coke. It’s rather dramatic. Mine only got maybe seven-feet tall, but I’ve seen them much higher on YouTube. [LAUGHTER] And so I do this kind of fun thing in front of the five-year olds. And then they ask me “Why did it explode?” And so one could lecture—maybe not to five-year-olds—but you might lecture to a bunch of chemistry students for fifteen minutes on the physics and chemistry behind this and then do the demonstration. Or you could start with the demonstration and have students conjecture. Why is this working? Why does this explode, right? Bless her heart, my daughter asked, “Why is it Diet Coke?” I was like, “That’s a good question.” Why is it diet and not regular Coke? So this is the idea behind time for telling—this is a term from the literature Schwartz and Bransford wrote about this back in the 90s—that if we can create these times for telling with our students where they’re ready to learn and they’re interested in learning, then they’re going to get a lot more out of it, they’ll learn more deeply. We can use technology to do this. One of the stories that I share in the book is a grad student in English at Purdue University, Alisha Karabinus, and she’s teaching a first-year writing composition course and she has her students play this text-based online game. It’s all text, and you’re typing in commands and telling your character where to go. You kind of wake up in this apartment and you’re not sure what’s happening and you have to navigate and walk through the apartment. And there’s this kind of sequence where you need to take a shower and so you walk into the bathroom and you try to take a shower and the game’s like, “You still have your clothes on, you have to take the clothes off.” And then they’re like, “You probably don’t want to walk into the shower with your clothes in your arms.” And so you’d have to put the clothes down and you have to take her watch off, right? Like there’s all these kind of step-by-step things. I’m of a certain age where I played games like this back in the 80s…

John: The old adventure games, yes.

Derek: Right. And so there’s a kind of, you know, interface here that you have to master and you have to learn. It needs to be very explicit about what you’re doing. So she has her students play this game outside of class and then they come in and they debrief the experience. And it’s really lovely because they get so frustrated with the interface, then she makes this nice, clever little pivot where she says, “Well, when you’re writing, when you’re trying to express yourself, you’ve got all these ideas in your head. If you’re not explicit with your reader, they’re not going to know what you’re actually saying.” And so she uses that to talk about transitions and topic sentences and things like this. And so I think it’s a really lovely example of using a little bit of technology that was not at all designed for teaching to give students an experience that then prepares them to learn this lesson about how they communicate and how they write.

Rebecca: Do you have any advice on how to find some of those key little demonstrations or technology pieces that could lead into particular ideas?

Derek: Yeah, I’ve got some more examples in my book. I mean, part of it is that I think—especially for the time for telling—there’s this kind of experiential piece that’s pretty great…

Rebecca: Yeah.

Derek: A lot of faculty will show a video clip. This is one way to kind of do it. The Office is very commonly used to introduce various topics in different courses. I have several examples in the book of games, either video games or board games, and so I think there’s some real value in this experiential piece. And so, there’s no silver bullet. I think a lot of this involves being open to taking something outside your area and bringing it in. In this case, part of it was the interface. It wasn’t the content of the game that was interesting, it was the interface of the game that really helped. And so those are things to look for. Is it the content? Is it the interface? Those are helpful. I also talk about what tools are designed for teaching that can help create this time for telling. And so my first book was all about teaching with classroom response systems—which used to be called clickers—and now in most places students bring their own devices and answer on their phones. But the idea is that you can pose a question to all of your students and they all answer and then you can show the distribution of answers up on the big screen. And if you’ve asked a question that really taps into some type of misconception that students have, and they get the question wrong, the technology is important here, because you want everyone to answer so they all have that experience of grappling with the question. So you need a way to hold all the students accountable for answering in a way to collect all their answers. So you need some tech for that. And then by displaying the distribution of answers and the wrong answers on the board, you let everyone know, “Hey, this is a hard question, right?” It’s not like everyone got this right? You’re split across these two different answers. I share an example in my book of a colleague here at Vanderbilt in the law school, Ed Cheng, and he’d ask a series of questions of his students about Carl and his rhinoceros. So this was a situation that was perhaps prone to disaster when Carl keeps a pet rhinoceros and so he plays out these different scenarios of things going wrong. And then he basically asked these multiple choice questions about who can sue whom for what. And so the first two questions are actually really straightforward, right? The rhinoceros escapes and runs into a car or something, and Carl should have known that was going to happen. And there’s clear cut answers to the first couple of these clicker questions that my colleague asks. But then the third one, it’s a little bit different. And the students, when they respond to the scenario about Carl and his escaped rhinoceros, they’re actually split across three different answer choices. Ed has this great move in class where he’s like, “Well, you’re kind of all right. There are parts of law that are really clear cut and there’s a clear answer, and we just talked about a couple of questions that fell under that category, but we’ve moved into this area where there’s actually not a clear answer and a good lawyer could argue any of these.” It’s critical for his law students to know when they’ve moved into that part of law because that’s when they have to really do the hard work and marshal the resources and make the arguments and work with evidence. And so he’s using this short technology exercise to create this moment where they’re like, “Oh, right, I need to really pay attention here.” It’s that time for telling moment that I think is really lovely and having the bar graph on the screen that has the three-way tie is really important to creating that moment.

Rebecca: I think those are really good examples that I think will help faculty get started.

John: Actually, we did talk about one in an earlier podcast where we had Doug McKee on and he was talking about using this technique in his econometrics class, where you give students a problem that’s just a little bit above what they’ve been working on and it forces them to recognize the need to develop new tools, and then they’re primed to be receptive to a solution if they don’t quite make it all the way there themselves.

Derek: Absolutely. And again, this is a little bit counterintuitive. I think some faculty are hesitant to give their students a problem they know they can’t finish, or they can’t solve, or they haven’t been fully prepared for. But by having that experience, starting class with this hard problem that they can’t quite finish and getting stuck and recognizing the limits in their mental models or their need for additional resources, then they’re ready for the second half of class when the faculty member’s like, “Oh, here’s the resource, here’s the concept, here’s the tool.” And again, very non-intuitive and one of the things I think that’s important about my book is that my focus is on using technology to accomplish these things. But all of these teaching principles are true regardless. You don’t have to use technology to create a time for telling, but it is an interesting and useful way to think about certain types of technology and how you might bring them into your classroom.

Rebecca: I think that’s an important point to hit. The technology is supposed to follow the pedagogy like what you said earlier.

Derek: Yeah.

Rebecca: So remembering that you need to make good teaching choices first, and then finding ways to support.

Derek: Yeah, and sometimes tech is not the answer, right? Or low-tech is a better choice, right? So I have a chapter on knowledge organizations and so this is the idea that when we all organize the information in our head in various ways and so you can kind of imagine in your head like this concept map of ideas and examples and facts, and there are connections between all of them and novices in a domain, right? When our students walk into our class, their knowledge organizations are not as robust, they don’t have as many nodes, they don’t have as many connections, and connections are not as meaningful, and part of our work as instructors is to help them develop more robust knowledge organizations. Well, if we just leave them to their own devices, they’ll do okay, but we can actually help them learn and see the big picture in our course, if we can give them activities that help them develop, construct, represent, and visualize their own knowledge organizations. And so I teach a first-year writing seminar at Vanderbilt—and I talk a little bit about my own teaching in the book, because I think it’s important that I’m using these tools myself and trying to figure out how they work—my first-year writing seminar is on cryptography. Codes and ciphers. And we talk about privacy and surveillance and the role of encryption in today’s society. As part of this, I teach a novel, even though I’m in the math department. [LAUGHTER] It’s not something I was trained to do in grad school, to teach a novel, but I do work a teaching center so I’ve picked up some ideas. But there’s this book called Little Brother by the author Cory Doctorow and it kind of imagines a terrorist incident that happens in San Francisco and then the kind of surveillance and security apparatus that comes after that and the lead character is this teenage hacker who’s kind of fighting against this. And so what I have my students do is they read the book, I have them blog about it in the course blog—so that is a digital technology that they use—but when they come to class, I ask them to get into small groups, I give each group a couple of large Post-it Notes—so these are the kind of five-inch by seven-inch brightly colored Post-it Notes—and some markers. And I say, “Your job in the group is to find two arguments in the book in favor of surveillance and two arguments in the book in favor of privacy.” And so they have to kind of page through the book, they’re looking for arguments made by specific characters in favor of one of those two things. And so the privacy arguments go on Post-it Notes of one color and surveillance arguments go on Post-it Notes of another color. And so then, once they’ve done that piece, the second phase is they in turn go up to the chalkboard and they put their Post-it Notes on the board one at a time, and they have to do two things here. One is they have to put practical arguments towards one end of the board. Like, “If we monitored everyone’s subway movements, are we actually going to catch bad guys?” Like that’s a practical argument. And then have to put principled arguments at the other end of the board. So one of the characters says that, “Hey, it’s about life, liberty, and the pursuit of happiness in that order. If you’re not alive, you can’t be happy, so we got to keep you safe.” That’s a principled argument. And so the students have to put their their Post-it Notes along this axis and then they also have to use the chalk to connect their argument to something already on the board. Because basically, there’s this really complex debate space around safety, and security, and privacy, and surveillance, and I want my students to know how complex that is and to start to see the relationships between some of those arguments and ideas. “This argument is a counter to that argument, or this argument is a support to that argument.” So by the end of class, they’ve constructed this debate map on the chalkboard out of Post-it Notes. And they have, I think, a much better sense of the complexity of this debate. They’ll do more with this. They’ll write about this topic throughout the semester. And so that debate map, that knowledge organization that they’ve constructed collaboratively, can then inform the arguments they make as they take positions within that debate later. And in this case—as I said—this is pretty low-tech, it’s Post-it Notes and chalkboards. I actually tried it once using some software, but having students build this map in a collaborative digital space at the same time was just too chaotic and so I needed to kind of slow the process down, and the Post-it Notes were really great for that. And so this is something I’ve used several times in my course and I think it’s a really great way to help students see how the ideas in an argument space are connected.

Rebecca: One of the things that I’ve liked about using Post-it Notes in some of the kinds of things that I do in my classrooms is that it is easy to change your mind too. You can easily pick it up and move it and it doesn’t seem as intimidating as trying to navigate software to make a decision or something. It somehow lowers a whole bunch of barriers and then it’s a little more flexible and fluid.

Derek: Yeah, absolutely.

John: And you’ve got them thinking about it among multiple dimensions and also making connections, which really, I would think would help them develop a lot of scaffolding there and a lot of connections that they wouldn’t necessarily do with their own reading. So it’s forcing them to develop better close reading skills and analytical skills, and so forth. It’s a wonderful exercise.

One of the principles of learning is that it helps for students to have lots of feedback opportunities and lots of practice, and I see you have a chapter on that. Could you tell us just a little bit about what you focus on there or some of the points that are made in that chapter?

Derek: Sure. So I actually start the chapter of the story about how I learned how to ski a couple of years ago, because learning to ski as an adult is a challenging process, as I found. [LAUGHTER] It involves much falling down. And every time you fall down when you’re learning how to ski, your body is getting a little bit of feedback about what works and what didn’t. And so in a very kind of physical motor skills way, to learn to ski, you have to practice skiing. You fail a lot, your body gets feedback, and then hopefully over time you get better at manipulating your limbs and controlling your muscles so you’re going kind of where you want to go. And so our students need this too, this is actually so key to learning is that we have to practice with the stuff that we’re learning, we have to do stuff with it and we have to get feedback on that practice. It’s a key part of learning. In the chapter, I use this as an opportunity to talk about the so-called flipped classroom because I think there’s a traditional model in some of our disciplines where students get an introduction to information during class. And then after class, they go and do something with it. They do the practice, they have a problem set, right? And the problem is that the practice and feedback part, it’s really important and also really hard. And so to have students do that when they’re left to their own devices, is a lost opportunity actually. And so the flipped classroom model says, “Let’s take some of that activity. Instead of doing it later on your own, let’s do it together collaboratively during class.” And so in the chapter I talk about some ways that some faculty have used technology creatively to help students practice the skills of their discipline during class. I mentioned the classroom response systems as certainly an option for this. I think sometimes when I talked to faculty around technology in the classroom, there’s sometimes an assumption that you’re talking about AV tech. We have a projector, we have some speakers… and that’s great, we need that, that’s helpful. But all my examples involves students using the technology because I think that’s really important. One of my favorites, actually is Kathryn Tomasek from Wheaton College. She’s a historian, and she wanted her students to practice doing the kind of close reading that historians do. When they get a primary-source document, a lot of that reading they do is looking at it line by line, word by word, figuring out who is that person? What is that term? What does it mean? Looking at the very building blocks of this primary-source document, because especially if you’re separated in time by one hundred or two hundred years, you got to do a lot of this close reading to kind of make sense of what it is. And so she had her students work with… she started with historical documents in her library’s special collections and asked her students to do what’s called a TEI. It’s a text-encoding initiative, it’s a way of marking up the text—kind of like HTML a little bit if you know web development—where you’re actually kind of tagging things in the text and labeling them as to what they are. So this is a date, this is a noun, this is a person, this is a location, or in her case, this is a theme that comes out in the sentence. And so her students, they would do this together in class. Like she’d take a piece of it and walk them through it collaboratively on the big screen and then have them take their own pieces and do this markup. And the neat part is the students were actually contributing to a larger digital history project because their markup would be kind of incorporated in this bigger database and shared online. And the work that they did with the primary source documents would then inform the writing and the argumentation that they do later in the semester. But in this case, she wanted to target a very particular skill that’s kind of close reading in history and she found a technology that digital historians use actually pretty regularly to create these opportunities for practice during class to help our students do this kind of work. I share another example of Richard Flagan from Caltech and he was doing chemical engineering. Very different course. But he used little mini projectors to kind of turn his lecture hall into an active learning classroom so his students could work in groups and do some coding—they were doing MATLAB coding in this case—and he found that when he introduced the coding in class and had them work on it after class, they would get hung up on these really small errors around grammar and syntax in the code. So he shifted that work into class to do group works in class and so then he’s able to kind of circulate among them, see what they’re doing on their little projector screens, and intervene and ask questions and help them. And so again, it’s kind of targeting these very particular skills that students need practice with that will inform often bigger projects later in the semester, but creating some time and space in class through technology to give them a chance to practice those skills and get feedback either from each other or from the instructor.

Rebecca: A lot of those examples I think are opportunities for faculty to also see where misconceptions are happening because it’s happening in class soyou can address them one-on-one, but you can also address those bigger themes that bubble up as well as a bigger group rather than having the same conversation 20 plus times.

Derek: Absolutely. Yeah,you may walk over and talk to one student or a small group or you may see a pattern across the students and then kind of take a moment to kind of gather everyone’s attention and try to kind of walk them through together as a whole class.

John: Doing some just in time teaching type of techniques, which is much more efficient use of class time.

Rebecca: That seems really tied to the knowledge organization that you were talking about as well because I think those same kinds of things happen when you’re doing those sorts of activities in class too, right? Like, “Oh, I didn’t realize that you thought this was connected to that,” right? And you can help negotiate that. [LAUGHTER]

Derek: Yeah, and these teaching principles overlap, right? So when I had my students do that debate map activity in class, we were doing practice and feedback. We were taking class time, it was just that this kind of analysis level where they were making connections across topics as opposed to Kathryn’s example of the close reading. That wasn’t necessarily a big picture kind of practice and feedback, it was a very skill focused practice and feedback.

Rebecca: I also really like that these are examples that don’t necessarily make feedback more work for faculty. It’s embedded in the practice in the classroom and it’s just when they need it. And it makes more sense because they’re getting it while they’re doing something so they’re probably more apt to listen to said feedback rather than getting it on an assignment that you hand back and they put it in the garbage or something.

Derek: There’s this book by Walvoord and Anderson called Effective Grading that you may be familiar with and I remember the first time I read about what they called light grading. L-i-g-h-t grading. I thought, “Oh, this makes so much sense. I don’t have to grade everything the students do very rigorously, I could give them a grade on the effort or I could give them a zero, one, or two if the work that they’ve invested in that little piece then shows up later.” So if I have them write a blog post before class to get ready for class, I don’t have to grade that very intensely because we’re going to talk about that material in class and that’s where they get the feedback. I may need the grade it enough to motivate them to do it, but I don’t have to give them detailed feedback at that stage, it will happen during class discussion. And I think that’s kind of freeing for instructors to know I don’t have to grade the heck out of everything. I can kind of design a sequence where students get the feedback they need apart from the grade itself.

Rebecca: I think faculty always appreciate those opportunities. [LAUGHTER]

John: And in the podcast that’ll be coming out a week before this, we talked briefly about specifications grading, which is a variation on the same theme.

Derek: Oh sure.

John: One of your chapters is entitled “Thin Slices of Learning.” Could you tell us a little bit about that?

Derek: I am so glad you asked. [LAUGHTER] Yeah, so this may be my my favorite chapter in the book just because I think the creativity that faculty brought to their use of technology in some of the examples I share, it’s just really amazing. And I also get to quote, one of my mentors a couple of times. Randy Bass is, as I like to say, the Vice Provost of Awesomeness at Georgetown University. [LAUGHTER] That’s not his actual title, but he gets to do some really amazing things there. He’s also really active in the Scholarship of Teaching and Learning world and I think he’s just a really deep thinker about how learning works and so there’s a couple of things that he talks about that I’ve heard him talk about before. He was doing some video projects with his students in an American Studies class a while back and he would look at their finished products—these short videos that they put together as a class assignment—and he realized that he wasn’t seen all of their learning. That—as he said it—there’s a lot of learning that’s left on the cutting room floor. And actually, in the book I talk about how my daughter wrote—she created a short film a couple years ago just because she wanted to—and she filmed, I think three hours of footage for a two-minute film. And so to decide what footage to use, what footage not to use, which angle, which take, even kind of which characters she wanted to include in the final product, there was a ton of decisions that went into those final two minutes. But if you just look at the two minutes, you may not know what those decisions are.Ttechnology, though, can be really good at making visible student learning, and in particular, thin slices of student learning. The kind of choices and sense making that they’re doing in the middle of learning, or creating, or designing, or producing something, and the more we can learn about how our students learn, the more we can kind of get those thin slices of learning in front of us. We can be responsive, we can be helpful, we can guide, and we can direct. And so the examples in this chapter about using technology to make visible (or sometimes audible) kinds of learning that students might not kind of share with us naturally. My all time favorite example of teaching with Twitter is from Margaret Rubega. She is a biology professor at the University of Connecticut. She’s also I think, Connecticut’s State Ornithologist, and she teaches a course on ornithology. So it’s a course on birds and so she has—wait for it—she has her students tweet about tweeters. [LAUGHTER] She’s so articulate about it, like students come to this class—and it’s a fairly large class, I don’t know, 40, 50 students—and they’ve seen really cool birds on National Geographic or YouTube and they think of birds as doing really amazing things in the Amazon and in Africa and far away. And she wants her students to know that birds in rural Connecticut also do really interesting things. Their ecology, their behavior, their biology is all very interesting. And so what she has them do as they’re learning about birds in the course is several times a semester they’re asked to tweet about their observations of birds as they go about their lives. So they’re on their way to work, they’re on the way to school, they see a bird, they see it do something or behave in a way that connects with what they’re learning in class. And she asked them to tweet about it. They have to kind of include their observation and where they are, and they have to connect to the course material, and they’ve got now 280 characters to do that on Twitter. Sometimes they get photos of birds that they see, which Twitter is really good at handling that. The thing I like about it is that it’s leveraging the field observation device that they carry around with them in their pockets—also known as their smartphone—or their regular phone, whatever it is. When they’re in the moment when they see that bird doing something, they’re able to kind of capture that and then share that back with her and with the entire class actually. One of my favorite tweets is a student who was walking by a golf course and he noticed the bird song sounded different in different parts of the golf course and in his tweet he conjectured that the golf course itself was dividing bird territory. [LAUGHTER] I was like, “That’s genius,” right? I don’t know if he’s right, but he’s really paying attention. And what Margaret’s doing is she’s helping her students practice transfer, taking what they learned in the classroom and apply it to real-world situations outside of the classroom and that’s one of the hardest things about learning, is how to transfer learning to new context. And she’s giving her students explicit practice in this, but then also making visible that practice by having this class hashtag on Twitter, #birdclass. And if you go search on Twitter for bird class you’ll see some of the tweets from her students because they’re sharing their observations with her, with each other. I just think it’s a really beautiful use of a very particular technology for a very particular reason. I love the bird class example.

Rebecca: That’s really fun.

John: Yeah, that’s a great example.

Rebecca: I couldn’t help but think that if my two-year-old could tweet, she’d be really into it right now. [LAUGHTER]

Derek: Does she have a lot of observations about life that she tries to share?

Rebecca: A lot about birds lately. [LAUGHTER]

Derek: But you think about all the learning that students do when they’re not with us that may be really important. I have another example from Mark Sample at Davidson College, where he had his students live tweet the film they were watching in his sci-fi class. They watched it on their own time in their own rooms or whatever, but it was Blade Runner and they would live tweet their observations about the film. And it’s one thing if we have our students read something or watch something and then write a response paper and bring that to class and then we discuss it, right? That’s great. But he was getting kind of a next level down. They are kind of immediate in the moment reactions to what they were seeing in the film and kind of surfacing that and making that visible. And this is a great use of technology. There’s other ways to do this but technology can be really good at this kind of thin slices of learning piece and that’s one of the connections I want folks to make in the book is that, if you think about it, “Hey, Twitter seems really useful. What can we do with Twitter and our teaching?” Well, there’s a lot of things you can do. But one of those is to surface thin slices of student learning. And that provides some focus for thinking about how you might use Twitter in your class for a very particular purpose.

Rebecca: Sounds really fun.

John: It does, and you can see this in other areas of biology or botany. I think Michelle Miller was on a while back and she talked about a class where students went out in the field to identify plants and tweet back photos and so forth.

Derek: Yeah, and I shared this with a grad student here in civil engineering and she has her students tweet about structural things they see in the built environment that connect to the material that they’re learning. I’ve ran into teacher educators who have their students—while respecting some privacy—but they’ll tweet about what they see in the field when they’re in classrooms and they’re observing teachers in action. They’ll tweet those observations and so yeah, I think there’s a lot of different uses for this kind of application.

Rebecca: This also moves a little bit into the idea of learning communities, because this is a community practice using a hashtag where you’re kind of seeing things outside, but you also have a whole chapter on just learning communities.

Derek: Yeah, and I mentioned the bird class, because there’s a lot of things going on with the bird class piece. And part of it is that yes, by making the students tweets visible to the other students, you have this other dynamic going on which is that the students are starting to learn from and with each other. And students can learn a lot from their professors certainly—and we have a lot of expertise and authority that we that we use in the classroom—but if you think about the places where you learn naturally when you’re picking up a new hobby, or you’ve got some interest of yours that you want to pursue, you’re often connecting with other people who do that too. So like, I have a friend who just went to a quilting conference in Nashville because she loves to quilt and she’s going to connect with other quilters. And that’s how she learns how to quilt, it’s this kind of peer-to-peer learning that she does. And so we can leverage that in the classroom. It’s always a little authentic. Students in a statistics class aren’t going to just get super passionate about statistics and learn from each other necessarily, they’re going to bring their own levels of intrinsic motivation to this. But if you think about all of the different perspectives and experiences that you have in the room with your students, they have a lot to bring and they can actually learn a lot from each other and you can learn a lot from your students. But you’ve got to create some mechanisms for that, it’s not necessarily going to happen naturally. And so bird class is a great example because as the students, I mean, they’re all in Connecticut, right? But other than that, they’re going to different places, they’re seeing different parts of town, then they go to different locations for spring break, and see different birds, and so they’re all bringing their kind of different perspective on this. And by making that visible, they can start to learn from each other. In my chapter I talk about, I use a social bookmarking tool called Diigo, which allows basically students to share links with each other. And so we create a group for a class and I give them these assignments, in my cryptography class especially. So, find an example of cryptography in the news or find an example of military cryptography or let’s find out something about the National Security Agency. And so what’s really cool is—especially for a course like this—students bring a lot of different interests into this topic. And so I’ll have students who have kind of like a literature interest—I had a Sherlock Holmes buff in the class once and so she was always finding really interesting examples of cryptography in literature to share—I had students who were always interested in kind of cybersecurity and computer science and so they’re bringing in kind of modern news and technology. I had one student—bless her heart—she loved Russia, she found a way to find a Russian connection to everything that she did. And so it was really great, right, because she found all these examples of cryptography, especially kind of Cold War espionage stuff that we wouldn’t have seen if she didn’t have this passion for Russia and then found resources and shared them with the class. And so by having students share these resources, in a shared space and then talk talking about them in class, I can really leverage the fact that we’ve got a number of individuals in this room that have different experiences and perspectives and if we can make advantage of that, we can actually all learn more deeply.

Rebecca: I’ve used Slack in my web design classes to do troubleshooting and technical help but I use the same Slack channel across semesters. And so what I found is that people who have graduated who are out in the field will sometimes randomly pop in and answer questions, and it’s really cool, but I remind the students that I’m not the only one that can answer questions. They can help each other out. But sometimes—you never know—some other lurker might pop in and help out. And they have some sort of vested interest, you know, because they were also in that class at one time.

Derek: Absolutely, and that’s one of the advantages. One thing that can happen when you shift away from a course management system, course management systems are good for a lot of things but they’re not good for semester to semester continuity. They kind of put courses in little boxes by semester and the students can’t get out of those boxes. And so once you move to Slack or social bookmarking—like my Diigo group, we’ve been doing it for like seven years. We’ve got hundreds of resources collected by students over time. Course blogs are really good for that too—and so that’s really exciting when you can use some technology to make some student work public and persistent in a way that invites future or past students to participate. It’s still a learning community, it’s just expanding beyond the time and space of this one particular course offering

John: One of the issues with learning management systems is—as Robin DeRosa and other people have called it—is that the assignments often take on the nature of a disposable assignment, that they’ve done the work and then at the end of the semester, they even lose access to it unless they keep it outside. And there’s a lot of advantages to having a sort of persistent work that you’re describing there.

Rebecca: It seems like you’re moving right into another chapter of Derek’s book on “Authentic Audiences.”

Derek: I mean this is the other thing—and again, the book is not a critique of course management systems—but I will say course management systems also make it hard for student work to escape, to be seen by anyone not in the course, and often that’s appropriate. When students are first learning a topic or a discipline, they need a private space to practice, and screw up, and say dumb things, and get feedback, and get better. And that’s true for the assignments, right? Sometimes we have students turn in an assignment to us and we’re the only one who looks at it because they’re still learning the skill set and they need some good practice on that. But when we have students construct work or produce work for authentic audiences outside of the course, that can be hugely motivating for students. Hugely motivating. I’ll quote Randy Bass again, he’s got this white paper where he coins this term social pedagogies. These are pedagogies where we’re asking students to construct their knowledge by representing that knowledge for an authentic audience other than the instructor, and it could just be each other. That can be really powerful as well. But when students see that the work that they’re doing is not disposable, it’s not going to be gone. Students often do write a paper and there’s only one human being on the planet who ever looks at it, but if you can build toward some assignments where students are writing or constructing or producing for an external audience or an authentic audience, there’s a lot of motivational benefits to this for students and they start to take their work very seriously and invest in it in ways that they don’t sometimes in the disposable assignments. One of my favorite examples, Jonathan Rattner teaches cinema and media arts here at Vanderbilt and he had connected with a colleague of his from grad school who was teaching a writing course, Bridget Draxler, she was at another institution. Jonathan was teaching students how to create short experimental films and they needed an audience to share that work with. Bridget was teaching her students to critique media and she wanted her students to find media to critique where they could interact with the creator, and so they just set up a course blog for the two of them, these two courses. It wasn’t public to the world but Jonathan’s students were creating media for her students and her students were critiquing it and then they would have this conversation. And this idea of connecting your course, with just one other course—somewhere else on your campus or maybe at another institution where you have colleagues working—all of a sudden, you have this really authentic audience for the work. And in this case, this was really intentional too, this wasn’t just a random pairing “We want to share stuff with someone,” but there was this kind of synthesis that worked well across the two courses. But that’s a fairly easy way to create some authentic audiences for your students. I also talked about Tim Foster who used to teach Spanish here—he’s out at one of the University of Texas schools now—and he had his students write for Wikipedia. This is actually becoming increasingly common in higher education where you have students write for Wikipedia. There’s certain standards that you have to follow and it’s kind of hard to get content to stick on Wikipedia because of that. He was actually teaching an introduction to Portuguese course—so this was first semester Portuguese language learners—and what they realized was that the Portuguese language page for Nashville on Wikipedia was very skimpy. And what his students didn’t know at first is that Portuguese Wikipedia is not just a translation of English Wikipedia. Portuguese speakers create their own Wikipedia. And so the national page was kind of skimpy. So as a class project, he had his students create content for the Portuguese language Wikipedia page for the city where we are. And so it was great as a language production task for them because they could focus on writing two or three sentences, first semester language learners, but they knew that actual people are going to look at this so they took it very seriously. Some of them went above and beyond. I think this is just a really powerful pedagogy. And again, you don’t have to use technology but this is something technology is actually good at, is connecting people across time and space. Having students use some technology to create something for an authentic audience can be really powerful.

Rebecca: I think you have one last chapter that we didn’t quite get to yet and that’s…

John: “Multimodal Assignments.”

Rebecca: Which, you know, technology is also really good at that whole multimodal thing, right? [LAUGHTER]

Derek: Sure. I was so close to calling this chapter “Beyond the Five-Page Paper.” [LAUGHTER] Because again, the five-page paper has its place where students write a thing, and it’s just text, and they give it to the professor, and they get feedback. Again, there’s a lot of practice and feedback that happens in activities like that. But there’s a lot of research that says, not so much that learning styles exist. The research actually doesn’t support this idea that I need to match my teaching modality to my students learning preference. So the matching hypothesis would say that I have some visual learners and some verbal learners and some kinesthetic learners and so I should do visual stuff with the visual learners and verbal stuff with the verbal learners and kinesthetic stuff with the kinesthetic learners. There’s no research that supports that actually. So there’s kind of this learning styles myth that I like to debunk when I can but where the research does support is multimodal learning. Now, we all learn better when we encounter stuff in multiple ways. And so I think this is the reason the learning styles feels so compelling to a lot of instructors, is that if they’re doing that, if they’re thinking about their lesson plan and saying, “Oh, I got to have some visual stuff, I got to have some verbal stuff, I got to have some activities.” It’s not that they match those with individual students, it’s that all students are benefiting from those three different modalities happening in the same classroom. This chapter is all about multimodal assignments, ways to tap into this dual coding that our brains do where we take in information in verbal ways and in visual ways and kind of put that together and it’s stronger. We’ve done a lot of work at Vanderbilt. We call it students as producers. This is kind of a course design and assignment design approach that we work a lot with here through our course design institute and elsewhere. It’s helping faculty move away from some of those traditional text-only assignments and moving to more open-ended assignments, more multimodal projects, student projects that have authentic audiences. And so actually, this chapter is kind of all Vanderbilt examples which sounds a little self-serving, but I just happen to know a lot of faculty here who are experimenting a lot in this area. I share an example for my own classroom about infographics in a stats class where I’m asking students to represent quantitative information visually. There’s an English grad student here, Kylie Korsnack, who has her students take a paper they wrote, and revise it in a different medium. So it starts off as a traditional paper but they have to revise it as a Prezi or a concept map or choose your own adventure novel, or one student did a Pinterest pin board. And so by kind of re-seeing their work, moving into this other sort of medium, the students are often able to see their work in new ways and realize, “Oh, that’s what my argument really was,” or, “Oh, my transitions are terrible. Now I know how things have to be connected.” And so there’s a lot of value in having students move into different media than straight text as a way to help them make sense of things. I’ve been experimenting a lot with podcasting. So I got this idea from Gilbert Gonzales, a colleague of mine here in health policy, who had his students create podcast episodes instead of research papers. And he really wanted them to be fluent with the language of healthcare policy. HMOs and PPOs and all this kind of stuff. And so an audio assignment made a lot of sense, actually, for the students. And he founded it with a lot more fun to listen to a few podcasts than grade a few papers [LAUGHTER]. And podcasting is a low bar, right? Not to say that what we’re doing isn’t super challenging here, but you can, you know, create a pretty decent podcast with your phone, right? It’s not going to be super high quality, but you can record and you can edit using some free software and so I now have my students do a podcast assignment in my cryptography course. And with about 25 minutes of in classroom technical training, they’re able to produce some interesting things and then we can focus on “How do you tell a story through audio?” Or in my class, how do you explain this kind of technical mathematical stuff that they’re studying through audio only without pictures? And so, again, all of these are about kind of moving to different modalities and shifting between modalities to help students see and understand the material in different ways. And if you keep doing that, they’ll start to kind of triangulate and, and make a lot more sense out of it.

John: I would think it would force them to think about it a bit more deeply to make connections that they might not otherwise. Just seeing things from a different perspective seems to have a lot of value in it.

Derek: Yeah. I would also add that when you move to a non-traditional assignment—this is something that I realized kind of late in writing the book—is that we asked you to do a podcast they walk in and they don’t know how to do podcasts. So Gilbert and I were like, “Okay, so we have to listen to some podcasts together and critique them, and then maybe come up with a rubric together, and they need to outline it and maybe even turning the script and get it approved.” We have this whole scaffolding process around preparing students to do this type of work. Well, some of our students come in and they don’t know how to write a five-page paper either. We assume they do, we assume they’re good at it, that they’ve learned that in high school or something. But for some of these traditional assignments, we have students who really struggle and so one of the things that non-traditional assignments do for faculty is help them realize, “Oh, I really have to get in the head of my students and figure out what’s the scaffolding they need.” And we really should be applying that to more traditional assignments as well, because a lot of our students struggle because we don’t have them submit a proposal, or get feedback on a rough draft, or practice how to find a credible source. These are all things that we that it’s easy to assume our students can do, but they can’t always actually do that. And so moving to a non-traditional assignment often then helps faculty move back to more traditional assignments with a new lens with greater intentionality.

Rebecca: So we have to wait until November to get this book? [LAUGHTER]

John: Yeah, I know I saw when you posted this on Twitter, I said “I’d like this now.” [LAUGHTER]

Derek: Well thank you, yes. The book production process is a long timeline as I’ve found.

John: Yeah.

Rebecca: It’s a good tease though, right? [LAUGHTER] So you were also just talking about how you’ve been experimenting with podcasts and you’ve been the host of Leading Lines since 2016. How did you get interested in doing all this podcasting stuff in the first place?

Derek: Part of it was that—at the time—I had a 45 minute commute to work so I was listening to a lot of podcasts and appreciating podcasts and wishing I had more podcasts like Tea for Teaching that talk about teaching, learning, and higher education. And so that was part of it. I think, also, I was involved in a pretty big online course project that involved a ton of video work. And I saw how powerful that was, but how much work it was to put together really high-quality video and I thought, “What if we had a podcast on educational technology?” There’s folks that I run into here on campus and elsewhere who are doing really cool things and I would just love to kind of give them a bigger audience for the innovative teaching that they’re doing, and producing a podcast seems way more tractable than producing a YouTube series. [LAUGHTER] So I mentioned this at a meeting here, we were having this meeting on campus with some other folks who deal with educational technology. One of our Associate Provosts, John Sloop, for digital learning, he’s like, “I would love to do a podcast.” We kind of both had been thinking about this idea for a while and so we combined forces. And so it’s the Center for Teaching, it’s our libraries, it’s our Institute for Digital Learning. We created Leading Lines, we’ve been doing it for a few years now, each episode is an interview with a faculty, staff, or grad student who’s kind of doing something interesting in the educational technology space. We call it Leading Lines because it has this kind of connotation of looking into the future. So leading lines in a photograph, are those kind of straight lines that draw your eye into the frame. And so we’re not really trying to predict the future—because I think that’s a fool’s errand—but I’d rather kind of shape it and influence it and so we’re looking for folks who are doing things that are kind of one or two steps down the road with technology. And it’s been really great, I mean several of the examples in my book are drawn from interviews I did for Leading Lines. It just gives me this occasion to talk to interesting people who are doing interesting things.

Rebecca: Now you know our secret. [LAUGHTER]

Derek: Right.

John: We’ve gotten this opportunity to talk to all these people doing some wonderful research that we wouldn’t be able to be in contact with so many of them otherwise.

Derek: Yeah, absolutely. I’m always referencing these people that I’ve met and the work that they’re doing and my other work here, connecting faculty, and it’s been a lot of fun. And it’s been fun to work here. So we have about six of us who do interviews for Leading Lines and so we have a little bigger team then you guys have and they go in different directions. Sometimes I’m like, “That’s really not an interesting topic,” and then they do an interview and it’s an interesting topic. And so it’s been really fun to kind of work with my colleagues here and having this collaborative project across multiple units at Vanderbilt, that’s been pretty great too.

Rebecca: Cool.

John: On one of the recent episodes, you had a discussion of the VandyVox project. And in particular you had a podcast from there. Could you tell us what this project is and how that came about?

Derek: So I had this idea actually just last summer. We were running a course design institute here at the teaching center and we had several faculty who are really interested in doing podcast projects because I think I had shared Gilbert Gonzales’s Health Policy Radio podcasts with them. And several faculty started thinking, “Oh I could really use this in my teaching,” and I thought, “This is great, but if we have Vanderbilt undergrads, especially, who are producing really interesting audio for class assignments all over campus, wouldn’t it be fun to curate that to have a podcast of podcasts?” So then I reached out to my colleagues at Vanderbilt student media and they’re like, “Yeah, that sounds great. We love to help students make media and share media with the world.” And so they were able to do all the heavy lifting on the technology piece, all I had to do was reach out to some faculty members and ask them to recommend some student produced audio for this and so this spring we launched VandyVox. It’s the best of student produced audio from all over campus. It’s a bit of a fudge, right? Like if some student has a Sports Radio Podcast, we’re not covering that. But if there’s an academic component to it, if it’s curricular or co-curricular, we’re happy to feature it on the podcast. And so it kind of serves two purposes. One is to kind of shine a spotlight on some student work, show this great stuff that our students are doing to provide some inspiration maybe for faculty and students to have students engaged in this kind of work. In the show notes for each episode, we have some background information about what the assignment was, or how the faculty members worked with students around this. So there’s a faculty development piece to it as well. But it’s been really fun to see what students are doing all over campus. You know, I highlighted some Health Policy Radio piece, we had a student from an anthropology course on health care politics. She created a 10 minute speculative fiction audio story…

Rebecca: Oh, cool.

Derek: …dramatized it as her project where she kind of imagined what would happen in the future with gene editing and baby selection. It’s just a really great sci fi kind of look at the course topic. Well researched, right? Like she turned in an annotated bibliography with all this so it’s all kind of backed up by the latest research. We had law students who were doing podcast episodes on immigration and refugee law talking to some immigrants and refugees. For that audio. We have Robbie Spivey in our Women’s and Gender Studies class teaches a course called Women Who Kill [LAUGHTER]—which is a great name for a course—and so she had her students do kind of true-crime podcasts about women who kill and how we make sense of that as a society. And then our last episode of season one, which came out recently, featured some work by Anna Butrico, who was a senior here last year, an English major. She did her senior honors thesis on podcasting and kind of connected it to ancient Greek rhetorical forms, which is really great. But her senior thesis had audio pieces to it. It’s hard to do a senior thesis on podcasting without creating a podcast and so we featured the audio introduction to her senior thesis, which I was really excited because Anna actually did a lot of work with podcasting while she was at Vanderbilt and her technical skills and her composition and storytelling skills are really strong. So it’s been really fun to kind of see something of a critical mass here at Vanderbilt around student podcasts and to be able to kind of highlight that a little bit. And I’m really excited, we’ve got some good stuff lined up for season two this fall as I’m kind of reaching out to more faculty and students about the the audio work that they’re doing. And again, part of it is getting started with a podcast is not hard. Doing it really well is still very hard, but the bar for entry is pretty low actually. And so if you want to have your students kind of move into a different modality—and again, you need to kind of be intentional about why you’re doing it and how it connects to your course goals—but podcasts offer a really great option for that. And I’ve just seening more and more faculty start to embrace this as a kind of creative output for students.

John: Going back a little bit, you mentioned that video project or the the video intensive project, I’m assuming those are the two MOOCs you have on teaching in STEM courses. I participated in the first one when it first came out.

Derek: Oh, that’s great.

John: It was a lot of fun, it was really useful. I didn’t do the second one. I think we both recommend those to a lot of faculty and encourage more people to take those. I believe they’re still running on Coursera?

Derek: Oh, yeah, we’re running one every semester. They’re not on Coursera anymore, they’re on edX. But you can always go to stemteachingcourse.org and you can find out information about those courses.

Rebecca: So you’ve already talked about the podcast that you’re working on and your book, the editing process and such that takes a long time, so you’ve got a lot of things in the cooker but we always wrap up by asking, what’s next?

Derek: Short run, we’re running a couple of course design institutes in the first of May and so that will occupy the next several weeks as kind of prep for that and those are always fun because I get to work with faculty. It’s on the theme of students as producers so we’ll be working with faculty around these creative nontraditional assignments. That’s always pretty exciting. There’s also—this is just an idea right now—but I keep running into faculty who are teaching with games or having their students design games, board games especially, as course assignments. I mentioned this text-based game that Alisha Karabinus uses and so I just keep finding examples of games and simulations that have a learning goal or learning purpose and so I’m hoping maybe this fall to put together a little one day symposium on campus on games for learning, games for social change, that kind of thing. I think that’d be a fun space to explore. And the other thing that I’m seeing—and I talked a little bit about this in the book—is this move towards active learning classrooms. I mentioned I like to walk into a classroom and see wheels on the chairs, because we can move them around and make them do what we want. The affordances that our classrooms have really matter for the choices we make as teachers. And so classrooms that are designed to facilitate small group work, student collaboration, active learning, this is a strong trend in higher education. I’m really kind of shocked how even from like three years ago, where I was having to tell people about this idea for the first time, now and here on our campus, our campus planners have decided this is a standard classroom configuration going forward. And so I see a lot of campuses moving towards active learning classrooms. Again, digital and analog technologies that support learning and so I want faculty to use them in intentional ways. And so I think we’re going to be doing a lot more with active learning classrooms here on campus, probably starting a learning community on that in the fall and I’m excited to dig into that work too.

Rebecca: Sounds like a lot of exciting things going on.

Derek: I try to stay busy. [LAUGHTER]

Rebecca: It’s going to be hard to keep up with all of them.

John: Well, we appreciate that and we’re looking forward to the book coming out.

Derek: Awesome.

Rebecca: Well thanks so much for joining us. It’s been really exciting and I know we all have a countdown now.

Derek: Thanks so much for having me on. This has been a really fun conversation. I’m happy to get the chance to share a little bit.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

80. Self-Regulated Learning

Most students arrive at college with serious misconceptions about effective learning strategies. In this episode, Dr. Linda Nilson joins us to examine what we as faculty can do to help students develop their metacognitive skills and become self-regulated learners.

Dr. Nilson is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning. In this episode we focus on discussing one of her books: Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-awareness and Learning Skills.

Show Notes

  • Linda Nilson—Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2013). Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-awareness and Learning Skills. Stylus Publishing, LLC..
  • Nilson, L. (2014). Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time. Stylus Publishing, LLC.
  • Professional and Organizational Development (POD)—Network in Higher Education Conference
  • Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77(6), 1121.

Transcript

This transcript has been edited to improve readability.

Rebecca: Most students arrive at college with serious misconceptions about effective learning strategies. In this episode, we examine what we as faculty can do to help students develop their metacognitive skills and become self-regulated learners.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson, the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. Dr. Nilson is the author of many superb books, book chapters, and articles on teaching and learning. Welcome.

Linda: I’m very honored to be here. Thank you.

Rebecca: Today’s teas are…

John: Are you drinking any tea?

Linda: Yes, I am drinking tea. I am drinking Lemon Lift.

Rebecca: Oh that sounds like a great way to start the day.

Linda: It is. It’s a very good way. Well, I also started it with coffee, but… [LAUGHTER]

John: And I’m drinking Ginger Peach Black tea.

Rebecca: And I have my Golden Monkey tea today.

John: We’ve invited you here today to talk about your book, Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-Awareness and Learning Skills. Could you define what it means to be a self-regulated learner?

Linda: Yes. Self-regulated learning is the conscious planning, monitoring, and evaluation of one’s learning for the purpose of maximizing it. That’s a very fancy way of putting it. It’s that voice in your head that asks you questions about your learning as you’re involved in some sort of learning task, questions like, “Okay, I’m going to do a reading now, what strategy works best for me?” Now you just might brush over that because you’ve done readings of this type a dozen times, a hundred times, whatever, but you’ve asked yourself that question along the way. “What’s my best strategy? What kind of a task is this? And monitoring: are my strategies working for me? Am I getting it? Can I paraphrase the last couple of paragraphs that I just read?” It’s a reading thing, but it works also in lecture. And then at the end, you evaluate yourself. “Well, let’s see. I had a goal, being able to recite five main points from this chapter, let’s see if I can do it,” without looking at the chapter of course. [LAUGHTER] So you evaluate your abilities, you evaluate your strategies. That’s really what it’s all about and it involves a great deal of talking to yourself.

Rebecca: So how did you get interested in talking to yourself? [LAUGHTER]

Linda: Well, I heard voices. [LAUGHTER] Well, how did I get interested in this topic? Actually, it was an accounting professor at Clemson that got me interested in it. This is 2007, right, this is a long time ago. And so she said to me, “What about giving a workshop on self-regulated learning?” In my head I said, “Huh? What’s that?” I’d never heard of it. And so I decided to go find out about it and it took me a few years to really get a workshop together on it and I decided, “Gosh, this is wonderful. This is learning how to learn. This is familiar to me,” because I’ve been talking to myself for years. [LAUGHTER] So I thought, “Okay, I’m not crazy. This is a learning strategy, a major learning strategy, one that you can use throughout your life.” And so I gave the workshop, I started giving workshops, like at the POD Network Conference—which is made up by people like me who go to this conference every year—and then I decided—well, I didn’t decide—a book publisher came up to me and said, “Please write a book on this, I will publish it.” Since I was in love with the topic anyway, I decided to do it. And so I did and delved into it deeply.

John: As you’ve described it, it sounds like part of this deals with improving student metacognition, but you note that it goes a bit further. Could you talk about the additional aspects of it?

Linda: Metacognition is the cognitive part of self-regulated learning, which is a major part of it. However, there are a couple of other elements to it that I don’t know that you could say are really focused on cognition. There is the emotional element to it, which involves getting yourself to be motivated and interested in the topic. Remembering, reviewing what your professor told you about the relevance of this topic, and thinking about it yourself. We can motivate ourselves, we can reframe a task for ourselves, and we can certainly reframe what is going on in terms of a learning experience. That’s a major, major part of it. The emotional part of the end is: “If you didn’t reach your goal, what do you do about it?” Do you give up, walk away, and say, “Well, I wasn’t born to do engineering,” or whatever the topic is. No, what you should say is, “Let’s try another strategy. Let’s look into possible strategies.” As instructors, we need to familiarize students with various strategies because they come to us—I like the phrase—“as feral children” in terms of the life of the mind and what they know about learning. We don’t have cognitive psychologists—unfortunately—teaching first grade or fifth grade, and so we need to equip them with how their mind works. There is one other element, a physical element, and that involves planning, monitoring, and evaluating your physical setting, where do you study best. If doing a reading or writing assignment, is it in a coffee shop, or do you have to essentially be in a soundproof booth where you don’t have any stimulation? How much coffee should you have? Or tea? [LAUGHTER] What kind of an environment should you set up for yourself—perhaps putting your digital distractions in another room. How should you schedule your breaks? Other things that you might want to consider is the amount of sleep that you have had because that can be a very important element of learning and writing. Some people study better to mild background noises as long as they are familiar music they’ve never heard before. You’ve got to try out these different things and find out your best setting.

John: In your book you describe how you became a self-regulated learner. Could you relay that story?

Linda: Yes, it was based on fear and terror. [LAUGHTER] As a child, I went to a private Catholic girls school—great education, but not in the sweetest of ways. From about fifth grade on, we had what was called recitation every single day in English and history classes. The nuns would ask a question and would randomly call out students’ names—we were in small classes so it wasn’t an absurd thing—and we had to get up and we’d better have the answer to the question. Now, not all the kids did, but I needed to be Little Miss Perfect because I wanted to get into college. Somehow I thought that the answers that I gave in fifth and sixth and seventh and eighth grade would or would not get me into college. So I learned to quiz myself while I was reading chapters or essays. Constantly quizzing myself. And another thing we had to do, usually later in the day, was called “exercise period.” We had 35 minutes to write an answer to an essay question that was generally related to the readings. And we couldn’t look at our books. It was out of desperation that I was trying different strategies so I could perform very well every day, and I was really quite successful. So when I was reading about self-regulated learning, I said, “My God, I was doing this as a child.” That’s why it sounded so familiar to me.

John: But that’s not the experience I think that the students we have entering our colleges have.

Linda: You are so right. You are so right. And we don’t do this to children anymore, okay? There is a good side to it—there really is—because you do buckle under and get very serious about your homework, very serious about your studies, or you look like an idiot the next day. Anyway, the sport I’m trying to think of is that sport where you get the ice or any kind of ice particles out of the way.…

Rebecca: Curling?

Linda: Curling! Curling, yes, we are curling teachers and curling parents. We try to clear the way for our students. We don’t want to put them in stressful situations. We don’t want to ask Johnny to read if Johnny might not be able to read—and I mean, get up and read well—and the problem with this is that students are denied the opportunity for achievement. And there is no achievement without the possibility of failure, there just isn’t. So students have no idea what fear and terror in school might be. There’s bullying and all that, but I mean from the learning experience. So no, they don’t have any kind of the experiences that I had.

Rebecca: How do we start coaching students then to become self-regulated learners if they’re coming out of this really different environment that’s much more supportive and doesn’t allow for failure seemingly?

Linda: They start failing in college. We’re still sort of, you know, curling them a little bit, but they are really facing a much greater challenge. They get insecure really quickly because they’ve been told how special they are and how smart they are, and then they begin to question that, because they’re not doing as well as they were in high school, where you could get an A relatively easily. Now, “Oh my, it can be really hard,” and then they start getting C’s, and then you have their attention. That’s a way that you can tell them that, “There are ways that you can get A’s, you did not learn how to study, here is a way to learn.” Does it involves a sort of effort? Sure, but it’s really just talking to yourself and deciding what strategies would be best for you, testing out strategies, seeing how they work, and you will be more successful. And there have been studies of students—like in developmental courses—that show that the students who are struggling the most tend to know the least about self-regulated learning strategies and start to do better if they use these strategies.Of course we’ve got to get them to use the strategies, we’ve got to explain these strategies. It can be life changing for them in the most positive way.

John: I think part of the issue is that faculty generally haven’t been taught these strategies themselves. They somehow found ways to be successful, so they become self-regulated learners, but faculty are the exceptions. They’re not the typical student, and they’ve never really been trained to teach students how to become more effective learners, in part because they never learned that directly themselves quite often. What can faculty do to be more effective in this way?

Linda: Well first of all, faculty have to realize that they’re the weird ones, and everybody else is normal. [LAUGHTER] So we have to stop projecting our learning abilities, our strategies, our interests in the life of the mind onto everybody else. We have to not only sell our material, but we have to equip students to learn our material. We don’t want to do that. We say, “They should know by now.” Well, guess what? They don’t. So what are you going to do about it? You’ve got to start from where they are. Teaching students learning strategies takes a couple of sentences every class. Now, if you really want to get into self-regulated learning activities and assignments with them, that might take a few minutes per class period, but you don’t have to do it every class period, and a lot of self-regulated learning activities can be homework, in which case they take no class time at all. This is so easy to do. This is why I think faculty have really been attracted to my book and why I’m asked to speak on it so often, because there are so many little things you can do that don’t take away from the content at all—rather, they reinforce the content—that make this huge difference in the performance of most students. You can’t always bring everybody along with you. There are some people—some students—who’d just as soon shoot themselves in their foot, but most do not. They find these activities so easy to do. They don’t take a lot of time and they get to know themselves and start doing better, so students don’t complain about this.

Rebecca: Can you describe what a couple of those activities might be?

Linda: Sure, absolutely. Well, let’s consider the different parts of the course and I’ll just give you just a few, some of my personal favorites. For starting a course for instance—starting and you can also end it with these sorts of activities—but one of them is a goal setting activity. You can assign this as homework, you can have students do it in class with students write on “How I earned an A in this course.” Now, you would be surprised and students will be surprised—C students and B students will be surprised—that they know what it takes to earn an A in a course, and they will come up with, “Well, I’ve got to come to class every day, don’t I? And in class I can’t fiddle around with my mobile device, and I have to start a paper sort of early and I have to keep up with the readings.” For many people, writing this down is goal setting for them. They think, “Well, you know, maybe I could do this, maybe this isn’t so absurd.” If you make a discussion out of it afterwards, the A students will say, “Yeah, I do these things, it’s not unrealistic.” And then the C and B and F students will say, “Well, let’s give it a whirl.” Then at the end of the course, you give them another little essay assignment, “How I earned an A in this course—or not.” [LAUGHTER] True confessions time, right? And so students assess how well they met their goals. Goal setting is definitely a part of self-regulated learning, the planning and then self-evaluation at the end. Another thing that you can do is you can give your students essay questions. If you give an essay final—or have any essays on it at all—you can give them the essays on the final to take the first day. This will not take much class time at all, because students will know very little, or they’ll try to BS an answer. So they will try but they can be really quite wrong. Now at the end, for the final, they correct their answers, and then rewrite these answers given the knowledge that they have gained throughout the course. This can be really interesting for faculty—for not just faculty… well, it can be interesting for them, too—because they can see exactly what they learned. So it is a measure of learning. Faculty will never get that comment on the student evaluations saying, “Well, I didn’t learn anything in this course.” Never again, that’s gone. So anyway, those are a couple of things that you can do. Little assignments you can make on the readings. Little reflection exercises like, “What did you think was the most important point in this reading? What surprised you the most? What connections can you make between what you read and your prior knowledge, what you already know? Or to your life? Or your emotional reactions to it, if the material is amenable to that?” So those are little reflections you can give on the readings. Another exercise, a self-testing exercise, is called “read, recall, review.” This is the best way to do reading. Forget about rereading, that’s what students really do… It’s really a waste of time. What students should do is to read a portion of the chapter or the whole chapter, put their notes away, close the book, and then recall as much as they can and write it down. Then they should go back and look in the chapter for what they forgot and what they might have gummed up. And they know that, “I didn’t really get that point.” And so they go back and look at it, and then they recall again. Read, recall, review. Studies that have been done on this showing it is so much more effective than rereading. It really doesn’t take that long, and then you actually have the material in your head, even in your long-term memory. You get retrieval practice, you get deliberate practice, so there’s nothing as good as testing yourself except—well, the nicest thing we can do for students is to test them. In lectures—I should say mini-lectures—it’s a good idea to have students do this. You stop, let’s say, every 15 minutes or so and have students do the same thing. Write down everything that they can recall, and then work with their neighbor to fill in the blanks—their own blanks—and ask any questions they have. First they ask their neighbor their questions, and then they ask you. This doesn’t take very long at all, maybe five minutes, but then you know that the students got it and can remember it. Again, most effective… studies done on this, too. So this makes students aware of their learning or their lack of learning. You can give students what are called active learning checks. You give your mini-lecture, and then you stop—and by the way, you can warn students you’re going to do this so they’re listening—and ask them, “Okay, what are the three major points in my last mini-lecture that I talked about in the last 15 minutes?” Then they write those things down—and it could be two things or four things depending—and turn them in. They don’t really have to turn them in, but you know, you might want to see them yourself. Then you reveal the three most important points, and they monitor and evaluate their learning skills. Now, students are motivated to want to learn how to listen to you, so they want to improve. According to a study that was done, they improve really quickly. The first time, 45 percent of students got all three points correct. By the third time, 75 percent of the students got these correct. Remarkable progress, really remarkable. Then there are meta-assignments. In a problem solving field like chemistry or math, we are denying students learning opportunities when all we do is mark the wrong answers as wrong or incomplete and then drop the subject. Students should be able to correct their mistakes to get half the points back, let’s say. In other words, they’re going to learn how to solve that problem if it’s the last thing they do. [LAUGHTER] Again, you give them some sort of an incentive, then they learn. There have been studies on this technique as well. It’s extremely effective. And students can learn not just from you, but in peer groups. Peers can help each other very effectively. There is a wrapper—they’re called “wrappers”—for an exam, a reflection that students do after they get their exams back where they answer questions like, “How did your expected grade compare with your actual grade? How do you feel about that?” So they have to look at the exam and your feedback. “How many hours did you study? Was that enough? What did you do while studying? Might you want to change your strategies? Why did you lose points? Were there any patterns that you see here? How are you going to study more effectively for the next exam?” This has been life changing for students because they’ve never thought about this before. They’ve never really looked at their exams, their mistakes. They drop them too, right? They don’t want to see what they did wrong. Yet these are the best learning opportunities possible, and they will remember them. We remember our mistakes, we learn from our mistakes, and it’s sad that we don’t stop and use those errors. These are just a small sample of self-regulated learning activities. I can give you many more. [LAUGHTER]

John: And there are many in the book, which we strongly recommend to people.

Linda: Yes, yes, yeah…

Rebecca: A lot of what you’re talking about seems tied to growth mindset as well.

Linda: Exactly, and this creates, this generates the growth mindset because students learn that they can learn, they can do better. Otherwise they feel like their learning is like the weather. “Maybe it’ll rain on me, and maybe not. [LAUGHTER] There’s really nothing I can do about it. Because it’s all about you, professor, you are responsible for my learning, just like the fates are responsible for the weather. [LAUGHTER] And if I’m not learning, you’re not a good instructor, or you’re pitching the material over my head, or your teaching strategies are wrong.” And so everybody else gets blamed, and then they start to realize, “Oh, I can do this.” Now, this isn’t the best news for them in the world because then they have to start taking responsibility for their learning. And that can be, for some students, a hard pill to swallow. For other students it will be very empowering, and what we want to encourage in students is that sense of empowerment.

John: And that’s especially important, I think, in freshman-level classes, because students generally don’t come in with that type of mindset. They’ve often been able to blame it on the teacher and do things over and over again until they get the grade they want or get the extensions and so forth with a focus on self-esteem in many classrooms.

Linda: Oh, yeah, self-esteem without achievement.

John: But it’s an adjustment. So if they come in with a fixed mindset, and they’re confronted with failure, it’s pretty easy to give up. So we need to encourage students, I think, to see failure as a learning opportunity as you’ve mentioned. As instructors, I think we have to somehow convince them of that, because they don’t come in naturally picking it up, but the techniques you’ve mentioned are very good for that.

Linda: You know, our whole society makes them feel they’re not responsible for their learning. Look at what happens in K-12. Students have to take standardized tests and if they don’t do well, who gets blamed and who suffers? The teacher and the school, and that’s nuts. In the final analysis, we teach ourselves. We are responsible for our own learning. Good teaching can make a big difference because we can be motivated or unmotivated by teaching. We can acquire learning strategies through teaching. So it’s not that students are just left adrift on their own, we do have to help them. We do have to put them in learning experiences where learning becomes attractive for them, or you can’t help but learn, right? But they’ve got to pick up that learning and run with it themselves.

Rebecca: So you mentioned the idea of encouraging students to see learning and the self-regulation as empowering. What about those students who are a little resistant to that because it’s surprising to them that they’re not getting it and they’re failing and that it’s going to be more work? What are some things that we can do to encourage those students to see things a little differently?

Linda: Yes, first of all, if they’re failing and they subconsciously want to—it happens, it really does —[LAUGHTER] there is not a whole lot you can do about it. They might need some counseling and they might need to get some help from professionals like psychologists. But again, it can be difficult for students to realize that the ball is in their court because it’s a whole different gestalt for them. The only cure for that is success—a little bit of success—where they start doing a little better, let’s say, on the quiz on the readings or they start being able to solve more problems. That’s really the only cure. And we are assuming that they want to be successful. Again, if they prefer failure, then they are responsible for their own failure.

Rebecca: Right, they’re the ones that are normal and we are not, right? [LAUGHTER]

Linda: Exactly.

Rebecca: Maybe that should be the refrain of this interview, right?

Linda: Yes. They are the normal ones and we are strange. And we always have been strange. We were the strange kids in school, too. [LAUGHTER]

John: In your book you mentioned the Dunning-Kruger effect as being a barrier to some students, that the students who don’t understand things as well often overestimate their understanding. How can we overcome that?

Linda: Self-regulated learning helps when we give them activities and assignments where they do self-evaluation, because the only way to learn self-evaluation is through practice, practice with feedback. And that feedback doesn’t even have to come from you; it can come from peers or a computer program. We don’t give students a lot of practice in self-evaluation, and they certainly haven’t had much of it in K-12l. But the nice thing is that when we have students look back to see if they met their goals, or to evaluate their study strategies, or to assess their mistakes and the reasons for their mistakes, it makes all the difference in the world. After low-stakes practice, you can introduce higher stakes self-evaluation assignments and see more savvy self-evaluations.

John: What recommendations do you have for faculty who’d like to start building more self-regulation? Are there small steps that faculty could take to get started on this path?

Linda: Absolutely. There’s a sense in which most of the assignments, most of the activities, are little things. Here’s a little thing you can start off the course with, I was talking about essay questions, but you can just have students do a little reflection the first day and then again on the last day about the subject matter, as in, “What do you think chemistry is? Why is it a science?” You can find out a lot about students’ misconceptions just by looking at these reflections. And then of course, they’ve hopefully corrected a lot of misconceptions by the end. This could take like all of five minutes the first day. There’s so many little things. Here are some ideas for experiential learning. It’s so easy for students not to make a connection between a simulation, an interesting role play, a service-learning experience, or field work to the course. So it’s important that whenever you do an experiential assignment or activity, students reflect on what they are learning—for a simulation, to look back and explain what their goals were, to evaluate how well they met their goals, to assess their strategies, to explain how their strategies changed and their responses to other players. It’s very important that students become conscious of what’s going on in their heads. Only by becoming conscious can they remember the strategies? [LAUGHTER] And then they can write them down and articulate them. You can have students do short papers associated with papers and projects where they record, while they are doing it, the process they are following. If you’ve given them a process to follow, they even have a skeletal outline of what they should be doing. This is a place also for self-evaluation. If you have students do a revision, oftentimes you give them feedback on what they should revise, and they may or may not read your advice. So you can have them paraphrase your feedback back to them and write out their goals for the revision. What are they going to do? What are their strategies for revision? These are just little things. Students start to realize the value of this. And again, this is an assignment where you can’t screw up. It’s not a test, it’s just a reflection of what’s going on in your head. Students like to learn about themselves. And this is like the reading reflections. This is no stress. How do you mess this up? It takes less stress to just write an honest answer than it is to make one up that sounds credible. [LAUGHTER] I want to make faculty aware that the activities don’t have to be graded at all. The assignments don’t really have to be graded. You “grade” them pass-fail. Students pass just by completing the assignment. Let’s say, you had them answering three questions, three reflections. Did they answer three reflections? Is it vaguely on the chapter? Okay it’s not about football, it’s something about what’s in the chapter. [LAUGHTER] And did they meet the length requirement? It’s always a good idea to give length requirements on these reflections because for students, length means depth. So if you ask them to write a minimum of 150 words, you know, they’ll tend to do that. Those who don’t fail.You don’t count every word that students write. You eyeball the reflection. Essentially you are “grading” pass-fail at a glance. It doesn’t take much time. Plus, it gets students to do the readings in the easiest way and most productive way for them. It’s all about them, and it’s not about us. We just have to hold them accountable in some very quick way. Even the longer assignments that you might associate with a paper or project can be graded pass-fail. You have to make them worth some points if you’re still on a point system. But there are alternatives, that’s what specifications grading is all about. You don’t have to use points. In any case, you do have to at least eyeball the reflections and give some value in your course however you are grading. That communicates to students that this is important to you, that you put value on this meta-assignment or assignment wrapper, as you might call it. The same thing with the post-exam wrapper, these reflections on this exam. You make students do it because it’s worth 10 points if they simply complete it and hand it in—even though it’s for them—and they will realize right away that there is some value to this. Again, for some students, it will be life changing in the most positive way. And they will start to realize the way that they’ve been preparing for or taking exams may not be the best. They will realize what they tend to do when they’re taking an exam, such as to misread the question, or to be careless, or to not budget their time, or to not really thoroughly study all the material. Cramming is not very effective. You don’t have to spend time grading these exercises or giving any feedback at all. They can give themselves their own feedback. If they did it and they get the 10 points, okay, that’s plenty of feedback for them. They did it. You regard it as their meeting the requirements of the assignment.

John: And this is a topic you cover in another book on specifications grading, which is also another book we’d like to recommend. We’ll include a link to both of those on the show notes.

Linda: The title of the book is Specifications Grading: Restoring Rigor, Motivating Students—it’s been found to be motivating—and Saving Faculty Time, saving you time. If there’s one thing we don’t have, it’s time. Time is really more precious to us than money. Otherwise, we’d be working in some venture capital firm or something. [LAUGHTER] But time is really quite a precious thing for us. So, in terms of these sort of assignments, for self-regulated learning assignments, they’re all what we’d call “specs graded.” You set out the specs, they’re very simple, and you just grade them pass-fail.

Rebecca: I think pointing out how it doesn’t have to be complicated for faculty is important because I think we all want students to learn. We all want them to be self-regulated learners.

John: We all want to give students feedback, but we don’t want to make it impossible for us to keep up with our work.

Rebecca: Yeah. Or feedback that’s going to get ignored anyways.

John: Right.

Linda: You’re worried about students reading the feedback, and our feedback is valuable. We’ve given it, we’ve taken the time, and so you make them paraphrase it back to you. And this could be a learning experience for us because we might be “misread.” Students might not understand something that we’ve said. Awkward—that’s my favorite one—a sentence structure is awkward. What does that mean? That student didn’t set out to write an awkward sentence. That in itself will not help them because they don’t know what you’re talking about, and this is most unfortunate. But again, it’s a learning experience for us and we can learn to express ourselves somewhat differently. Too often, students get back a paper from us and look at the grade, read the paragraph at the end of the paper, and put it in their “circular file.” They dump it. They don’t read that feedback, so how are they going to get better? So paraphrasing our feedback back to us can be a very valuable exercise for them. And you can let them gain back some points for it. I just think that faculty should look at themselves as responsible for helping our students learn. They don’t come to us with those skills. We can be the finest instructor in the world, have the most interesting classes, hold their attention, and motivate them, but if they don’t know how to process that material in their own minds, it’s all for naught. Now, maybe, hopefully, seniors have learned to learn their material along the line. And by the way, there can be different learning strategies for different subject matter. There are different self-regulated learning activities and assignments for problem solving mathematically-based fields, and different ones for the social sciences and humanities. There can different kinds of assignments, different kinds of readings, actually different kinds of lectures. So we have to respect that. But we have to become conscious of study strategies, learning strategies, our strategies, and other strategies that are out there. But self-regulated learning strategies, to my mind, they’re the shortest distance between two points. Shortest distance between ignorance and learning because it’s all going on in your head, and it’s so powerful. The value of it to students becomes evident rather quickly.

Rebecca: And it’s a skill that can help through a whole lifetime, not just while they’re in college and I think helping students realize that is also really valuable.

Linda: Like no other generation before, these younger generations are going to have to learn to learn on their own. They’re going to have to keep up with their field, whatever their field is, and they might have to—will very likely have to—pivot into another field because their first field might run its course. They’re going to have to learn on their own. They aren’t going to have employers holding their hand. Not at all. They’re probably going to have to learn online, where you really are responsible for your own processing, more so than you might feel let’s say in a face-to-face class, and for your own motivating as well. There needs to be more motivation than simple fear that you will go hungry and won’t be able to get a job. [LAUGHTER] Yes, students are going to really have to learn how to learn. If they consider that a bitter pill, that’s too bad. This is reality, this is life, and most of them have not learned that life is hard. Many of them are wondering where their next meal is coming from, but a lot of students have not. Students need to learn along the line that life is not easy, that nobody does curling on their path. And they will face challenges, but if they have the strategies for facing these challenges, no problem. They needn’t be paralyzed. They needn’t freeze.

John: You foster some really good advice and I think our listeners will appreciate this and it’s really powerful.

Rebecca: Yeah, I agree. We’re all wondering, what’s next for you?

Linda: I’m actually Director Emeritus. I’m actually retired from Clemson University. But you know how academics are, [LAUGHTER] they don’t disappear, they just sort of like fade away. So I’m trying to ease into retirement because it’s not an easy thing to do? Not when you love what you’ve been doing. But I have sworn off writing books. That’s progress! [LAUGHTER] I’ve written some articles and chapters in other people’s books, so that’s fine. And I’m still traveling to give keynotes and faculty workshops. That’s hard to give up because it’s interesting to go somewhere else, somewhere new.” And I still give webinars and podcasts. But eventually I won’t be doing that anymore. Ultimately I want to work with animals. I do love animals but I’m still busy doing this and still loving doing this, but also loving just as much not having to do bureaucratic tasks for the university [LAUGHTER] and not having to stay up until two in the morning doing my email. When I’m traveling, not have to worry about, what’s going on back at the office. So I’m not complaining about retirement. I really like where I’m at right now, but I know that I will eventually fade into the sunset. That’s okay because then I’ll reinvent myself.

Rebecca: Sounds like some self-regulation was going on there, I’m pretty sure. [LAUGHTER]

Linda: Yes, I’m trying, I’m trying to retire but not too quickly.

John: Well, we’re glad you haven’t, fully yet.

Rebecca: This was really great. Thank you so much for spending some time with us today.

Linda: Well thank you for this opportunity. I hope that I have helped some faculty members out there to help them help their students to achieve more, because again, we all do want our students to learn. We’re all in love with our material, it’s worth learning, and we just have to help our students do that. So thank you ever so much, and thank all of you listeners for listening.

Rebecca: Thank you so much.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

73. The Injustice League

Difficult conversations like those around injustice and inequity can be challenging to facilitate no matter the student body, but first-year students have additional barriers to overcome like establishing a sense of belonging on campus. In this episode, Dr. Margaret Schmuhl joins us to discuss how comic books and programming outside of the classroom can help first-year students develop the confidence to engage with complex social issues. Maggie is an assistant professor in the Department of Criminal Justice at the State University of New York at Oswego.

Show Notes

  • Mya Brown – Assistant Professor in the Theatre department at SUNY Oswego
  • Amy Bidwell – Associate professor in the Department of Health Promotion and Wellness at SUNY Oswego
  • ACUE – Association of College and University Educators – certificate of effective college instruction

Transcript

Rebecca: Difficult conversations like those around injustice and inequity can be challenging to facilitate no matter the student body, but first-year students have additional barriers to overcome, like establishing a sense of belonging on campus. In this episode, we examine how comic books and programming outside of the classroom can help first-year students develop the confidence to engage with complex social issues.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Margaret Schmuhl an assistant professor in the Department of Criminal Justice at the State University of New York at Oswego. Welcome back Maggie.

Maggie: Hi everyone.

John: Good to have you back.

Maggie: It’s good to be here.

John: Our teas today are….

Maggie: I am having a black ginger and peach tea.

Rebecca: Oh, one of John’s favorites.

John: It is [LAUGHTER]

Rebecca: I’m having English afternoon tea.

John: And I have Tea Forte black currant tea.

Rebecca: Oh, another one of your favorites.

John: I have many favorites.

Maggie: That’s a favorite of mine too.

John: We invited you here to discuss your first-year Signature Course here at as Oswego called the Injustice League: Crime, Justice, and Inequality in Comic Books. Sounds like a really fun course. Can you tell us a little bit about the course and how it differs from your other introductory criminal justice courses?

Maggie: This class was a lot of fun to teach. In the class we read various comic books, we watched different superhero movies and we talked about, within those comic books, what it means to have justice, to recognize injustice, and how society responds to crime and maintains or perpetuates various inequalities in those stories and movies.

Rebecca: How does that class differ from the other classes that you teach in your subject area? Because it’s a First-Year Signature Course, so that has particular meaning at our institution.

Maggie: Yeah. So the signature courses here at SUNY Oswego are about bringing a student engagement aspect to our academic course content. And so in this class there’s a balance of introducing our subject matter (in my case, Criminal Justice Studies) to the students but through a really fun way, but also working with students to help them with their academic success and getting engaged with each other and with our campus community.

Rebecca: Are these usually majors that are in this class or non majors?

Maggie: So I actually had a mix of majors and non majors. I had probably about 19 students. I had about 10 or so majors and so about half of the class were non majors. And they came from a variety of disciplines, including computer science, communication studies, and we even had some undeclared majors in the course.

John: And this is part of a broader initiative that we talked about in a prior podcast, and we will include a link to that in the show notes for anyone who wants to learn more about the first-year signature courses here. So one of the purposes of this, as you said, is to build more of engagement with the college community and also amongst themselves so that students will feel more connected. And one of the first things you did, I believe, was take them on a field trip. Could you tell us a little bit about that field trip?

Maggie: In Oswego, there’s a local comic shop that we ordered some of the students comic books from. In the very first class I asked them which comic books they were interested in purchasing and how many students we’re going to accompany me to the comic book shop here in town. And so to get to the comic book shop, you have to take the public bus, or at least if you don’t have a car, and many first year students don’t, they have to learn how to take the public bus. And so part of this field trip was not only obtaining some of the course materials for the class, but also getting the students familiar with public transit in the town and how to navigate a new place with them. So we arrived at the comic book shop and the owner was very gracious to us. She made us cookies and we had coffee and the students picked up their books and some of them even got some additional materials. We had a lot of fun. It was an amusing trip bringing a bunch of college students on a bus, and some of them their first time using public transit, and the bus drivers were even entertained by the group of us, so we had a really good time.

Rebecca: I think you also discovered the infrequency of the buses….

Maggie: Oh yes.

Rebecca: …in our town, right?

Maggie: Yeah…

Maggie: Oh yes.

John: Particularly on weekends.

Maggie: Yeah, particularly on weekends. We did wait about an hour for the bus on Sunday. So that was a little bit of a lag, but we made it through.

John: I should note that the comic book shop is actually owned by the wife of a former member of my department. It’s Arlene Spizman who runs that store.

Maggie: Yeah, Arlene was wonderful.

John: She’s a very nice person.

Maggie: I didn’t realize she had that connection.

John: In fact, I just finished a paper with her husband.

Rebecca: I’m sure it can be difficult to have an authentic conversation about justice in general, especially with a diverse population of students and maybe students that don’t know each other very well. How did talking about comic books as a way to get into the topic help facilitate those discussions?

Maggie: Comic books offer a different world for students to experience some of the concepts and some of the issues that we struggle with as a society. And so to be able to visually see these issues play out across the panels, it’s a place where students don’t feel nervous or threatened, it feels safe. They’re taking these comic books and they’re finding ways to relate with them and work out some of their preconceived notions or feel like it’s okay to start working on some of these biases and issues in society.

Rebecca: It seems like it has a lot in common with some of the other topics that we’ve talked about on the podcast before, like simulations and role playing, where it’s a place to escape the real world and talk about something really challenging in a so-called fake environment, but really they’re working out real-life issues and biases and all kinds of things that can be really difficult to talk about, but it’s a lot easier to talk about character that’s not real.

Maggie: Absolutely. Yeah. One of the comic books we read is an X-Men comic book called God Loves, Man Kills and we talked a lot about the concept of othering and what it means to target out and marginalize a group of people and in a lot of ways X-Men plays out what has happened in race relations in society and in other groups who have been historically marginalized. And so for students to consume this information through a comic book, they can better reflect on their own experiences and start to understand the position of others in society.

Rebecca: It probably also makes it a lot easier to make mistakes when you’re talking about that. I think sometimes students don’t want to talk about touchy topics because they’re afraid of offending someone or saying something in the wrong way but if it’s not about anybody real….

Maggie:Yeah

Rebecca: …then it’s not going to hurt someone’s feelings.

Maggie: Yeah, absolutely…. And another thing that we did in this course right from the get go was to set ground rules for discussion and conversation. And so I had the students come up with various guidelines for discussions and we would write them down so that we could refer back to them as we continued throughout the semester, so that they all understood that they had a responsibility to each other to make sure that everyone was comfortable and safe in this classroom. It really helped to facilitate a lot of these very difficult discussions in a very similar way that comic books themselves kind of help us talk about very critical and very upsetting social issues.

John: They also come in probably very familiar with many of these comics because they’ve seen them in movies, and some of them may have read some of these as well. Could you give us some specific examples of some issues in criminal justice that you were able to address using comic books?

Maggie: In terms of the classroom breakup, we have many students who were avid comic book readers. And we had many students who were somewhat interested in comic books but were more in tuned with the recent TV shows and movies that have come out of Marvel and DC Cinematic Universes. And so there’s, you know, quite a range of knowledge around this. But for the most part, most all of the students were interested in some kind of criminal justice aspect of their studies. And so, one of the concepts that we tried to discuss in this class was the issue of mass incarceration in society. And so what we did was we read a book called Bitch Planet, and in this planet, women are incarcerated on another planet. It has a lot of strong connections to some of the ways that society has restricted women’s rights throughout history and how the punishment of women has differed across time and across place. So to be able to see these concepts in a comic book and to talk about the parallels that exist in our society was a really, really cool process for the students.

John: Did you mostly focus on comic books they were familiar with, or you mentioned Bitch Planet, which was one that perhaps many of them hadn’t seen before. Did you bring in many that were things that they hadn’t expected or that they were less familiar with?

Maggie: There was really a mix. I even had some criminology textbooks that had various criminological theories played out in comic book form and we read a few of those to give us a baseline of various theoretical perspectives on criminal behavior. But most of the comic books I’d say we’re falling in the mainstream. I think that’s what students were typically looking forward to, but they really did enjoy the new reboot of Miss Marvel, with Kamala Khan and Bitch Planet and those were perhaps a little more on the periphery than Black Panther and X-Men.

John: How did students react to this? Did they generally find it interesting? Were some students troubled by using comic books? What about the imbalance between those students who were very avid comic book fans with those who were less familiar? How did that play out?

Maggie: Some of the very avid comic book fans in the class had a lot more context to really draw from when discussing histories of the Joker or Black Panther and the development of the character over time. But because comic books have become so popular in mainstream media, with TV shows on Netflix and pretty much a new Marvel movie coming out each year, that students really had a lot to draw from. Students didn’t need a great depth of knowledge of comic books prior to coming to this class.

John: For those students who were avid comic book fans, was it a little more challenging, perhaps, than they expected to look at some of these things through perhaps a more critical lens?

Maggie: I think that comic books, even if you don’t have a great background of reading comic books, or knowing the development of various characters, I think comic books allow for anyone to just pick them up and start thinking about them in a different way. They’re relatively quick reads, which really helps. Students can read them a couple of times and start to reflect back on some of the course concepts and theories that we discussed and how they apply and pull out those very specific examples. So I think the medium of comic books really provides a great range of abilities for students.

Rebecca: What are some of the things that you learned from teaching this class that you started employing in other classes?

Maggie: Oh, that’s a good question. One of the things that teaching the Injustice League has helped me with my other courses is to really think about being explicit with what I expect and what I hope students learn from various assignments and activities. In the Injustice League, students are entering college for the very first time and so they may not always understand why we’re reading this particular article or how it relates to the comic book that we’re reading today. And so for me to slow down as an educator and say that “Here’s why we’re doing this. Here’s what the research is showing us about why low-stakes testing in this class is a good thing.” That’s helped me in my other courses be more explicit with why I’m making decisions in various teaching practices.

Rebecca: Have you done anything else that’s related to bringing more comic books to other classes or field trips or some of the other things that brought the fun piece to the class that I think really energized the group as a whole?

Maggie: In my research methods class, I’m hoping that students will be able to assist in it by going out into the community and surveying people about dating formerly incarcerated persons. And so I think to get them out into the community and to start locating various areas of the community will bring some of that campus engagement aspect to it. In my crime-mapping class we actually started geocoding some of the locations around campus and so these are more upper-division courses but I’m trying to, even though the winter months make it a little more difficult to get outside, but trying to get outside of the classroom and really talk about how important it is to be connected to our community and to understand our relationship with the community.

John: I believe there was also some type of a video or a movie that you showed and I think other classes participated in that. Could you tell us a little bit about that and how it fits into other classes?

Maggie: There was a collaborative effort among some of the first-year signature course instructors to bring our students together in a common place. We ended up watching a episode of Luke Cage and this particular episode really spoke to a lot of the different courses that were being offered in the Signature Course program. Obviously my course, as one that deals with comic books and crime and justice and inequalities, Luke Cage is a very good example of many of the concepts that we talked about in class. But Mya Brown in the theater department also taught a class called Blackish Mirror and it followed the development of black characters on television. And so this was also a really good place for her class to talk about how various stereotypes that they had learned existed and/or were resisted against in Luke Cage. We also had a professor from political science and from communication studies, talk about political organizing or activism in Luke Cage, as well as narrative and the use of narrative in TV shows. We even had a signature course instructor in the health and wellness department… their class made snacks for the students to enjoy at the event.

John: Healthy snacks.

Maggie: Yeah, healthy snacks and it was brilliant. The students loved it. They created a snack mix that could be created and replicated by using ingredients found on campus. So that was a really cool way to bring in even a discipline that’s not necessarily focused on examining social inequalities in media to this event, and so it really spoke to a lot of students across various disciplines.

John: And we should note that, that person was Amy Bidwell, who was on an earlier episode. Were there any surprises in teaching the class that you didn’t expect?

Maggie: The class was a lot of fun. I don’t think I’ve had as much fun teaching any other classes I had teaching this class. It was really fun to pick up a hobby of mine, something like reading comic books, to bring this to the classroom and to start and challenge students to think about the media they consume in a new way, and how it reflects what we do in society and various values that society has. One of the most surprising things in the classroom was really how much of a community the students had at the end of the class. They had been speaking about other courses and working together on other projects and planning their course schedules for the next semester so that others would be in their courses together and so that was a really cool outcome of the class.

John: …and I believe you also opened an Instagram account for the class.

Maggie: I did and so you can follow it @the_injustice_league_oz… each word is underscored. I won’t say that I have many followers on the Instagram account but a lot of the students who did follow it seemed to really enjoy it.

John: And are you going to be teaching this again?

Maggie: I will be teaching this class next fall. So I’m very much looking forward to the next cohort of Injustice League members.

Rebecca: Did you carry on the superhero fantasy world theme throughout the class? You talked about rules for discussion or rules for engagement at the beginning. It’s almost like world building. Did you think about theming that more? Could you talk about how you might have done some of that?

Maggie: All of the designing my syllabus was all thinking of the class as being a part of a group of superheroes as opposed to just a group of students in the class. I even designed the midterm exam to look like a top-secret mission directive from their Professor S, which is me. The secret mission was about identifying various concepts that we talked about in class and applying them to a new comic book that we hadn’t read in the class. And so, in this midterm exam, they got to explore some of their favorites that we may not have gotten to touch on the class. It was a good opportunity for them to get creative and think about how these theories and concerns about justice translate across various stories.

John: And that way, you’re giving them some autonomy, but you’re also helping them develop transfer skills so they can take the things you learned and apply them in new circumstances, which is a really good practice.

Maggie: Even one of my students, when we were discussing moral panics, stopped into my office hours one day and was ecstatic because he had just realized that his journalism course was talking about moral panics, and so to be able to identify these concepts across disciplines was also a really cool outcome of the class.

Rebecca: You talked a bit about the class being really fun to teach. And part of that’s because you brought your hobby and your discipline together. But were there other things that made the class fun? I can imagine that you’ve all thought about yourselves as a part of a league. So maybe that you felt more connected to your students, or am I kind of projecting?

Maggie: Oh, absolutely. So, I called myself Professor S as a play on Professor X in X-men and so the students really loved that and they had a really good time with the way we even addressed each other in the class. The Instagram account even helped create more of a community by bringing in various pictures of each other doing or identifying various comic things across our everyday lives and interactions.

John: How did you first get interested in comic books?

Maggie: Actually, my first interest in comic books came from graphic novels and reading Persepolis as a kid. But, of course, I fell in love with the Marvel Cinematic Universe and I’ve watched pretty much all of the movies in chronological order.

Rebecca: Of course [LAUGHTER]

Maggie: I called it “research,” the summer I rewatched them all, it was a wonderful time. After reading a novel like Persepolis it was also something that really got me interested in criminal justice and society and inequalities in social life.

John: Did the students seem more open to discussing some of these issues having been exposed to them through comic books?

Maggie: Comic books reflect a lot of what is going on currently in society and they provide us a way to talk about really difficult topics of racism and sexism and things that occur and that people and students are experiencing in their everyday lives. So using comic books to facilitate these conversations is really important for students just beginning to question some of these processes.

John: One aspect of this course, as you said before, was to help introduce students to college life and help them create bonds and connections. But that also frees you up quite a bit because you don’t have a standard curriculum. Is this the first time you’ve ever taught a class where you didn’t have a fixed amount of material you had to cover in the course?

Maggie: Yeah, so this class was really flexible in that way. As I look back on the class, I’d say that it’s equally as important for us to be talking about some of the content about comic books and the sociological and criminological aspects of them as it was to help students become more connected to their community and to their campus, but also to ensure that they will be successful students at moving forward. And so this class really allowed me to work on some of their questions that would just come up, like calculating a GPA or registering for classes. And so the flexibility that exists in this class lets me respond to the students and their concerns in the moment and to occasionally tie-in some of those issues in current events to what we’re discussing in these comic books.

Rebecca: I could also imagine that it allows for the tangents that might occur as you start talking about something related to the comic book but you think it’s a valuable discussion. But if you have a finite amount of material in a finite amount of time, you might not be able to b go down those rabbit holes, but they can be such valuable conversations.

Maggie: Yeah, absolutely. You know, the flexibility of this course isn’t just to my benefit, but to the students benefit, where they can ask questions and we don’t have to worry about how much material we get through. We don’t have that curriculum that requires various elements to be covered and so the students can explore some of their questions in a very meaningful way.

Rebecca: I think it might be useful to just clarify that these classes are not part of any specific major, and they’re not a prerequisite for anything. So that’s what we’ve been talking about in terms of them being kind of freeing. I don’t know if we explicitly stated that.

John: Did you get to know the students better than you would in a typical introductory class?

Maggie: O ne of the good things about this particular classes is that there were only 19 students, and so it really allowed for me to get to know each of them individually and be able to see their personalities through our discussions and to have that comfort level with the classroom to talk about what made their day not so great today and what they really enjoyed about the weekend. And so to have that sort of informal relationship in a very formal setting was a really cool experience.

Rebecca: I know that one of the things that I’ve been thinking about after hearing many of the faculty who taught the signature classes talk about their classes is just finding ways to have some more of those informal opportunities in class, but also thinking very carefully about the content that I think that needs to be covered versus what maybe actually needs to be covered. There tends to be a disconnect, We think we need to cram in so much stuff. What are some of the key principles and things? And can we go into more depth for some of those if students are interested? And I’ve allowed that to happen a little bit this semester, and it’s been really delightful, I think, for everybody involved.

Maggie: Yeah, that was one of the things that I struggled with in the class. At the very beginning I was treating the class like a topics course and cramming, or at least planning to cram, a ton of information in. A few weeks in, I realized that it just wasn’t going to work for this type of class, that this class really did need time to facilitate these relationships and to help students learn and navigate their first semester here on campus. And so to have that flexibility for them to be able to explore their questions and concerns on campus and off campus was a important part of this class.

Rebecca: Sometimes I think that these functional aspects of being a student can get in the way of learning. So spending the time and just addressing those concerns that are preoccupying a student can free them up to actually think about the content and spend time investigating it. So, if they’re really concerned about figuring out their GPA or really concerned about making sure they’re registering for the right classes, addressing that concern up front can actually free them to focus on learning.

Maggie: Yeah, absolutely. And a lot of students, they come into the classroom and they think that college is going to be just like their high school experiences and so that studying in the same ways is going to be effective for them or that GPA’s and what credit hours are. There’s a lot of new information that makes transitioning to college more difficult one then, say, transitioning from their middle school to their high school . And so I think this does give them the time in class to talk with a faculty member to try to work out some of these questions in a way that they may not get to in their other courses and so it does certainly alleviate some of their anxiety around these issues.

Rebecca: When there’s not a context like that I think the option is going to office hours or something and that can be really intimidating, I think, for first-year students, or they just have no idea what office hours are for, which is another thing.

Maggie: Right, or how to book an appointment…

Rebecca: Yeah.

Maggie: …and all of that. And so I know many of the First-Year Signature Course instructors, including myself, tried to have individual meetings with students to break the barrier of setting up an appointment for office hours and how to draft emails to your professors and such and so I think it really helps them not be as nervous about getting the help they need and the resources that they may need in the future moving forward.

John: Because in the past if they were called into go to someone’s office after class…

Rebecca: Right, it was a bad thing, yeah.

John: Exactly. And so, you know, that’s something they do need to get past and it takes a while often and by then sometimes a little too late. So that’s really helpful.

Rebecca: Speaking of criminal justice, right? [LAUGHTER]

Maggie: Colleges, it’s a different culture than they’re used to and so to get assimilated to that culture is really important in many different ways.

Rebecca: Right, it’s like mentoring instead of a penal system.

Maggie: Right? [LAUGHTER]

Rebecca: Kind of a weird word flip there.

Maggie: Yeah, absolutely.

John: And one of the nice things about this whole program is it was set up as a randomized controlled experiment where students were asked if they were interested in courses and then they were randomly assigned or not assigned. And there’s some work that’s being done right now analyzing how their outcomes compared to outcomes of the students who were not in one of these groups, and they’ll be followed a bit to see how this works overall. So, I’m looking forward to seeing more, but the preliminary results they have, as were reported in the meeting this morning, were fairly positive.

Maggie: Yeah, retention was really good and so hopefully that’ll continue.

John: Semester-to-semester retention….

Maggie: for underrepresented populations, yeah. There was…

John: …was 100% retention semester to semester.

Maggie: Yeah.

John: It’ll be interesting to see if that persists, because that has not always been the experience of Freshmen.

Maggie: Right, and hopefully it does and I think one of the things this Signature Course program is trying to promote are those students and faculty relationships and that if students have a strong bond with a staff member or faculty that they’ll be more successful in all aspects of their academic life.

Rebecca: We always wrap up by asking, what next?

Maggie: Well next, I’m currently meeting with various new faculty members for the Signature Course program so we’re going to work our way through more course prep, and I’m very excited to meet the newest members of the Injustice League next Fall.

John: And you’re also joining the cohort of people in ACUE…

Maggie: Yes.

John: …which is starting up here on campus very shortly.

Rebecca: …another league.

Maggie: Yeah, another league of sorts. [LAUGHTER] I’m very excited… very excited for that as well.

John: T hank you. It’s been a lot of fun talking to you about this course and I wish I could take most of these courses.

Rebecca: I know, they’re always so much fun to hear about, but I think they give us lots of prompts and interesting things that we can start to consider in other contexts too.

Maggie: Absolutely. Thank you for having me.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

71. Small Teaching Online

Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, Flower Darby joins us to explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

Show Notes

Transcript

John: Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, we explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Flower Darby an Instructional Designer an Adjunct Instructor in several disciplines and the author (with James Lang) of Small Teaching Online, which is scheduled for release in June 2019. Welcome Flower.

Flower: Hi John. Hi Rebecca. Thank you for having me. I appreciate that. It’s good to be here.

Rebecca: We’re really glad that you’re joining us as well. Today our teas are:

Flower: I am drinking Builders tea. Good, strong cuppa here.

Rebecca: Sounds yummy.

John: We have some of that next door. I am drinking ginger peach gree n tea.

Rebecca: I have my Golden Monkey again today.

John: We ran a faculty reading group here in the Fall semester of 2017 based on Small Teaching. Many faculty found that to be highly inspirational and we had over 100 people participate in that. One of the things that came up quite a bit is how this might be applied online. So there’s a lot of people interested in your forthcoming book. Could you tell us a little bit about how this book came about?

Flower: Sure. So Jim Lang came to my campus—Northern Arizona University—in January 2018 and delivered a talk on Small Teaching and as we know, the book has been very impactful for faculty around the country and around the world. And while he was at our campus, when it came time for the question and answers, somebody raised their hand and said, “Sure, but how do you do this online?” And Jim’s immediate response was, “That’s the first question I always get at every talk that I give,” and he said, “I don’t know. I would need a co-author because I don’t know how to do this online, but that would be a great book.” So I thought about that for a few days and then I approached him and I said, “Pick me. I would love write that book with you. I can see the value of it, I can see the need for it.” So that’s how the conversation began.

Rebecca: How does this extend the approach that was used in Small Teaching?

Flower: Well, it follows the same principles for certain that there is learning science that we can draw on to help us make the everyday decisions in our teaching and learning that have really an outsized impact on student learning and outcomes. So there are little things that we do on a day-to-day basis and we can draw from the research to discover what will have the most impact. Again, understanding that in order for faculty to really be able to implement something new, it’s got to be feasible. It must be doable. The daunting overhaul of a major course redesign is so off-putting that most faculty won’t get around to it, myself included. When I have gone to multiple workshops and conferences and sessions or read about an approach. And I, “That is a great idea,” and I spend about five minutes thinking about how I might incorporate that into my class and then I say, “Too much work. Too much time. I don’t have that time available. I don’t want to implement something that’s only half baked,” and the idea gets left out. So in our online classes, there are so many things that we can do that are on that small scale but will have that outsized impact on our students’ engagement and their learning. And so that’s what this book sets out to do, is to explain a lot of those principles and draw on the research that we have to show faculty how they can make these changes in their online classes.

Rebecca: We’ve talked many times on our podcast about the lack of preparation for faculty teaching in general and that’s certainly true for online teaching. You might have taught a face-to-face class, and then all of a sudden, now you’re teaching an online class and boom, you have to figure it out. Can you help us think through what are some things that faculty can do as they’re new or getting used to being an online teacher?

Flower: Sure, and I think that’s really the point here. Centuries, millennia, compared to the way that we teach and we coach and we mentor face to face, or even as we’re doing here using video conferencing software, but it’s a real-time interaction. Well, online teaching is very, very recent, say 20 years or so. And faculty don’t have the experience that they bring into the physical classroom. You may have heard of the phrase of the apprenticeship of observation coined by Dan Lortie. And this is the idea that by the time a teacher steps into a classroom to teach, he or she has had years and years of experience in a physical classroom being a student and observing what happens and how things go and thinking, if somebody chooses to be a teacher, then they’ve clearly put a lot of thought into how they want to teach. Well, we simply don’t have that for online. I do expect this to change in coming years. But the fact is right now that most of our faculty have either never taken an online class or if they have, it may be a very limited experience, not the years and years that they came out of K-12 with. And the same, quite frankly, is true for our students. They’re also pretty inexperienced at an online classroom. And the way this pans out is that literally faculty and students both don’t know what’s supposed to happen in an online class. They don’t have the social norms, they don’t know what the classroom looks like. If you think about it, when you walk into a physical classroom to start teaching, you know what’s in the room and you know what’s supposed to happen. You see the desks or the tables, you see a lectern at the front, you see a whiteboard or a projection screen, and students and faculty understand what is supposed to happen. Students go and sit in the desks, they face front, they wait for the faculty to come to the podium. It’s rare that a student would walk into a classroom and at the beginning of the hour, just step into the lectern. Students know that’s not what they do. But my argument is we don’t have that kind of social norming convention for online classes…yet. I think it’s coming, but right now many of the people who find themselves in our online learning environments go into that space, and they don’t know what things should look like, they don’t know where the light switch is, they don’t know where the desks are, where the whiteboard is. So just that whole lack of experience is rather disconcerting. And it’s hard to know what to do. Faculty don’t have experience—they have haven’t seen models, students are equally unprepared—so there’s a lot of work to be done here just to understand what should happen in an online class, what the furniture is, where it should be to facilitate learning. That’s where those gaps happen for faculty. You ask, “How can faculty prepare themselves?” I could talk for days about that question. It’s a growing need and some institutions are beginning to recognize the importance of doing a much more thorough job of preparing faculty to teach online. But I will argue that those institutions are still pretty few and far between. I would say, based on my research and my experience, the vast majority of faculty who are teaching online have not had specific development in that area. They have not observed peers’ classes. In fact, what can happen is a negative effect. Very commonly, when faculty begin teaching online, they are handed somebody else’s content. We’ve seen that happen and that’s a mercy in a way because that way faculty who are new to teaching online don’t also have to develop the course. But what can easily happen is that the content that might be given to a new faculty member might not actually be exemplary in the design and the delivery of that material. So then what happens is the only experience that faculty get is observing the content and the structure of a less than ideal example and then that’s the model that they have and they think, “Oh, I guess this is how it is”. So work can be done on developing better exemplars, better development programs. I believe, as faculty are coming out of online graduate programs down the line a little way, I believe, will have better experienced faculty and students. A lot of research going on in this area, but that work is all to be done.

Rebecca: As you were talking, I thought, you provide a nice model, it’s a nice way of thinking about it, you don’t know where the furniture is.

Flower: Mmm-hmm

Rebecca: It sticks with me. I was thinking about that the experience that a lot of faculty and students have is more in the realm of social media and so they’re looking for cues that are similar to those kinds of environments. The activity that’s happening in those environments is really different than the kinds of activities we would expect to happen in an online platform for learning.

Flower: Right, that’s a great point. We interact with other people so much online and on our devices using social media and what’s interesting to me is that we can really engage with people in those online spaces. Somebody tweets something that’s a little bit incendiary or provocative and you get all kinds of people jumping in and commenting and you know, sometimes things get heated, or a really heartwarming moment is tweeted or shared on Instagram, and people are all over that post. But the opposite is kind of true in our online classes. Indeed, I feel like we could bring in some of the techniques from social media into online classes. I’m not saying that faculty should all have a component of Twitter or Instagram in their online classes, but what I’m saying is that it’s possible to deeply engage people in online interactions. And that’s not a feature that, I would say, generally characterizes online classes—we usually hear the opposite, that it’s not engaging, it’s difficult to drum up those discussion posts—and I feel like if we could draw some of those principles from how we interact with people online, in social media, of using our devices, if we could bring those into the online classes right away, we’ll see more engagement and engagement precedes learning. Students have to want to be there in order to learn when we’re engaging them and if you could imagine posting a discussion post and then you can’t wait to see what people are responding. We do that all the time on Facebook or Twitter sending something out and then, “Oh let me see! Did people like that? Did people say anything?” And we just naturally are drawn into those spaces to check and see what are people’s reactions? Well, if we could design that kind of a discussion board for online classes, where it’s so interesting and engaging that people want to rush back and see who’s talking to them, who’s replying to them, that would go away way to improving the online learning experience for both faculty and students.

John: That’s not an experience though that many people teaching online find in their discussion forums. Are there any hints or tips that you can give people to make their discussion forums a bit more engaging so that students don’t wait until the last minute to do the standard three posts or whatever is required in that course?

Flower: Great question, John, and a big one. And again, thinking about Small Teaching ways of making small changes, I heard of an example recently where faculty asked students to reply to their peers posts using a GIF that just represented—one of those funny moving little images that sort of expressed—their reaction. And that’s an example of bringing in new ways of engaging and it’s not rocket science. It’s also perhaps a little more fun, which is important to bring into an online class. A great way of sort of getting students to think differently. But if that idea doesn’t resonate with you, maybe you might want to try offering options in your discussion board questions. I’ve supported over 100 faculty, I might even say, hundreds of faculty in the design and development of their online courses and what I see sometimes is one question for students to answer and oftentimes it’s kind of black and white. It’s hard to discuss a question like that. So first of all, craft questions that are discussable, that there’s some debate around that you can make different arguments or points of view. Tie those questions to students’ experiences. How is the content impacting them personally? Where do they see these concepts in their own life and experience? And, even better, provide three or four different questions that students could choose to respond to and then ideally, everyone isn’t all talking about the same question, so that’s more of a natural way of fostering some conversation in an online discussion.

John: One of the nice things about tying it to personal things, I would think that that would also help build more of a sense of community within the group because the students get to know each other a little bit better, which may affect their engagement in other activities,

Flower: Right. Anything that we can do to increase the value and the relevance of what we’re asking students to do online is hugely impactful, and it doesn’t have to take much. I have a colleague who teaches an online First-Year Seminar course, which in a way is a bit of an oxymoron because First-Year Seminar courses are often designed to really hook in our first-year students who are transitioning to university life, but she was tasked with developing and teaching a really highly engaging and supportive Freshman First-Year Seminar class. And one of the things that she does is she brings in a discussion board and one of the prompts is, “If you could be a superhero, what would your superpowers be?” And again, maybe on the surface some people might think that’s a bit trivial, but what she’s doing is she’s getting students to talk about character traits and hero qualities and concepts that rely and relate to the material that they’re engaging with… yet in a fun and a more personal way. And it certainly does a lot to foster those relationships that are so important for online classes to build that community. Absolutely.

Rebecca: I think one of the methods that we hear a lot about in terms of online learning is the ability to do quizzing and retrieval practice and interleaving through quizzing. But are there some other ways that we can integrate some of these evidence-based practices that aren’t maybe the typical solutions that we tend to think of online?

Flower: I think one of the most underutilized functions of the Learning Management System is what we call adaptive release or conditional release. And I actually want to pause here and say that these Learning Management Systems have come a long way in recent years, and they still have a long way to go. [LAUGHTER]

Rebecca and John: Yes. [LAUGHTER]

Flower: For many faculty and students, the functionality is lacking, the furniture is stark, they’re not attractive places to be and as I said earlier, engagement precedes learning. If you were to ask yourself for online faculty who are listening, “Do you want to be in your online class?” I suspect many faculty would struggle to answer with a resounding “Yes.” And so here’s a shoutout to our LMS developers to think about space design and the experience of students and faculty in these spaces. Having said that, there is some very interesting functionality that is oftentimes underutilized and I would argue that’s because, again, faculty may not have the preparation and the exemplars to begin teaching online. With adaptive or conditional release—it’s called different things in different systems—you can set a task that then opens up the rest of the content in that module. And I love to use this. You can use it equally effectively at the end of an online module or at the very beginning to open the next module. Now what you can do with this is you can embed retrieval practice exercise. Or, you know, drawing from Jim Lang’s book, Small Teaching, a predicting exercise works equally well. A curiosity provoking exercise, and all it really has to be is an assignment where students submit whatever it might be. A two sentence summary of what their big takeaways were from the previous module, or predicting what might be in the coming module or posing some questions… this can be written, it could be a recorded submission for students who might find it easier to talk through their ideas. Once students submit that element, then the rest of the module opens. Before they do that they can’t access any of the content. These things don’t even have to be graded, you can just set them to be worth zero points, but they are mandatory because the students can’t proceed with the content until they submit them. So when you think about that feature, there’s a lot of creative things that you can do that don’t impact faculty grading time. That’s a big tenet of the new book is we can’t overburden faculty with grading and yet tie into those practices that we know from the research are effective.

John: One of the chapters of your forthcoming book is on fostering student persistence and success. Could you give us perhaps a strategy or two that might be useful in encouraging student persistence? Because I know one of the problems in online classes is they often have higher drop, fail, and withdrawal rates than face-to-face classes. What are some techniques that faculty can do to help improve student persistence in the class and the program?

Flower: That’s right, great question. As you point out, the attrition rates in online classes are remarkably higher. And we also find that for students who are less prepared for higher education, if they don’t succeed in that class, then the odds increase dramatically that they won’t actually persist and attain a college degree, and that’s a problem. But as I was saying earlier, a global concern is that online classes are not nice places to be. And if your listeners have any pushback on that, please feel free to reach out and engage with me on that assertion. But what can we do to just make the place a little more pleasant? How can we be warm and friendly and supportive and encouraging? How can we allow our humanity—even our personality—to show through? I was speaking with a good friend and a colleague of mine just a few months ago when I was delivering a little talk about this book and he was telling me though he’s been teaching online for 10 years, that he’d never thought of just being himself in his online class. And he explained to me that he loves teaching in person—he’s quite a character, super dynamic, very engaging, funny, loves to interact with his students in the classroom—and yet, he told me when he goes into his online class, it’s like a robot. There is no trace of his personality. And other people are saying this too, just be yourself in those online classes and make a deliberate effort to infuse warmth. But a specific strategy that people might want to try is to assign a goals contract as one of the items that are due in the opening module—or the orientation module—and a goals contract, you’ll see different kinds of variations around, but here’s the two pieces that I really like. A lot of people are talking about assigning sort of a memo of understanding or a contract where students agree that in this online class, they should schedule set times, they should plan on X number of hours per week, they should reach out immediately if they have questions. People are doing that. I like to embed a different element as well, which is to require students to set a couple of goals and it can be literally two. What are two goals that you have for your learning, or your success, your ability to earn an A in this class? And then an interesting twist is to ask students to identify one potential challenge. It’s still the case, I have my students all the time saying, “Well, my computer is in the shop, [LAUGHTER] it’s sort of all of a sudden, it busted and now it’s at the technician and I can’t do my online tasks.” So helping students to think in advance about a scenario such as that and of course, in that particular case, many campuses have computer labs or libraries where students can go and access another way to get into the course but maybe they haven’t thought about it in advance. So in the goals contract, ask students to set two goals, identify one potential challenge that might come up, and identify a strategy for how they can address that particular challenge. And certainly, identifying one challenge is not going to cover the range of things that happen in life during the course of an online class, but I think it sets the tone to get students thinking that one little hiccup doesn’t mean that we’re all done with this online class and we just have to sort of fade away and stop participating. And then what you can also do, periodically throughout the class, is you can ask students to revisit those goals that they set for themselves. How are they doing with that? What kind of progress are they making? Are there some strategies that aren’t working for them? Do they need to recommit to the intentional and deliberate scheduling of their class time? Just helping students be very explicit about what their plan is to succeed and finish the course.

Rebecca: What I like about what you’re saying is, it switches from really having the faculty member impose everything, and have the students be co-authors of the class to some extent, and they have some ownership over the space, which generally means that they’ll probably commit more.

Flower: What we know about online learning is that students must have a higher degree of self regulation, self direction, they must be more motivated, and be able to manage their time well. And if students don’t have those things, it’s much less likely that they’ll persist and finish an online class. And yet, when you think about it, online classes work directly against a student’s ability to do those things and here’s what I mean. When you are teaching in person, when you’re a student in an in-person class, you know that every Tuesday and Thursday at 9:30 you’re supposed to be in the classroom and it’s a natural way to help students hold themselves accountable for doing the work. Now I know sometimes students come to classes and they’re not fully prepared, but there’s still that built in mechanism where they’re going to be in the same room with their faculty member with other students. There’s a social element of accountability that’s like, “Well I know I’m supposed to show up and I should have my stuff done,” or “There’s a test next week and I need to be ready.” Well those real time interactions and those interactions with physical people don’t tend to happen in the typical asynchronous online course. Very often—I would say 99% of the time, probably—an online student is sitting at home by himself or at the coffee shop by herself. If she has a quick question about something, she can’t do what she does in the classroom and say, “Hey, did you understand what we’re supposed to do on that particular assignment?” or “Hey, faculty member, can you just re-explain that? I’m not quite there yet.” There’s no way to get that immediate response, that immediate quick guidance that might take two minutes in a physical classroom. So, students don’t have the accountability, they don’t have the physical presence of the instructor or the student, and so we have to go above and beyond in our efforts to build in structures that help students develop the kinds of self-regulated skills, the kinds of self-directed learning skills. Many of our students are not coming in with those skills already but we know students who do have those skills will be much more successful, so let’s build that into our curriculum. Let’s help them develop some of those, let’s talk to them about the importance of monitoring their own learning, and let’s structure exercises that will help them to do this. I’m pulling a lot of this material from Linda Nilson’s book—It’s called Creating Self-Regulated Learners—Although that book is not necessarily focused for an online environment, I think it can be hugely helpful to our online students to be very transparent with them about the importance of developing these habits, these behaviors for success. And as I said, structuring exercises and graded assignments that help them to do that, to hold them accountable.

Rebecca: Following up with what you just said, there’s a chapter in your forthcoming book called “Creating Autonomy.” Can you talk a little bit about small ways that we can give students autonomy in the classroom and in an online space?

Flower: Sure. And again, let’s be sure to keep that focus on small, doable, feasible changes, things that you could do in maybe a 15-minute work session and have it rolled out for your online class. One thing that we could do is to develop a self-enroll group structure. Many online faculty like to bring in collaborative learning tasks to, again, foster that community and the peer to peer instruction and learning that is so important as we know, but I think oftentimes we sort of assume that what we should do is purposefully group students, and there’s certainly value to be found in designing purposeful groups. But what can also be very interesting is to allow students to enroll themselves in groups that might cover a range of different topics. For example, sometimes I teach Educational Technology online classes. And if I were teaching that class today, I might offer five different groups that students can sign up for on a first-come first-serve basis. And one might be virtual reality, and one might be mobile learning, and one might be writing in digital spaces. So students could naturally choose a topic that they’re more interested in pursuing and when students have that level of autonomy, to make that choice of what their going to focus on, that’s one way of embedding just an opportunity for students to exercise that autonomy. Another even easier way I’ve already mentioned here is to offer students a choice between whether they want to submit a written task or whether they prefer to record on video or audio. Students carry these amazing devices in their pockets all the time with high- tech recording equipment embedded right in them. And students love the freedom of just being able to talk through their ideas, their responses, you can get a much more authentic response from students. Teach them how to use the recording software, or how to upload the video or the audio clip into the LMS and now you’ve got an easy choice that you can give students. If you prefer to write this, go ahead. If you prefer to record it, do it that way.

John: One thing that struck me is I used VoiceThread last year in an online class and I expected they’d actually use the video option with it very often. I gave them the choice of whether they use just voice or voice and video or use a video recording and yet none of them ever presented on video, which surprised me, given how common that is in social media. Why might that be?

Flower: Well, sure. I also require video discussions in some of my classes. And what I have learned is that people are nervous, especially in an academic setting about how they come across on camera. I feel like audio is a little bit less threatening, but sometimes people don’t like the way they look. And, you know, faculty too. [LAUGHTER] A lot of faculty are uncomfortable with recording on a video, and yet, it’s the way of the future. So right now, currently, I’m teaching my graduate level class on technological fluency and leadership. I require those video discussions and I say to them, “Are you nervous about doing this? Well, I want you to do it anyway,” because video interviews right, over Skype, or Zoom, as we’re doing here today, or video resumes. These are a thing that are happening and helping people to get more comfortable with showing their face on camera. I also talked to them a lot about the importance of seeing their peers faces and how much we can learn just from that. In the program that I teach now, students tend to take classes with the same people. But in my class, they always say, “I’ve never put a face to a name, how nice it is to see you,” and it makes a huge impact in terms of that community element. But I talk to my students very explicitly. Now, it’s also really important to think about situations where a student may not want to represent their face and there can be very good reasons. I had a tragic situation just last year where a student did not post the video, she posted a static picture of herself. I came to find out at the end of the semester. When she did that I was like, “Oh well, dock a few points, whatever. That was weird. Why’d she do that?” I later found out that she had been in a domestic abuse situation and she was ashamed of the way that her face looked because it was still very visible—the damage—and it just struck me to the core. An arbitrary decision that I made that it’s so important to talk to each other and look each other in the eye and she had a really, really strong reason for not wanting to do that. So back to that topic of offering a choice, what I do now is I tell my students, “If there’s a really good reason that you don’t want to show your face on the video, please send me a quick note. You don’t even have to tell me details, but just explain that you’re going to choose to do this other thing instead,” and posting a static picture is still pretty effective. So I think it’s very important to remember that our online students are people. They have lives and we need to be thinking about the decisions that we’re making in our teaching and how that might come across to a student… how it might induce anxiety in ways that we never anticipated.

Rebecca: One of the things that I wanted to follow up on and see discussed, the self-enrolling groups and collaborative work online. I think we have clear ideas about how that might work in a physical classroom, but not always a good clear way of how we can coach students through collaborative learning online. So even small, quick things that came up in Small Teaching, like Think-Pair-Share, you can envision how to do that in a classroom, but maybe have no idea how to do that in an online classroom.

Flower: It’s a great question, Rebecca. And I know that there’s actually a lot of pushback from online students and sometimes online faculty about the value of collaborative learning activities. It just so happens that my husband is in an online Master’s program right now and so I’m living with the student experience. And it’s frustrating to our online students—many of whom are not traditional 18 to 24 year olds, they might be returning adults—and one of the reasons (in fact, a primary reason) that our online students choose that modality is because they have busy lives. A big percentage of our students have jobs, families, obligations, and they need to do their work when they have time. That might be 8 pm, that might be 11 pm after all the kids are in bed. It might be 6 am, I like to do my online class at 6 am. When you require students to work in groups in an online setting, you’re removing that degree of scheduling flexibility that students value in an online class. So if you choose to require online activities, I have certainly moved towards lower stakes and opportunities that don’t require real-time meetings between students online. And you mentioned a great one. Think-Pair-Share can be set up in an online class. So there’s lots of ways that you can do this but the first thing that came to my mind is you could set up groups of two, and you could auto-enroll students in a group of two, and then they have their own individual discussion boards. In most Learning Management Systems when you have groups you can have kind of a private discussion board where students can interact with each other there, or I’m a big fan of letting some of the learning come outside of the Learning Management System. So let students know who their buddy is, have them exchange phone numbers, and they can just talk on the phone. Sometimes we forget those simple solutions. But a Think-Pair-Share—and so many ways that you could set this up, you could change it from module to module so people are always working with somebody else—just share an idea, discuss something, take it offline, come back and just write or record a quick summary of how that interaction went. When it’s not such a high stakes assignment, students can better engage in those opportunities. It’s so much easier to find 15 minutes to talk with one person than it is to find an hour with four working adults who all have family obligations. So I love the idea of lowering the stakes and embedding lots of little opportunities for students to work in pairs or in groups of three where it’s easier to coordinate. There’s less pressure about the online group member who never does the work—sorry, but that’s a thing—and just help students see other ways of interacting. Now, with my instructional designer hat on I want to remind us of the importance of making sure that online collaborative work aligns with the outcomes of the course. Very important to think about why you’re asking students to work together. Does this actually relate to what you want them to learn and get out of the course? Very important to pause, ask yourself some of those questions before you randomly assign group work because we should have group work, which I’m guilty of doing. [LAUGHTER] It’s an easy thing to do, “I guess we should have group work,” but really pausing to think carefully about the purpose of that. And then again, maybe thinking creatively about those lower stake ways of connecting students and facilitating some more authentic interactions. Maybe they’re going to text each other. That’s fine, they’re talking. We do a lot of talking on text these days. Help students connect in ways that are not so stilted, which is often what we see in the use of the discussion board and the LMS.

Rebecca: I found that too, I use Slack a lot in my classes, because it’s a common platform for designers and people in that realm to communicate professionally, and they love it. It’s convenient, it’s on their phones, takes it away from a clunky interface…

Flower: Sure.

Rebecca: .. some of the LMS’s have and it’s really productive. And they’re able to do that midnight chat with each other.

Flower: Yes, absolutely. Again, let’s think creatively about tools that students already have. I honestly believe that a lot of Learning Management Systems actually raise barriers to student learning because most of them—although this is getting better—most of them don’t have a super robust mobile app and so a student, really to engage with coursework, has to find a place where they can sit down and log into the computer and access the course and jump through a million hoops before they can even get to where the learning is. Whereas if we take some of that learning into apps that they’re already using or things that they’re doing on their phone anyway where it’s in their pocket, we can communicate in real time. Now I need to exercise caution here because many faculty think, “Oh great. I’ll do Slack, and I’ll do VoiceThread, and I’ll do Flipgrid, and I’ll do Twitter, and I’ll do Pinterest, and it’s just going to be so interesting and fun.” Well, if there’s a reason for using some of those tools, absolutely. If those tools are just shiny entertainment—bells and whistles—then you may want to think again. Another important consideration if you’re asking students to use tools that are not in the Learning Management System is whether those tools are fully accessible for students, whether there’s any fee that’s involved, whether students might have to set up a new account with a new password, that might just be a hassle. So really you want to think carefully about what you’re asking students to do. Are the tools fully accessible and usable and cost friendly? Do they support your learning outcomes? And yet, if a tool that you’re thinking about using passes all those tests, then by all means jump right in. This semester I’m using Remind which is the simplest tool on the planet and the most effective. [LAUGHTER] It’s more in use in K-12 currently than in higher ed. It’s simply a text app that anonymizes people’s phone numbers. So I invite my students to sign up for my Remind list. I don’t require it. But then I can easily send a quick little 140 character reminder, “Don’t forget this assessment is due on this particular day,” or “New content has just been released. Login when you get a chance.” The message goes right to where the students are and because I make it optional, nobody is required to have the annoying instructor on their phone all the time. But students who want some additional support with managing deadlines and the class experience really appreciate the use of the simple tool called Remind.

Rebecca: I think one of the things that we know from a lot of evidence-based practices and books that have come out—including Small Teaching—is that frequent feedback is useful. But we also know that frequent feedback can seem really daunting to a faculty member, and time consuming. So are there ways that you would suggest managing some feedback opportunities online, but keeping it easy, quick, and reasonable?

Flower: Sure. Another great question. Another underutilized approach—at least in my experience supporting the faculty that I work with—is the ability to embed feedback into auto-graded multiple choice or true- false types of quizzes within the Learning Management System. So in most of these systems you can design feedback that will show up for students as soon as they submit the quiz. You can set those quizzes to show students which questions they got right or wrong and in the wrong answers you can embed feedback that says, “Please review pages 32 to 35 of this chapter. That is where you’ll find this information.” Similarly, you could encourage or embed challenging feedback and by that I mean, “Great, you totally know this material. If you’re interested in learning more, you may want to check out this website or this resource,” to offer students a range of experiences and engage students at their different levels of experience with the content. To be fair, setting up that kind of embedded feedback takes a little bit of time in the first place, but many of us teach those online courses over and over again, and once you’ve done that work, you can benefit from it time and time again. If you’re not sure how to do that in your Learning Management System, just about every institution has a Learning Management System support team with instructional designers or system admins, help desk folks who can walk you through the creation of that kind of embedded feedback. And it’s timely, it’s right there when the students are thinking about that problem in the first place, it’s relevant, and it’s a great way to automate some useful feedback for student learning.

John: You have a chapter in this forthcoming book on developing as an online instructor. Are there some general suggestions that you can give to faculty who’d like to improve and develop new skills or improve skills as an online instructor? Besides buying the book, of course. [LAUGHTER]

Flower: Right. Great question. Again, what this comes down to for me is that it’s just new. It’s just new for a lot of people. And to be honest, I suspect that many online faculty didn’t really set out to be great online faculty and many faculty are not finding the experience quite as rewarding as they might find the classroom experience. In fact, I have some data to back me up on that. The 2017 survey of faculty and information technology from EDUCAUSE Center of Analysis and Research found that of over 13,000 faculty respondents, 91% said that they don’t prefer to teach online. 9% said, “Great, I love to teach online.” That’s 91% of us who would rather teach anywhere else. [LAUGHTER] So how can we cultivate that joy, that buzz that we get in the classroom? We love teaching. If we didn’t, we wouldn’t be doing it because we don’t get paid enough. How can we cultivate that for ourselves? Now a barrier or a common challenge is time. Who has time to go and learn how to do a whole new skill? It’s different than teaching in person. But there are, again, small things that we can do to increase our awareness. One of the most effective things that you can do as an instructor is to seek an experienced and a thriving online instructor and ask to shadow that class. Ask to be added into that class shell and just observe. How does that person interact with students? What are the structures? What is the teaching? What happens while the class is in session? That can be hugely impactful, it’s usually free, [LAUGHTER] and faculty can invest the amount of time that they have. In fact, this is how I first got started with online teaching over 10 years ago, is before I was going to teach a class. Luckily, this offer was made to me the semester prior to my first online class, just to observe another class and see what happens in there. Simple structure could easily be set up for faculty who are scheduled to teach a new class in the Fall, have them observe or shadow a class in the Spring or the Summer, and yet an often overlooked solution. Certainly there’s lots of online resources. There’s podcasts like this, there are blogs that people are writing about innovative things that they’re doing, but sometimes just finding a thriving online faculty to interact with, shadow, observe, be mentored by, can be the most effective way to learn how to do this better.

John: I even sometimes encourage faculty to join a MOOC, because often you can find some interesting practices there that scale without necessarily requiring much effort on the part of the instructor.

Flower: That’s right. That’s one of my other recommendations and I hope I haven’t given all of the book away here. [LAUGHTER] But one of the other recommendations in that chapter is just to take an online course in whatever form that you can. Whether it’s a MOOC, a lot of organizations like the Online Learning Consortium, Quality Matters, offer online professional development opportunities for faculty. Even if it’s not about teaching online, just go take a class that is online. Or maybe personal interest. Sign up and take Spanish online. And having the experience as an online student is hugely impactful to help you understand what your students are going through. Even as a faculty if you’re taking a course and you’re reading the instructions going: “Now, what am I supposed to do with that?” Immediately, you have much more clarity about what your students might be experiencing and then you can take steps to address those kinds of gaps or areas of concern that might be in your own class… that you may not have previously seen before.

Rebecca: I think the recommendation of taking a course outside of your normal domain or area of expertise is key because you’ve got students who are in an environment they’re not familiar with, with a topic they’re not familiar with. And so to kind of simulate that, I think is key.

Flower: Right?

Rebecca: I know I’ve done that in the past and it’s like, “Oh yes, I forgot what it was like to be a beginner.”

Flower: Absolutely. In fact, I had a really interesting process or experience this past fall semester where I was supporting a redesign in a large cap biology class of liberal studies—or general education biology class—large enrollment. My background is in English literature… the humanities. I don’t think I ever took a hard science class in college because I did an honors program where we could do more sort of ethical concerns related to science. But I went to that class frequently throughout that semester and I clearly remember the first day. 240 students and me and I was sitting in the lecture hall with the students and it was just very, very impactful. Putting me in a situation that was foreign to me—I don’t teach large cap classes, I don’t know a thing about biology—I do now, I know a little more [LAUGHTER]—But I was a novice learner in a very foreign environment and that’s what our students are in our online classes, which is really quite anxiety producing if you think about it. Going into an unknown space, not knowing what’s expected, you don’t know how to get ahold of your faculty member a lot of the time. So just being intentional about helping students be more comfortable and more at ease in our online classes—be more available to them—can make a big difference. And again, you get that insight differently when you choose to place yourself in a situation where you’re a novice, and you’re not really sure what to expect. That’s a great point.

John: Are there any other topics that we should address that we haven’t raised yet? Anything else you’d like to emphasize?

Flower: You know, really only one thing comes to mind and that is an insight that I had literally this past week, which is that I feel like sometimes online faculty—myself included—have somehow developed the notion that we don’t really need to talk with our students. And let me explain what I mean by that. Again, I’m teaching an eight week—it’s an accelerated graduate level course right now—I’m busy. My students are busy. And on a whim a couple of weeks ago, I said, “Well, I know you have this assignment coming up by Sunday night, I’ll be available on Saturday between the hours of 1 to 5pm.” I don’t like to work on Sundays. I tell my students that if you want to just pick up the phone and call me on Saturday, go ahead. So that weekend, I did. I had a student who called me and she was a chatty Cathy, and we stayed on the phone for quite some time, but she got a better understanding of the assignment and how to be successful. Well two weeks later, which was this past weekend, it was my daughter’s 11th birthday and I was right in the middle of finalizing all the food preparation and everything else. And lo and behold, there’s my phone ringing and I can tell that it’s not a connection of mine. And I went, “Uh-oh, it’s one of my students,” [LAUGHTER] because I had said Saturdays 1 to five and that same student who had called me a couple of weeks prior called and we had a great conversation. 15 minutes, I was able to keep chopping the carrots while I was talking with her. And it just occurred to me, that wasn’t really a convenient time for me personally because I was doing that final party prep, but so what? The student needed help in that moment and just taking the time to answer the phone and talking through a couple of quick questions, it was helpful for her, and it just got me thinking about how, you know what, I don’t think a lot of us really talk to our online students, like, literally talk on the phone. I know some faculty have the online office hours, I know people are using video conferencing systems, I’m available, but one of the things I’ve started doing is just saying, “Hey, if you have a quick question, just call me. We’ll talk it through.” And sometimes a five-minute conversation can ease that student’s anxiety and answer a few questions. This happened to me again yesterday where a student was like, “Before I submit tonight, can I please just check in with you?” I talked with her while I was commuting to campus and it’s just a way of talking person-to-person, humanizing the online learning experience. But like I said, I think somewhere along the line personally I had formed this opinion that we don’t actually talk to our online students. And I don’t know why that’s a perception because if you’re teaching in person you talk with your students. If there’s somebody who has a question after class, you stay a few minutes after and answer those questions. But I think for online faculty somehow we’ve missed that connection and it can be a powerful and so simple solution to helping our students thrive and succeed. I think faculty and students both overlook some of those simple solutions. It doesn’t have to be a long, tedious, written interaction in a discussion forum. It could be a phone call, and so much can be conveyed through the tone of voice and emphasis, just as I’m doing here today. And as we all do, when we’re teaching live. Just picking up the phone and calling the students or inviting them to call you. Simple, powerful.

Rebecca: I think you’re pointing to something that I know I’ve experienced even though I don’t teach online regularly. It’s just online communication is always written and it feels daunting and it feels really time consuming. And it feels like, “Oh I got to sit down and dedicate time to do this.” So it’s nice to be reminded that there’s other ways to respond.

Flower: Just in my own work somewhere along the line, I forgot about the phone in my day-to-day job. My full-time job is as an instructional designer and it seems like we never just pick up the phone anymore. It’s always email. And as you said, it just takes longer, especially if you have a little bit of confusion and you’re going back and forth on email. I literally in the past few months, I’ve just remembered how to pick up the phone and call somebody. Have a five-minute conversation, you get your questions answered. And just reminding ourselves of the importance of real- time interactions sometimes, and moving away from the requirement that everything needs to be written all the time. I’m a big fan of video announcements, I do that all the time in my online classes and again, the reason I do it is because tone of voice, inflection, emphasis, and funny faces sometimes, or just emphasis where I might just kind of widen my eyes a little bit to explain that, you know, “This is really important. Pay attention and focus.” Just finding these other forms of communication apart from writing can make a big difference in the online learning experience as well.

John: We always end with the question, what are you doing next?

Flower: Well, I’m not quite done with this project. [LAUGHTER] So I’m wrapping up this book development. But what’s really making me passionate now is to really focus on being a crusader for online education. It’s undervalued. It’s under-supported. I know that faculty don’t see the joy of teaching online and I know that students approach it the same way like, “Well, I have to get this degree and I guess this is a convenient way to do it.” I just want to advocate for how online learning and teaching can be impactful, can be rewarding, and joy giving, and you don’t see that reflected even in the coverage of teaching in higher education. Most of the time, the focus is on what we’re doing in the classroom and that’s so important, but there’s a big gap. What are we doing in our online classrooms? I just want to move into that space and encourage people to think about how they teach in person, and how to do those things in their online classes in ways that are not so daunting that they never get around to it.

Rebecca: This has been really great. I’m looking forward to picking up your book and maybe thinking about teaching online. [LAUGHTER]

Flower: That’s right. And if you don’t mind, Rebecca, I’ll just pick up on that, which is that again, I think a lot of faculty don’t say, “Hey, wow, what a cool opportunity. I totally want to teach online.” For many faculty it’s a daunting prospect, “I don’t know how to do this.” But it can be a really great way to reinvigorate your teaching—to find new ways of finding and addressing those challenges. Keep in mind institutions have the support professionals, instructional designers and such, who can help if you’re thinking about moving into online teaching. Talk with some of those faculty support folks, talk with your colleagues, and jump right in. It’s more fun than a lot of people think.

Rebecca: Well thank you so much for joining us today.

John: Thank you.

Flower: Thank you. What an absolute privilege and honor to be here. Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, and Jacob Alverson.

67. Iterative OER Development

Imagine course materials that are always up to date and evolve continually to become better at supporting student learning. In this, Dr. Steven Greenlaw joins us to discuss how some publishers of open educational resources are trying to set up sustainable practices to achieve these goals. Steve is a Professor of Economics at the University of Mary Washington and the author of the OpenStax Economics textbooks. He has also developed the materials for Lumen Learning’s Waymaker Introductory Economics texts.

Show Notes

Additional Resources

Transcript

John: Imagine course materials that are always up to date and evolve continually to become better at supporting student learning. In this episode, we discuss how some publishers of open educational resources are trying to set up sustainable practices to achieve these goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Steven Greenlaw, a Professor of Economics at the University of Mary Washington and the author of the OpenStax OER Economics textbooks. He has also developed the materials for Lumen Learning’s Waymaker Introductory Economics texts. Welcome, Steve.

Steve: Thank you.

Rebecca: Today teas are:

Steve: I’m drinking coffee. Thank you.

John: …and I have Enchanted Forest Fruits black tea from Epcot which I picked up while I was out there for the OLC conference where I last saw you, Steve.

Rebecca: You’ll never guess what I’m drinking.

John: English afternoon?

Rebecca: Yeah, it’s my favorite.

Steve: Well, honestly, I switched to tea in the afternoon.

Rebecca: See…

Steve: But in the mornings, I tend to drink coffee.

Rebecca: Yeah, you and many other people.

John: What prompted your interest in using and developing OER materials?

Steve: I have to say the developing came first. For a long time, I’ve experimented with textbooks going back into the 1980s, which at least John can remember. And I came to the conclusion that that it didn’t really seem to matter what principles book you used. Students needed a book, particularly for the analytical parts of the course: the models and things like that. But whichever book I used, they seem to learn just as well. And more recently, I’ve come to the conclusion that intro textbooks are commodities, that where companies are going to make their money is in the aftermarket products. But we’re not there yet. At least, the majority of the textbook industry is not there yet. So I had that and I didn’t really pay much attention in the 2000s about what textbook I was using, because I didn’t really think it mattered. But I did notice how high textbook prices were going and it was around that point that I became aware of and interested in OER. Again, this is dating myself, but when I was in college during the mid-1970s, I remember a teacher in my intermediate macro class—John, that’s for you—saying he would never assign texts for a course that collectively cost more than $10. [LAUGHTER] And so that’s sort of my base year. So, I sort of had this in the back of my head, I basically tried to choose around the least expensive textbook that I thought would work. And then out of the blue, OpenStax contacted me and said they had funding to create a principles of micro-macro text, and would I be interested in helping them out. I actually jumped at the opportunity, it sounded like a lot of fun. At that point I had already published one textbook commercially for an upper level course and I knew something about the commercial publishing process. I knew that I didn’t really want to go through that again, but I did want to get my ideas out there. One of the things about commercial publishing is they ask you, “What are all the innovative things you want to do?” and then once they have you on contract, they say, “Oh, but you have to do it like everybody else’s.” So that was the start. A year after the OpenStax book got published, I got contacted by Lumen Learning who said essentially the same thing. They said, “We’re building this digital platform, and we wondered if you would like to be the principals subject matter expert.” That’s the term of art that I become a SME.

John: So could you tell us a little bit about Lumen Learning’s project and the Waymaker version of this?

Steve: Sure.

John: What does it add?

Steve: It adds a lot. So, just to be clear, I wrote the OpenStax principles book. And we can talk about that process later if you want to, especially about peer review and things like that. And then I wrote the Lumen Learning Waymaker version, which was essentially an improved version. When we did the OpenStax principles book, we did it in an incredibly short period of time, I think it was nine months. So, when I did Waymaker for the first time, it allowed me to flesh out some of the things that weren’t ideal in the OpenStax book. And then OpenStax came back to me maybe three years ago and said, “We have funding for a new edition. Would you like to do that?” so I wrote a second formal edition for the OpenStax principles book. And then right after that, I did the same thing for Lumen. So, in my mind, I’ve gone through four versions of this now. And it’s not done and that’s part of the beauty of OER… at least the OER business. So to get back to your question, the OpenStax principles book is a textbook, it’s available in print and a variety of online options. My particular favorite is the phone app. So if I’m in class and a student asked me a question about something, I could literally look it up on my phone. Waymaker is a very different animal. It’s digital courseware so it’s a more immersive, interactive experience for students. And it’s not available in print. For example, how would you show a video or do a simulation in a print textbook? You can’t. The most you could do was provide a URL or something and have the student go out to that. In Waymaker, it’s all in one. So Waymaker, aside from text, it includes video, it includes animations, it includes simulations. Just to give you a specific example, instead of students looking at a graph of supply and demand, they actually get to climb in and take it for a test drive. Students really liked that. Many students seem to get it in a way that’s just looking at a two-dimensional graph, or reading text it is much harder for them.

John: I saw you present on this at the OLC conference…

Steve: Yep.

John: …And you demonstrate this. What software did you use to create those interactive graphs?

Steve: Those little interactives are H5C… maybe… it’s called? [It is H5P]

John: Okay.

Steve: It’s a European company, and it’s open source, and it’s really easy to do. I can say that even though I didn’t create the interactives. That’s the joy of working with a company… they actually have people to do the stuff that you don’t know how to do… unlike my earlier career, when I was the programmer, I was the graphic designer and all of those other things. Talking a little more about Waymaker, it’s more than a source of course content. It’s designed to teach students to study more deeply and more effectively. I don’t know about your students, but my students don’t seem to have learned how to study well. They’re very good at the game of school, but they’re not so good at learning. And I don’t mean that as a criticism. It’s just sort of a fact. They think study means read, highlight, read again, highlight again. When we know a lot from cognitive science now, that learning comes from working with the material. As I like to say, “the best way for students to learn economics is to do economics.” So Waymaker emphasizes mastery learning and personalized tutoring. The tutoring comes both from the software and also from the instructor. It’s designed to give students actionable feedback so that they can make their own decisions about how to allocate their study time. This is a really different way of learning, so I’m going to say it again a little bit differently. Assessment is integral to the learning process, it’s not just or even primarily about the grades. Rather, the assessment is designed to make students interact more deeply with the content and interact in a more intelligent meta-cognitive way. I can go into more detail about what it looks like from the students perspective, if you want.

John: Sure. Could you talk a little bit more about that? It’s a great approach. I tried to do that myself, but it’s always an add-on. Having it integrated is a nice feature, and one of the reasons why I’m planning to adopt your package in the fall.

Steve: This is really different for students, but also for the instructor. I’ve been working on this product for three years. When it finally came out in beta, I thought I knew what was going on, and I was really surprised at how little I knew about how it actually worked. Waymaker is organized into modules, which are analogous to chapters in the text. Students begin each module with the “show what you know,” which is basically a formative assessment. The purpose of that is to identify what content they already know. So, it gives them feedback on how they can efficiently use their study time. So, if there’s stuff that they absolutely already know, they don’t need to read about it again, they can just go into the stuff that they don’t know.

John: And even if they don’t, it activates prior knowledge. And it helps them make connections so that they can learn more effectively…

Steve: …Yes.

John: So there’s a lot of benefits, even for the areas they don’t know.

Steve: Yes. And I’m actually adding a little exercise for my first day of class next week, where I put my students in small groups. Some of whom who’ve had the first semester, and some of them who have not. And I’m going to give them a basically a problem to work with, knowing that some of them won’t really know what to do with it. But I want the groups to start working together. But anyway, I digress. So, as students progress through the content, there are a series of learning activities. The original one is called a “self-check.” It’s basically a short formative quiz. The purpose of the quizzes is not summative assessment. But as I said before, it’s to help students think more about their learning. Think about the idea of a Socratic tutor. The tutor doesn’t ask questions to assess the students’ knowledge, but rather to help them work through the content, help them really understand it. So what happens in Waymaker is: the student reads a page a text, or watches a video, or plays a simulation. And then they’re posed a very short quiz, like one or two questions. If they pass the quiz, the “gate” opens and they move to the next section. If they don’t pass the quiz—and on a one-question quiz, either you get it or you don’t—Waymaker suggests that they review the content before attempting the quiz again. They can take those quizzes as many times as they want to. So they can really build some expertise. There are other sorts of learning activities, but I want to focus on the quizzes today. At the end of the modules, students take a module quiz—essentially a chapter test—which is summative. Again, if they fail to achieve mastery—and the default mastery level is 80%, so it’s pretty high level. As an instructor, you can change that to whatever you want. But I like 80%. So if they don’t achieve 80%, they’re encouraged to study again and they’re given information about what areas to study. And then they can take the module quiz one more time. They’re only allowed to take the module quizzes twice. Now, here’s where it starts to get really interesting from the teacher’s point-of-view. The instructor receives reports from the module quizzes whenever a student fails. So for me, the first really good thing about Waymaker was that I don’t have to go to some website and look at some spreadsheet and see which students are struggling. Rather, anytime a student fails, I get pinged from the software. So it says, “so and so…” Well it’s a little boilerplate language… but basically it says they worked through the module, and they scored a 46 on the module quiz. You might want to reach out to them at that point. So the software is flexible. So you can get these things in real time, you can get them once a day, you can get them once a week, if you want to. I get them once a day. That seems reasonably quick for me. If the students taken the quiz at three in the morning, I’m not up anyway, so it hardly matters. It’s not like I’m going to give them that fast feedback. But what happens is I get that information, and then I get to decide, “What am I going to do about it?” If someone gets a 76 on their first attempt, I generally figure, “Okay, they’re gonna figure this out.” And so I don’t worry about it. If someone gets a 46, then I immediately want to reach out to them and say, “Hey, I see that you’re struggling with this. You know you can take it again. Go back and review the material. And if you’re not sure that you understand it, let me know and I will work with you on this. Because the goal here is mastery. It’s not anything else. Anyway, Waymaker helps me, the instructor, make better, more efficient use of my time. In any given week, Waymaker allows me to know two important things. It allows me to reach out only to those students that need my help. And it lets me know what topics the class is struggling with, so that I can tailor my in-class time to the material where the students need help and not spend it on material where they already know this stuff. Basically, it gives me a better feel for the effectiveness of my teaching and student learning. And that’s really, really important I think as a teacher. I’m embarrassed to think of my early years and teaching, when if I got all the way through the 50 minutes, I counted that as a successful day.

[LAUGHTER]

John: I think many of us started like that.

Rebecca: It ties really nicely to your blog post series that you’ve just recently published. The first one being the critical importance of instructional design…

Steve:Yup…

Rebecca: …where you talk a lot about the instructor’s role is designing the experiences, rather than delivering content. Can you talk a little bit more about how Waymaker helps you do that as an instructor?

Steve: There’s a “just-in-time teaching” element to this. I have a course outline, I know what I’m supposed to be doing on a week-to-week basis. But what happens on any given day depends on the stuff that came before it. I’m absolutely not wedded to the calendar. If the students haven’t figured out what we did on Monday, I’m going to start by spending a little more time on that. But also because of the feedback that I’m getting from Waymaker, there are times when I spend 90% of the class on 10% of the material. Because that’s what I know students are having trouble with. I know that if it’s something analytical, probably what I’m going to want to do is instead of talking to them about it—I mean, certainly I’m going to talk to them about it—I put together some group activities. I do a lot of group activities, small groups, generally two to three people. And then I essentially turn the classroom into a lab experience for that day. They seem to enjoy it more, they seem to get more out of it than me just lecturing over the content. After all the content is in the book. I don’t need to just repeat that stuff. So I guess that’s my short answer to your question.

Rebecca: Can you give an example of the kinds of activities that you’re doing with your students?

Steve: Oh, sure. Supply and demand is the first real model that the students work with. And so one of my learning goals is that they ought to be able to take a scenario… something happens… use supply and demand to analyze the effect on the market for x, gasoline or something like that. Typically what happens is, hopefully they will have read the material in Waymaker. Typically, I spend a day talking about “here’s how you would do it” and then generally what’s going to happen is, I spend a day where I have a couple of problems, like three is all that we’re going to have time to do. And I say, “Get in groups of two or three.” Basically, I count the number of students that showed up that day, because my classes are pretty small. And if it’s divisible by three, I put them in groups of three, if it’s divisible by two, I put them in groups of two. And then I say, “Okay, here’s a problem,” I show them the problem. And I say, “Take 10-minutes to work through this, draw the graphs.” And then they know that I’m going to call some of the groups up to present the results to everyone else. So there’s a little bit of competition. It’s not very stressful. It’s a little stressful for people that don’t like to speak in class, but you’re not there by yourself. You’re there with your group, so it works better that way. So I do a couple of those problems until I’m convinced that most people know what they’re doing. So that would be an example.

John: You also mentioned—when I saw you present at the Online Learning Consortium—how you use some of that feedback to improve the text in your current edition. Could you talk a little bit about that process of revision and creation of the text?

Steve: Sure. While I can’t take all the credit. From the beginning of Waymaker, at least from when I began to get involved… once I realized how integral the assessment process was to Waymaker, I pressed Lumen to make sure that the assessment questions were good. One of the things that I’ve noticed over the years is, test banks seem to be the lowest priority of textbook publishers. Because after all, they’re selling the text but they’re giving away the test bank. So what I want, I guess what we all want, is that the questions in the test bank that Waymaker uses, are discriminating correctly. And that’s harder than you might imagine. To their credit, Lumens put a tremendous amount of effort into this. And more generally, into the design the courseware. This has resulted in a process of continuous improvement. Now, continuous improvement is not a term that excites most faculty. I think that’s a fair statement John? [LAUGHTER]

John: Yes.

Steve: But what it really means is that, Lumen has an ongoing process for improving OER, making it more effective every single semester. And they’ve done this, and we’re now in your five and a half. So how does it work? I have a short answer and a long answer. The short answer is, after every semester Lumen downloads the data from every student who’s given them permission at every school using Waymaker across the country. And then they analyze the data. The analysis identifies where the students are having problems. At that point, we go in and either revise the content to make it clearer, or add some learning activities. Or else we revise the assessments to better capture student learning. We do this a little bit in a panicky way over the winter break, because we only have a month. But we do it intensely every summer. Here’s the longer answer. Over time, we’ve gotten better at doing this more efficiently. Lumen has developed something called “RISE Analysis.” RISE is an acronym. I don’t remember what the letters mean. [LAUGHTER] But basically it asked the question, “Which course materials would benefit the most from improvement?” Or to put it differently, “Which changes would have the greatest impact on learning?” So what we’ve done—and this is all programmed now. So Lumen has dozens of Waymaker courses, not just an economics. Though, I like to think that some of the most interesting stuff is started in the econ Waymaker platform. I’m not just making that up, it’s actually true. [LAUGHTER] So, instead of just doing the aggregate sweep on the data, we particularly look at student learning outcomes. And everything in Waymaker is driven by the student learning outcomes. This is out of order, but let me just throw this in for a minute. The way that Waymaker started is they brought together—I want to say 50 principles instructors from everything from community colleges up to R1s. And we spent four days together. And we asked the question, “What do you have to have in your principles courses?’ And so from that we created a list of primary learning outcomes. And then we drilled down and we now have secondary and tertiary outcomes. So the assessment questions in the test banks are coded down to the third level. So everything is really granular, if you want to think about it in those terms. What we look at is not just which student learning outcomes are students struggling with. But rather, which student learning outcomes where students are doing relatively poorly, are they putting a lot of time and effort into. Because that’s where we’re going to get the biggest bang for the buck in terms of fixing things. So what we do is we look at three things. We look at, “Are the questions badly worded?” We’re mostly done with that at this point. “Are the questions testing what they’re supposed to be testing?” There are some psychometric tests that allow you to do that. And then finally, what we do is—after we’ve exhausted all those—we look at the content and we create new content, or different types of learning activities, and we integrate those into the course. So, the interactives that you saw at OLC John, they were the big new innovation from last summer. So we do this, and then we teach the courses again, and then we start the cycle all over again. So, the process just goes on. It’s not continuous, as in every day, but it’s continuous, as in regular. I’ve used the courseware since the first year, and the courseware has gotten noticeably better. Fewer students are failing to achieve mastery on the module quizzes. And fewer of them are crashing and burning. More of them are in the 60 to 70% range when they fail. But what’s really cool is Lumen has shown no sign that they’re ready to quit, that they’re done with this. As long as they’re willing to do this, I think I’m willing to do this.

Rebecca: I like the iterative process.

Steve: Yeah.

Rebecca: That’s something that, as a designer, I’m very comfortable with… that I do all the time, especially designing online. But one of the things that’s really interesting about this model is that, as the author of the textbook, you don’t just have this finished thing. It’s an ongoing…

Steve: …It is.

Rebecca: … thing. So that’s a really different model of authorship.

Steve: Yes, it is. I think it’s fair to say that we make small changes all the time. And then every summer, we make larger changes. And that’s pretty interesting. Because as a user—as you pointed out—I can see that this is helping.

Rebecca: Yeah, that’s really exciting.

Steve: Right now, the hardest part is getting students to trust the process. Because it’s a very different model of learning. And so one of the things that I’m going to do this semester is, build in opportunities for me to remind them that this is a different process, and that they need to trust the process. One of the things that I did last year, which seemed to help with that was I started using exam wrappers after the midterm exams. And ask them to think about how they were studying, and what they would do differently, and what I could do to help them. It’s real easy to see in 30 seconds, I can tell if they’re taking it seriously or not. And if they’re taking it seriously, I learn a whole lot from what they say. So, anyway, just another little wrinkle.

Rebecca: So we’ve talked a lot about the students and the different learning process for students. You talked a little bit about the different processes being the expert, or the writer of the book. And you also mentioned earlier about the peer-review process for an OER being a bit different. Can you talk a little bit about that?

Steve: Sure, and that’s really important. First of all, people have a wrong idea about how OER is produced. The OER that I have experience with is working with OER publishers. It’s not the loan faculty member working in their spare time in their basement, or something like that. Both the OpenStax and the Lumen experience for me, have been very much a team effort. There have been a lot of people involved. So this is really important because one of the concerns about OER textbooks is their presumed lack of quality. There was an article in The Chronicle about that, today in fact. I have to tell you that the peer-review process that I went through with OpenStax was extensive. The way we did this is, OpenStax purchased a manuscript from Tim Taylor—a prominent economist—as the basis for the first edition. They sent copies of the manuscript out to about two-dozen reviewers all over the country, asking them to identify strengths and weaknesses. Based on those review comments, I rewrote each chapter. Each chapter was then sent out to half a dozen new reviewers. And again, the reviewers were from a range of schools, from community colleges through research universities. I took that feedback and I revised each chapter again before it went through the editorial review and production process. I have to say, this was much more detailed and extensive then when I worked with a commercial publisher. The review process for Lumen was similar, there was a lot of peer review involved. And as I said before, I’ve now written two formal editions of both texts. We’ve gotten lots of feedback from users. I’m pretty happy with that.

Rebecca: Do you find that the difference between OER and a commercial publisher is that you keep getting this feedback from users? And that you’re able to revise based on the use of other faculty, rather than working in a silo?

Steve: If I’d written the principles book for a commercial publisher, I would be better able to answer that. I got no formal feedback on my commercial book. I got a lot of comments from people at conferences and things like that. But we have gotten tons of feedback on the OER books, and that is interesting. You can’t satisfy everybody. Somebody says, “This chapter is too long.” Somebody says the same chapter is too short. But, in general, the feedback has been really, really helpful. And we’ve tried to incorporate it as soon as possible. And with these digital text, it’s really easy to do. I can literally go in and edit if I have five-minutes on the fly. And then it’s out there.

John: While with regular publishers, there’s usually a three-year cycle on intro textbooks.

Steve: Yes. And that’s the other thing that—now I’m not a typical user, but I know that if I want to make a change, it’s going to be done by the next semester. The same thing is generally true of other people who give us feedback. Though, they don’t necessarily know that. W e take that feedback very seriously. And there is no three-year review process. So that’s wonderful.

Rebecca: I love the user-centered design process, like that’s clearly what’s being used.

Steve: Yep, we try.

John: And that iterative process is what we should all be doing with our courses, all the time…

Steve: …Yes.

John: …But the fact that you’re doing it makes it easier for instructors who perhaps, don’t have to do as much of that.

Steve: Yeah. But again, let me just say one thing; Waymaker is not my course, Waymaker is my text. So there’s whole levels to my course that go beyond Waymaker. That’s just one element of it. Not that I’m disagreeing with what you said.

John: I’ve seen you present at conferences on teaching principles, for decades now. And I know you’re constantly changing how you’re teaching your courses and trying new things there. And you’ve been doing a lot of great work for quite a while.

Steve: Thank you.

John: Going back a little bit though, to the question of mastery quizzing. When students take the quizzes at the end of a block, you said there’s one or two questions. When they do it a second time, do they get the same question or different questions?

Steve: No. We are adding questions fairly regularly, and so the test banks are getting larger. From the beginning, I think we started with 2000 questions. But again, that’s across the whole book. The questions are randomly chosen, so the odds are that students would get different questions at the self-check level, at the section level. There’s a different test bank for the self checks than there is for the module quizzes. But there are similar questions. In fact, we wrote two at a time basically when we did that.

John: This is a question more generally about Waymaker. Does it do any type of interleaved practice, where later in the course, does it call back earlier sections? Or is it just based on the current module?

Steve: No, it’s just based on the current module. But my more nuanced response to that is, economics is sort of cumulative. But I have thought about that, we just haven’t thought of a way to build it in yet.

John: In my classes, I’ve been adding that the last couple years where I just randomly pull in questions and the module quizzes from earlier modules. Maybe 10 to 15%, building up to about 20% at the end, just to help do a little bit more spaced practice as well.

Steve: I think I know how you could do that pretty easily. Because instructors have access to the test bank that their students are using, so that you can edit your own questions. But what that also means is that you could move questions from earlier into the course to later in the course. So I think there’s a way to do that.

John: Excellent.

Steve: So John, we learned all this in our graduate training, right?

John: [LAUGHTER] You know, it’s getting a little bit better. Some people are learning these things. We have someone in my department who actually came out of Kentucky where he had a lot of training and teaching and learning. But it’s still pretty uncommon.

Steve: Yep.

John: You mentioned two ways in which, OER materials are developed. Some by primary developers, such as the OpenStax and Lumen. And others, with people working in their basements…

Steve: …Yes.

John: …or working in a dark room somewhere. Which is how I often do a lot of my work. Is that process sustainable? And what role do for-profit publishers such as Lumen play in providing these services, or in continuing the development of OER materials?

Steve: There are a couple questions here. One is, is the development process for published OER materials, or OER materials created by publishers. Is that sustainable? And then the second one is, is the individual scholar model sustainable? And those are very different questions. The individual scholar model, I don’t know if sustainable is the right word. I have a colleague who did this, she did it all on her own. I’m so impressed. She didn’t have any support from the school other than a small summer grant. And she did it without any sort of extrinsic motivators. I think that over time, at least at schools like yours and mine, faculty are going to get credit in tenure and promotion, for creating OER, especially open textbooks. I think that’s really important. I think that people will eventually be able to get sabbatical leaves to create these materials. And I think that’s really important to keep that side of the OER creation process going. As far as revision, I don’t know enough about that to really answer that. But I’m curious. I may have to go talk to my colleague Katie now. As far as the publishers go, and I don’t mean the traditional publishers, every publisher has a plan for how they’re going to do this. Some work better than others. I know something about OpenStax and I know a lot about Lumen, about what their sustainability plan is. OpenStax have develop partnerships with a variety of ancillary publishers like Sapling Learning or Knewton. These people provide aftermarket functionality for the OpenStax books, and in return, they get kickbacks from these ancillary publishers. And by kickbacks, I don’t mean anything pejorative about that. I just mean that they contribute financially. I don’t know any more about how sustainable that model is. I know that that’s what OpenStax has been using. Lumen from the beginning, has been a commercial publisher. It took me two years to figure out how a commercial publisher could make money giving their content away. Maybe others haven’t thought about that, but I sure did. So, the short answer is, Lumen gives the content away, but charges a very modest amount, $25, for the intelligent backend. All the feedback that goes both to the students, and the instructors. Today, you personally, either of you, could go and get a copy of the Lumen Principles and Micro book, or the Principles of Macro book, and it’s yours forever, you can do with it what you want. But if you want to take the full Waymaker course, they charge $25. The idea is, that amount of money is both affordable to students, but also enough to maintain revisions and corrections, and keep the servers running and all of those things. So that’s the answer to that question. And I will say that every semester, I try to be completely transparent, and say, “If you don’t want to pay the $25, you can get all the content for free. But here’s what you lose.” In five years, I’ve never had a student who didn’t pay the $25, because they thought it was like beer money for the weekend, or something. Compared to spending 300 bucks on a traditional text that was nothing to them.

John: What are some of the barriers that you see to faculty adopting OER? You mentioned that people may have this perception of lower quality…

Steve: …Yes .

John: …but there’s quite a bit of evidence that the quality is not weaker in any way. And I think you had done some studies on that a while back, didn’t you?

STEVE. Yes. The number one problem I think is misinformation. The majority of faculty today don’t know what’s available in their discipline. Many of my colleagues have told me, “Yeah, OER sounds like a great idea, but there’s nothing available in my field.” Now, that’s flat out wrong. For your listeners, there is OER available for nearly every Gen-Ed course taught today. So that’s number one, is lack of knowledge of what’s available. Number two is, as you mentioned before, the belief that OER is inferior, that there’s no peer review. And that’s just not true. There’s a couple things here. One is that OER publishers don’t have a sales force, and so it’s going to take longer to get the word out. There’s been a lot of progress over the last few years. But at my school, we’re only in the second-year of our formal OER initiative. So we’ll see how it goes. The other thing that I think gets in the way of adoption of OER is path dependence, and the unwillingness of many faculty to change their textbooks because of the fixed costs involved. “I’m going to have to go through my lecture notes and make sure that I’m using all the same terms as the textbook does,” and that sort of thing. I don’t know the answer to that question. I know that some schools have used financial incentives, fairly modest financial incentives, to get faculty to try to make the switch. As far as my own assessment goes, every summer, I do statistical analysis of the effectiveness of the texts that I’m using. I looked at both the OpenStax Principles book, and also most recently, the Waymaker package. What I’ve looked at is, textbook alone, textbook with ancillary website, digital courseware, and because I used to teach a writing intensive version of the principles courses, I also looked at writing intensive. And what I found is pretty predictable, at least from somebody who has done this for a while. What I found is that there is no significant difference between student learning using OER, with commercial textbooks. I found that using either courseware or an ancillary website improves student learning outcomes, regardless of what the text is that you’re using. And I’ve also found that writing intensive courses seem to work better than non-writing intensive courses, because the students are getting into it in more detail. Over the last two years, I’ve been doing a randomized control trial, where I can really drill down and see what’s going on. And what I found is that using the full Waymaker package seems to have a statistically significant positive impact on student learning. So I’m going to rerun the analysis using last semester data, which I haven’t had a chance to get yet, but I’m anxious to see how that goes too. I believe this stuff works. And so I think sooner or later, more and more publishers—the commercial publishers too—are going to move towards digital courseware type products.

John: I think most of them have started to at least.

Steve: Yes, but it’s like turning the Titanic. Their base is so large that it’s going to take a while before even all of those people get on-board with this.

John: One thing I was wondering is whether you see more collaboration or competition in OER textbooks?

Steve: Initially, there was more collaboration in the early years. And the reason why is because anybody who was doing OER, was increasing the interest in users for everybody’s OER. Now, I think we’re going to see more competition between the users. Especially as more publishers are going to adaptive and personalized learning type courseware. I think that’s a way that publishers are going to be able to say, “Well, yeah, we’re doing that. But we’re doing better in our own particular way.” So I think there’s going to be a fair amount of product differentiation. And it will be harder for faculty, it’s going to take more work to dig in and see exactly what’s going on. I would love to see more published assessment of efficacy on the part of the commercial publishers. They’re only now starting to do that, and the studies that they publish are heavily controlled by them. So it’s not clear that they’re telling us about all of their things, just the ones that work. But at least it’s a start.

John: One of the things I see in most of those studies is comment to the effect that, “Students who use our adaptive learning platform have letter grades on average, one letter grade higher or point eight points higher.”

Steve: Yes, that’s right.

John: And there’s no evidence that they’ve done any control for the students who chose to use it versus those who didn’t.

Steve: That’s right.

John: But it would be nice if we could see more research on that.

Steve: And I think we will. At least I’m hopeful.

John: Earlier you told us a little bit about how your course is structured with some “just-in-time teaching,” and some activities there where you have students work on problems. Could you tell us a little bit more about how you structure your course so that it’s not duplicating the textbook?

Steve: The first thing that I would say is that, my intro course looks like almost anyone else’s Principles of Micro or Macro course. If you look at the course outline, it has all the normal topics in it. A very slight difference is, instead of assigning students chapters to read and problem sets to do, students have modules with content and learning activities to complete. There is some difference between my face-to-face sections and my online sections, because I teach both. My face-to-face sections are pretty much the way I described them to you earlier. My general approach is to do Socratic lecturing with a lot of in-class activities, like the supply and demand problems that I mentioned. I also like to have formal in-class discussions on interesting questions that don’t have a right answer. In the macro class, I spend a day talking about what is money. And I spend the day talking about what is government. And those are things that aren’t done in the same way and the same degree with a textbook, whether it’s Waymaker or something else. My online course is roughly similar. But what I do is I add group and individual activities to the online course to mimic what I do in class. I also have a weekly Google Hangout, a synchronous Google Hangout, where I can give students guidance about what I think they should be doing. And I can give little mini lectures on things that I know students have trouble with. But it also gives them a chance to ask me individual questions in a real time basis, one on one. Not a lot of students come to those Hangouts. I usually have between five and ten, and my classes are about 35. But more than 90% of the students watch the recordings. Google Hangouts are automatically recorded and archived in YouTube. So the students seem to like that a lot.

John: You mentioned that a number of people at Mary Washington have switched over, what proportion, would you say, of the faculty at Mary Washington has moved to using OER?

Steve: Single digits, a handful, probably less than ten at this point. But this semester, I have two new people. So I’m excited about that. And we haven’t yet given them any money or anything to do this. I’ve just been talking to people. I was invited to the College of Business’s summer retreat, and I gave a little talk about OER. And I got two people who expressed an interest in following up. One of whom has already done it. So I think we’re getting there. We just have to be patient.

Rebecca: So we normally wrap up by asking, well, what’s next?

Steve: What’s next for me is I’m continuing to iterate to improve Waymaker. I’m going to continue doing my own statistical analysis. So I get access to the aggregate analysis that Lumen does, but I also have my own analysis. So I can tailor that to my particular students. I also want to do something this semester that I’ve wanted to do for a long time, but have never done it. And that is to write a new non-traditional chapter for the micro book, which is relatively easy to do. It’s just really a question of me sitting down and doing it. So I know it’s doable, but I do want to actually make my version of Waymaker different from the standard version. In part, because it’ll better match the way I teach. But also because I want to see that it’s relatively easy to do so that I can talk about that to faculty.

John: Very good.

Steve: I’m going to the CTREE conference this summer to talk about Waymaker. And this is the first time we’ve actually reached out to a disciplinary conference. So I think that’ll be fun.

John: You know, I always want to go to the CTREE conference, but I teach at Duke in the summer and it runs right into that. So I haven’t been able to go. And we should note that the CTREE conference is a Conference on Teaching and Research and Economic Education.

Steve: I love to talk about this stuff, because I believe it.

Rebecca: Yeah, it was really interesting.

John: Thank you.

Steve: Oh, you’re very welcome. Thanks for the opportunity.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kelly Knight, Kim Fischer, and Jacob Alverson.

[MUSIC]

65. Retrieval Practice

Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, Dr. Michelle Miller joins us to discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Her academic background is in cognitive psychology and her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications.

Show Notes

  • Miller, M. (2014). Minds Online: Teaching Effectively With Technology. Cambridge, MA: Harvard University Press.
  • Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968.
  • Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 1199327.
  • Kahoot
  • Bray, Niki (2018). 43 to 0: How One University Instructor Eliminated Failure Using Gamified Learning. Blog post
  • Retrievalpractice.org
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Pennebaker, J. W., Gosling, S. D., & Ferrell, J. D. (2013). Daily online testing in large classes: Boosting college performance while reducing achievement gaps. PloS one, 8(11), e79774.
  • Pauk, W. (1984). The new SQ4R.
  • Thomas, E. L., & Robinson, H. A. (1972). Improving reading in every class. (a discussion of PQ4R)

Transcript

John: Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, we discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today we’re welcoming back Dr. Michelle Miller. Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications. Welcome back. Michelle.

Michelle: Thank you so much. It’s so great to be here today.

Rebecca: We’re so happy to have you again. Today’s teas are:

Michelle: Well, I’m drinking Coco Loco, which is a blend from a local tea shop here in Flagstaff, Arizona. Steep Leaf Tea. And Coco Loco is a lot like what it sounds like. It’s chocolate and banana. So tea snobs may scoff at my choice, but it’s wonderful.

John: IIt sounds good.

Rebecca: And I think I saw a nice silver teapot that was poured into a green and blue tea mug.

Michelle: Yup.

Rebecca: A nice tall one.

Michelle: That’s what I need.

John: And I’m drinking ginger, peach green tea.

Rebecca: I went with a Christmas tea today. So Michelle, we invited you here today to talk a little bit about retrieval practice. Can you first start with defining that for us and letting us know what it is?

Michelle: Right. So retrieval practice is essentially the act of pulling something out of memory. So that is, in memory research, what we would term retrieval. So something is stored in memory and we want to pull it out so we can actively use that information, have it in our conscious minds and so forth. And so we go through this usually very fast process called retrieval. So retrieval practice is specifically the act of doing it, and we contextualize that with learning. So, when I’m trying to learn something or I’m the process of learning something, to say, “Oh, what was that fact I remembered, or what can I say about this?” When we do that, it produces something that we can call the testing effect. So this is kind of the clearest example… not the only… but the clearest example of retrieval practice in action during learning is when we sit down to take a quiz, take a test or something like that. So all the excitement that’s happened around retrieval practice in higher education, and really in the rest of education today, is around this finding, which has been replicated many, many times: that tests are, as one person put it, not neutral events in learning. When we take a test on something, that has a very powerful effect on our ability to remember it in the future. So, really simplified down to its core, tests help us remember in the future; when we take a test it strengthens our memory. So that’s what retrieval practice is and it can, as we’ll maybe talk about today, take many, many forms in learning settings. And I did want to clarify too, this is something that I definitely don’t want to take any kind of credit for discovering this. This has been around and has been known about for a long, long time. Some of the big names who are associated with this: Jeffrey Karpicke, Robert Bjork. Roddy Roediger, there are quite a few really heavy hitting cognitive scientists and cognitive psychologists who have established this. But there are many, many of us who are out there trying to disseminate this to other teachers around the world so that we can all tap into the power of this. And I have done a little bit of work in this area with my colleague here at Northern Arizona University, Laurie Dixon, who’s another psychologist… and we teamed up some time ago to look at a very practical implementation of retrieval practice in an Introduction to Psychology course that we conducted some years ago and this is a course, you can imagine, where just trying to get students to perform even a little bit better is a big project. So we examined how even something kind of basic… it was very high tech at the time… but just basic web quizzes that came packaged with the textbook. We said, “Well, if we actually assigned students to do these as part of the course, and if they went through and treated these as opportunities to learn, not just assessments in and of themselves, would that have any systematic impact on course performance?” And we found that in fact, there was a significant improvement associated with that. So that’s kind of the landscape of what retrieval practice is, and why we’ve been so interested in discussing this in the psychology of teaching and learning.

John: In fact, I saw you present on that about 11 or 12 years ago in Orlando at one of the NCAT conferences and it convinced me to completely revise how I was giving my classes and it’s made a big difference and resulted in some significant improvements in student learning. Given that we know so much about retrieval practice. Why are faculty resistant to doing this?

Michelle: Wow, that’s a great question, and that is one that I have been really facing a lot these days in my own practice, talking to other faculty members, and disseminating this through some different activities I do in this area. It is easy for those of us who work in this area, cognitive psychologists in particular, but a lot of us who like you heard about this a long time ago. We’ve seen the power of it. We forget that to other faculty, this can be a very off-putting concept. And so it’s really great for us to think about why that is. And I always think that’s really good for me to kind of go back to that and say “Yeah, not everybody is sold on this idea. And there are good reasons for that.” So like with a lot of things that we talk about in teaching and learning, I think that these really break down into two neat categories: there’s the philosophical issues that people have with it, and then there’s more practical and logistical issues with it. So kind of tackling those one at a time. Philosophically, when people say, “Yeah, I understand about the research but this just goes against something that I believe as a teacher or how I want my classes to be. Here are some ways that that can play out. First off is this idea, that I’ve heard in one form or another quite a few times, and that is this notion of superficial learning. So “Okay, sure, there’s a study that showed that maybe people retain something a little bit better. But surely that’s not this deep learning, whatever that is.” That’s a concept that we all want. So do tests and exams… just testing… create just a superficial form of learning? And while, of course, I understand that, and I absolutely applaud faculty for really thinking deeply about that issue, and caring about it. I Here’s the thing… we got to define that. We social scientists, that’s what we do… we have to kind of break things down and say, “Okay, what does deep learning mean?” and I don’t know that anybody has kind of definitively done that. But when I look at that, I say, “Well, this is not just one research study that showed a little improvement in a lab test. There’s quite a few studies that do use realistic types of materials. It’s not all just contrived laboratory studies. Furthermore, there’s also studies that show that when students engage in more quizzing and testing on material that they actually are able to transfer that learning better. And that is a very, very big deal in teaching and learning as a lot of us know is not just getting students to be able to solve a problem in one context or work a concept in one context, but can they do it in the next circumstance? And that very difficult process is aided by quizzing… and to me, what could be deeper learning than learning that transfers? So that’s part of it. Some of it is perceptions around multiple choice quizzes and tests. There’s an assumption too that if we’re talking about quizzing we must be talking about multiple choice questions… and first off, sometimes in larger classes, those are the final assessments… in that Introduction to Psychology course that we studied years back, that’s what the assessments were… so, having students practice in that format I don’t think that we should necessarily dismiss that. And as we can talk about in a little bit, there’s lots of ways to induce retrieval practice that actually don’t involve multiple choice questions. So, there’s a bit of that as well. And something that I’ve talked about with some faculty recently, too, is this baggage around K through 12. And maybe that’s something that’s resonant with you all.

John: Yeah, that’s given the testing effect a somewhat bad name, because high-stakes testing is being used in a lot of what’s going on with K to 12. But that I don’t think is what retrieval practice as you’re suggesting is all about.

Michelle: Right, and I have to be cautious here. I really like how you laid that issue out in K through 12, that there is a reputation problem… and that has happened because of the high-stakes standardized testing policy in the United States. And I got to be careful because I don’t want to represent myself as an expert in K through 12 or in K through 12 education policy. But I don’t think you have to be an expert in that to know that there’s been a lot of same pretty well justified public pushback against over-testing in K through 12. And yeah, I think that we absolutely do have to be aware of that. Students come to us in higher education… that’s a system that many of them have been through and our faculty are very aware and very cognizant of that too. So, nobody’s a blank slate here, not our students, not our fellow faculty. We have assumptions and ideas and experiences about testing that happen. I think those can be addressed. But, yeah, that is definitely another very big barrier. We got to differentiate between high-stakes standardized testing for the reasons it’s done in K through 12, and low-stakes testing and quizzing for learning as proponents of retrieval practice would have it.

Rebecca: Some of the pushback I’ve heard from faculty fall into two categories related to this as well. One is that they assume that retrieval practice is best implemented in 100, 200 level introductory classes instead of upper level 300, 400, graduate-level classes. And then the other area is that paper and pencil tests don’t make sense and all disciplines. And so they assume that a test has to be in a paper/pencil format, which could be online testing, or it could be multiple choice or it could be essay questions. But I think that, from being someone in the arts, like there’s other ways to test beyond that, but we don’t think of those as tests.

Michelle: Right. That’s another great lens through which to look at this issue; that we do need to broaden the definition to draw more attention to this and to make it a more appealing concept. But yes, how can we make it broadly appeal across lots of disparate disciplines? Not only does it not have to be a multiple choice type of exam, maybe it’s not a pencil and paper exam at all. And we as faculty have to think about what makes sense there. You make a good point about the levels concept. I think, these days most of us have heard of, “Well, there’s one particular Bloom’s taxonomy…” which is a wonderful framework for getting us thinking about being systematic about what we’re asking students to do with the information that we’re bringing to a course and trying to do things like align the teaching we do with the assessments that we have. That’s wonderful. However, I think it does ingrain in us that idea that “Well, just knowing things is sort of at the bottom… You sort of get that out of the way, and then we go on to the good stuff.” And from a cognitive perspective, these relationships are much more fluid and much more interdependent, so that yes, absolutely, the higher thinking that is what we want. That is what we should want. Or if we’re in highly applied disciplines (if we’re in the arts, for example), we need students to be able to do things with that information. But they have to have that. So I think it challenges us to think of new ways with that concept as well.

John: One of the barriers I think some people have is they don’t like to grade tests and so forth. But one of the things you mentioned in your book is that the testing effects been known for a long time, but it was really difficult to implement in terms of low-stakes testing, particularly when you’re teaching at a larger scale. But, as you’ve done yourself, and as you suggest in your book, computer technology makes it easy to automate some of this… certainly more easily for multiple choice and free response and similar things. But it makes it a whole lot easier for both students to have multiple attempts at learning something using some type of mastery quizzing and it makes it a whole lot easier for faculty who don’t have to spend all their time grading.

Michelle: Right and that’s absolutely where the practical stuff comes in. So, we’ve worked through some the philosophical objections, that: “No, this does not turn your classroom into some terrible assembly-line concept of learning. It’s not going to create a bad relationship with your students. It’s not going to simply carry on a legacy of bad policy as people perceive it. It’s not going to do those things.” And then, we do get to: “Alright, but what is this going to do to my life as a faculty member?” …and this is important stuff. Those who know me know I’m a big fan of James Lang’s work in his books. One of his more recent books is called Small Teaching and there’s a lot of different takes on it in that book, but it really hits home with respect to “We do have to think about not everybody’s in a position to, nor should we even try sometimes, to just take everything down to the foundation and rebuild.” That we do really need to think about “Do I want to do this, if it’s going to create 800 more questions for me to grade?” Is this a sort of a situation where, “Well, I don’t want multiple choice, so I’m going to have to give these open-ended questions. I’m gonna have to give feedback and I have 200 students, what will happen? If I am going to go with multiple choice questions… well, how am I going to do this? to have to write all of this?” And yes, it’s one of the most powerful outcomes of the educational technology revolution that makes it workable, and scalable in a sense, even with large classes to do these types of things and to bring them in. So, that is definitely a message that I hope faculty think about if they’re on the borderline of wanting to bring in more retrieval practice into their classes.

Rebecca: I’m in a discipline where the multiple-choice questions are using things digitally doesn’t always work for testing and practicing some of these basic things, and there’s not a good way to automatically grade it. But one of the strategies that I’ve used is actually some self grading, which has actually worked pretty well. I just check and I have them write notes about anything that they got wrong. So, it demonstrates that they’ve tried to understand when we go over it, and I give credit based on how thorough those notes are, rather than whether or not they got the question right or wrong. And that’s made a difference in my classes. And had I not come up with that solution, I think I would have abandoned it because it would have been too much work. But it it actually is working pretty well.

John: When we had a reading group on Small Teaching last year, one of the things that was widely adopted by faculty was a very simple form of retrieval practice where they had students at the start of each class reflect back on what they had done in the previous class. And most of them have continued to do that in subsequent classes as well.

Rebecca: One of the other barriers that faculty might raise is the idea that students don’t take low-stakes things seriously, or that they don’t put the same kind of time into it that they might for something that’s high stakes. Can you talk a little bit about how we might help students find value in retrieval practice and subsequently also with the faculty then?

Michelle: Right, so that, ”Well, we can put it out there, but will they do it?” I’ve kind of crossed this philosophical and practical barrier for myself of giving some credit for pretty much anything that I am hoping that students will do. I put the work in to set it up and I do believe as a teacher that there’s reason to believe that will help their performance, that I need to work it into the syllabus somewhere. I don’t think it detracts from learning, necessarily, to say, “Yeah, there’s some points associated with this.” And especially with our students who, for many of us, are going in a million directions at once. They’re juggling jobs, multiple classes, sometimes their own families. So having an incentive in the form of points—having some kind of a payoff—I think, helps them make that decision that this is at least worth the time to do. I think the other thing that we probably can all do more of, and that I’ve done more over the years, is framing and honestly marketing this to students… communicating with them about why. And when students disengage from an activity like this, when they say, “Ah, why do I have to sit down and do this thing? This is just another test. Oh, no.” …really conveying the excitement and the goodwill that we have in setting those things up can go a very, very long way. Of course, a student, if they just look at it inside and they have no context for why this was put into place, they’re going to have them say, “Well, maybe I won’t do that.” But when we can market to students, we can say “There is a lot of research that shows that this is a very, very good use of your time. And hey, you’ve probably taken a lot of tests in your life that were really about measuring or sorting you and figuring out what you know and what you could do. This is a very different kind of test.” So that can go along way and get the C students nodding: “Alright, I get it. I get why she put this assessment right here.” I think a lot of us have hit on the practical strategy too, that the little Easter eggs or goodies that we plant in the form of questions that get re-used on the higher stakes assessment. So most of us will have tests for measurement at some point in our courses. And yes, students really do pick up on it when you use one, two or more of those items that were in, say, the gamified quiz that you ran in class or the reading quiz that they did beforehand. They can see those and say, “Oh, wow, I got feedback on that. I got an opportunity to practice…” and if it draws in a few more students who see it as almost a legitimate form of cheating, honestly…. like a fun and sanctioned form of getting an advance sneak peek at the exam, then great! Then that maybe is an opportunity for them to come in and see that. So there is that. Actually taking it seriously ourselves, not just in the form of saying, “Well, here’s the points I’m going to give you for this,” but spending class time on it. That’s a big bridge for a lot of us to cross, right? Because we as teachers tend to be very focused on “Oh my gosh, class is for covering material. We use all these sort of distance metaphors to talk about what we want to do with our class time. But if I say, “You know what, guys, I believe in this, and I believe in it enough to where we’re going to spend the entire class period before the final exam….” I did that twice last week myself, when running exams. Or we’re going to spend five or 10 minutes of the beginning of every class period doing this, as one project recently published about doing. If you show yourself doing that, and offer them that, I think that also goes a long way towards it. And I guess to just say, well, taking it seriously… here again, what does that look like? What does that mean to different people? And we can kind of a little tongue-in-cheek say, “Well, why do we have to take it so seriously?” Sometimes games and learning can happen when it is presented in a more fun context. So not everything has to be deadly serious or spending hours and hours and hours of stressful time on. There are occasions when a light-hearted approach can be perfectly good and can still get us involved in that really critical activity of retrieval

Rebecca: I can share an example of doing that in my classes. We’ve done design challenges and sometimes we challenge other classes that are happening at the same time. That reinforces some of the basic principles that we think that students should be doing and reminds them of it… and they might work in a team…and then we have a competition. And it’s fun and what have you. And students like those, it breaks up the day, it makes it more fun. And then I’ve also done things where I give class time to do little design challenges in class that might be individual and then they can level up to working with a partner to finish solving a problem or something… and students value that. They recognize that next time they’re trying to do a project on their own,that it’s easier because they’ve had that practice or that opportunity for the retrieval practice. And my students have actually ended up asking for more of those opportunities.

Michelle: Great.

John: Could you go back just a little bit and tell us what you did in those couple of classes last week before your final. We’re recording this, by the way, in early December during finals periods in both of our campuses, but we’ll be releasing it a few weeks later… to put that in context.

Michelle: Oh, okay. Well, I’d love to. …and I’ve been talking about retrieval practice as you pointed out for years, and I still discover new ways to infuse this into courses. And the context for this is my cognitive psychology undergraduate course. It’s a 200 level. So it’s a lower-division course. And it’s about 60 or 70 students. And as you can imagine, it is a bit of a tough sell. For many of the students it’s their first encounter with this side of psychology. It’s not as intuitive as some other areas of psychology. So there’s a lot to learn and a lot of motivation to be done. One of the ways we bring this and in this course is using a technology called Kahoot, that’s spelled K-A-H-O-O-T,, and it’s really very intuitive and functions very smoothly… relatively free of bugs. That’s good stuff. It’s a program for doing gamified quizzes of various kinds. What I did, and at different points in the semester, and then really amped up in the last week of the semester is running these gamified quizzes. And this is something, by the way, that I hit on and got the idea to try based on a colleague named Niki Bray, who’s from Tennessee, and has actually done some really systematic work in reformulating some of her courses around in class quizzing in just really ingenious way. So I saw some of her presentation and I said I’ve got to try this for myself. I went with multiple choice questions. Kahoot does have some parameters… questions do have to be short…very, very short. And to some people that may be off-putting, but you can put together quite a few of these. And so we would put this up on the projector and students have the option of dialing with either their laptop or their smartphone and weighing in on each question. The neat thing about it is it has an algorithm for giving points based on your speed as well as your accuracy and it’s got a little leaderboard so you can actually have a little in-class competition. Now some people who use this do require all students to do it and they actually issue points for performance. Now, I presented it very much as a practice activity… and made it very, very clear because of my philosophy, I’m not going to assume that all students have devices or have smartphones or laptops or that they want to do that. But I said, “Look, remember we’re talking about retrieval practice, guys. So the real meat and potatoes of this is not buzzing it on your phone. That’s fun. But the real benefit of this activity is what you’re doing sitting there in your seat. And you could be doing this with a piece of paper if you want.” And that is what a few students opt to do. They try to answer the questions, they know what they need to go back and review and so on. And it’s nice because it spits out at the end, a whole report that tells me right away… Okay, which questions do we need to revisit? Which ones did students have the hardest time with and so on. That’s one of the things that I just brought in. And yeah, it was a big deal. I sacrifice a chapter of “coverage” so that we would have more time at the end of the semester. But to me, I would rather have students going into the final knowing that they’ve had this retrieval practice and they have a better chance of performing really well. Earning a good grade on this material I care about than honestly cramming in a little bit more mileage in terms of the quantity.

Rebecca: Sounds like fun.

Michelle: You know what, it does really bring a fun factor. There’s been a lot of different variations on in-class polling, and I will admit to this, I actually purchased and am the proud owner of a physical buzz-in quiz device, complete with a whole spaghetti nest of wires and an incredibly abrasive, buzzer sound, and everything. So previous to this, my educational technology did include… I think I could have up to eight intrepid volunteers who would play a quiz game and then I would have to appoint a points keeper and all this… and props are always a lot of fun. So when I say I believe in bringing in retrieval practice, I really do walk that walk. But I will say doing it by a smartphone does allow for more participation, and I don’t have to worry as much about the minutiae of scorekeeping and stuff like that.

John: I played with Kahoot a little bit at my classes at Duke and students have loved it.

Rebecca: We’ve talked a little bit about framing things so that students take the practice seriously. What do we do about the students who just push back, it’s like “This is too much work. This is a lot of extra time…” or that sort of argument.

Michelle: I think that that’s another piece of that barrier to more faculty adopting this, not just the work involved, but realistically, student opinions. Student evaluations matter a lot to faculty life. And of course, we all want to have that wonderfully rewarding semester, not the semester where we feel like we’re at odds with our students. So I do think a little piece of this is we do anticipate sometimes worse and more pushback than what actually happens in the end. So I think we have a fair amount of sort of a dread factor when we go into something new like this. But that said, when students have an issue with more quizzing or more testing, here’s how those come out. I think, first of all, we do have to sometimes separate out the technology aspects of it from the quizzing or testing, per se. So, as John mentioned, this is one of the amazing things that educational technology does. But the flip side of that is, if you’ve ever used technology for education, in any shape or form, you know that it breaks down. And when it breaks down at a big class, that’s a headache for everybody, it’s a misery. So if you’re trying it, and things are not working out, you just got to figure out “Okay, how much of this is the assignment, the activity per se, and how much of it is that wonky quizzing thing that I got from the publisher and it fell apart, or students hanging up when they’re trying to dial in with the poll and fine tuning those. I mean, when I first used kahoot, I decided it would be a lovely idea to put four chapters worth of material into a 40-question quiz. And when you get used to as you know, that they took a long time, I mean, 12 questions can keep you going for a very long time, especially if you’re discussing… and some students got kicked out part way through and then they weren’t on the leaderboard and I could have set the whole thing aside. But really, that was more I needed to get the technology working. So that’s a big piece of it. I think that sometimes it is a perception issue with the timing. Now, those of us who are just all over this as a teaching technique, we like to do reading quizzes before we talk about that material in class. So chapter three is up on the syllabus and your chapter three reading quizzes due on Sunday night before we do that…, and that can provoke a fair amount of confusion and honestly griping with students. They say, “Why did I get tested on this when we didn’t do it… we didn’t cover it in class yet?” And what do you know, that’s another framing and communications issue. Once we know that that is why we’re doing that you get so much less on that side of things. But here too we do also ourselves have to follow along with what we say. If we say you’re doing this reading quiz so that we’re establishing a foundation. we don’t have to teach everything in the class itself. We could spend the time applying. Well, guess what, then you do have to do that. So if I assign you to do the reading quiz on chapter three, and then on that Monday morning I go, “Okay, we’re going to go over chapter three starting at the top. And I’m going to show you all the PowerPoints for all these things that you already read.” Well, yes, then student morale and student support will fall apart at that point.

Rebecca: Those are very good reminders.

John: Yeah, I’ve pretty much adopted this approach all through my class beginning when I first saw you present it. So I have students do the reading in advance, I have them take reading quizzes with repeated attempts allowed on those. And then in class, I have them working on clicker questions, and I and the TAs go around and help them when they get stuck on problems. But there is an adjustment and students, especially when I first started doing it, would generally say, “But you’re not teaching me” …and you do have to sell students on this a bit. One source of resistance is that when students take quizzes, they often get negative feedback. When they read something and they read it over again, it looks more and more familiar. They’ve got that whole fluency illusion thing going and they become comfortable with it, and they feel that they’re learning it… or similarly, if they hear someone give a really clear presentation on a topic, it feels comfortable. They feel like they’re learning it until they get some type of summative assessment where they get negative feedback, and then they feel the test was somehow tricky. But it’s a bit harder to convince them that actually working through retrieval practice, watching videos at their own time and pace, reading material as needed, and then spending class time working through those problems is as effective. I’m getting better at it. But it’s been a long time trying to convince students and I did take a bit of a hit in my evaluations, especially the first few times.

Rebecca: Do you mean learning’s not easy?

John: Students would like learning to be easy.

Michelle: You know, it’s funny, I think almost in a way… see what you think about this idea. But it’s almost a mirror image of our illusions as teachers, right? That I gave a wonderful clear lecture and I assigned wonderful readings and I saw students highlighting them. Therefore, students must have assimilated this knowledge and it must be in there. So I think it challenges our students but it also challenges us as well. It can be quite an eye-opening experience to running something like a Kahoot. And that brilliant point that I gave this great example for…. what do you know…. 7% of the students actually nailed it. And so we can all use a reality check… teachers and the students. And you mentioned re-reading, that’s another one where I’ve had some pretty intensive conversations with other faculty, I’ll kind of say, “Oh, well, and there’s this great research that shows that students tend to re-read when they’re studying and we know that from a memory standpoint, that is really, really ineffective.” And faculty will say, “Whoa, whoa, whoa.I want my students to be re-reading.” Of course, now when they say re-reading, they may be picturing deeply interrogating a text… annotating it… looking at it from a different perspective. And absolutely, that’s a wonderful part of scholarship. But that isn’t what we’re talking about. We’re talking about students re-reading as a study form and mistaking highlighting for deep interrogation. So, just like with the rest of knowledge bases versus higher-order skills. This is another words “both and” it cannot be “either or.” Yes, we want students re-reading in the right ways, but students or teachers cannot mistake that for learning sometimes.

Rebecca: So, let’s say you’ve just convinced all of our listeners that we need to be doing this, how do we bake it into our course designs?

Michelle: Well, I think that, really getting creative with this, and as I talked to faculty when I visit other schools or talk to faculty of my own institution, I just see all of these new ideas all the time about how to do this. Once you do get that critical epiphany of alright, a test doesn’t have to look like a test on the surface… it does not have to be the ritual of “Okay, you’ve got a number 2 pencil that’s breaking and I’m standing over you while you have a panic attack for an hour.” That’s not what it is—that it’s about the retrieval. Once you get that, all kinds of creativity opens up. So I’ve also started sometimes on the very first day with the syllabus quiz. So, especially if you have a smaller class, we all struggle with that “I spent all week writing the syllabus and it’s incredibly important, and then I’m getting questions about stuff that’s on it all the way through the semester.” So, really on the first day, what I do is I take a very light-hearted approach and I divide the students up into teams, just physical teams. Everybody’s got a copy of the syllabus, sometimes I fake them out and I say, “Okay, we’re going to go over the syllabus point by point…” and I say, “No, you can read the syllabus. We’re actually going to do this other thing which research shows will actually help you remember it.” So the task here is that each team formulates a set of questions… you can make just a few. like three… so a little bit of teamwork. So formulate three good questions off the syllabus. And you’d be amazed, they come up with questions that even I can’t answer sometimes without having to cheat. So they talk about it. And then of course, you go around the room in some arrangement and each team gets to ask the other team their questions, and if you really want you could keep score and everybody loves bragging rights for being the team that stumped the other teams and won the most points. So, it can literally start right then. With that idea that I don’t read stuff to you in this class, this is about you. And I’m also not piling a huge amount of work on myself either apart from being the moderator and the MC and having written the thing in the first place. I don’t have to write questions. They’re writing the questions and they’re answering them too. So those are some of the ways that we can do that. I think especially in our larger classes we do want to think about things like peer grading or peer review of open-ended question responses. We do want to take advantage of things like publishers’ test banks to set those up as reading quizzes. And you’d mentioned earlier about “Well, what about this not being suitable for upper-division classes?” I had an upper-division class that just wrap this semester where we had reading quizzes as well. It may have been an upper-division course but it also serve that purpose of “Hey, you take a basic quiz over the chapter on Sunday and that really sets the stage for us to have a more substantive discussion.” All of those things. are ways that we can do this. I think open-ended reflection as well… so tests that look nothing like a test but are still retrieval practice. You’d mentioned about reflecting on what you learned last class period. So this sometimes goes by the name “brain dump.” And I did a bit of this last semester as well. So, this by the way, I do want to credit the great website retrievalpractice.org. So retrieval practice is actually that high profile it has its own website. It’s an absolute treasure trove of ideas. So, with the brain dump the way that I did it, or similar to what it sounds like you did, every now and again we start off class with you writing down everything you remember from last time on a piece of paper. I had students then turn to their neighbor and compare notes and see what they came up with. And I didn’t grade those. This was not a heap of grading for me. But in the end what they did turn into me for accountability and a few points was a very short reflection, just on an index card. So, “What surprised you?” and I review those and they’re very eye-opening but again I’m not there to police or micromanage what they put on their cards. So, that’s retrieval practice too. There’s lots of different flavors that we can bring in.

John: I do something similar with asking students to reflect on the reading in each module we work through. But because I teach a fairly large class, I didn’t want to have to deal with all the index cards. So I just have them fill up a simple Google form. And then I can skim through it and assign grades much more easily than shuffling paper. But it’s the same basic idea and it’s worked quite well.

Michelle: Absolutely.

Rebecca: Do you have any other examples of the kinds of little tests that faculty can run that don’t look like tests?

Michelle: Right? So tests that don’t look like tests. Well, here’s another that was probably a little bit more practical for a small class, and this is how I ran this. It, like the brain-dump exercise the way I ran it, was also very cyclical and very student generated. So we started out each class at the beginning of each week with students generating a set of quiz questions based out of the assigned reading. So we didn’t actually, in this particular example, have pre-quizzes or something like that. But students came in knowing that they can bring their book if they wanted. But they would need to sit down and write for me three questions, whether short-answer or multiple choice. Then I can flip through those and I would really quickly put those together for a quiz that went out the subsequent day. So, we’re alternating between generating questions and then returning to those questions. And then I would have pass out. I told them treat this like a realistic test. Actually try to retrieve everything, but, you know, when times up, you’re going to get to go back to it. And then they would grade it themselves. So I didn’t actually do the grading either. And that is really great for spurring discussion. And “Oh, my gosh, I thought I knew the difference between reliability and validity, but now that I tried to answer it, I realized that I didn’t.” And then you can throw it back out to the student who now has bragging rights for having had their question selected and say, “Hey, what did you mean? What was the right answer? And why did you put that down there?” And at the end of the day, they kept that quiz too. So it was really very much in their own hands to do. So there’s that. Other creative ideas that I’ve run across over the last couple of semesters… There’s a great project out there, run by Bruce Kirchhoff at University of North Carolina at Greensboro. So I got the wonderful pleasure of talking to that group last year. And he and some colleagues working in the area of botany actually put together a freestanding, custom built mobile app that students could take with them that presented different kinds of quizzes over the sorts of things that botany students really need to know like the back of their hands like how to identify different plants and how to discriminate among different examples and they found some empirical evidence that this actually raised performance up quite a bit. I’ve heard too of another Professor put together some surveys in Qualtrics, the surveying software took advantage of its ability to actually text message people and send them the questions. So this was an opt in sort of activity and it’s one that they didn’t just have it run 24-7 because it was a little intrusive, but what it did is it sent students questions that they could answer at different intervals throughout the day, which also takes advantage of another principle from applied memory research which is spacing. So students are getting these unpredictable questions, they have the option to answer them, and they could be happening even when they’re not in class. So those are some other ways. Some of them look like tests, some of them don’t. But those are creative ways to get students engaged in that practice.

Rebecca: As a faculty member, we often advise students and mentor students who might be struggling in other classes, ones that we don’t have control over. Are there ways that we can help those students use this methodology to do well in those classes where it might not be embedded?

Michelle: Right. And that’s so great that you bring that in as well because ultimately that is what we need as teachers and that maybe circles back to yet one other piece of objection that I’ve seen… actually this time in a published article from a few years back that said, well are we doing too much for students? Are we scaffolding them too much so they’re going to grow really dependent on these kinds of aids like reading quizzes, and reading questions. But if we also have in our mind very intentionally, that what we want students to have at the end of the day is also something they can walk away with, I think that we do have to be very mindful of like, “Okay, let’s not create the impression that just because retrieval practice is so important that you have to sit there and wait for me to put together this specific kind of reading quiz for you.” So I think here, the really powerful message is once again that one students take this to heart… once they’ve not just been told this, that “Oh well, you should quiz yourself as a study strategy,” but they’ve seen that I believe in it to the point where I’m going to put time, energy, and work into it as part of my class. And maybe they’ve even seen the results… they’ve now had their own little before and after experience of what happens when I do this… that they can be more likely to take this forward. So having more faculty across the curriculum endorse this is a powerful idea that “Hey, this is how your mind works. This is how your brain works. Your brain doesn’t just soak up stuff that’s in front of it. You soak up stuff that you have to answer questions about.” That I think is going to be a powerful message. And there’s actually another article out there that I just absolutely love it was done by a psychologist named Pennebaker and some colleagues at University of Texas at Austin some time ago. They replaced high-stakes assessments in their Introduction to Psychology course with these mobile in-class quizzes that were done every day. And one of the things that they report in this article is not just that students did better in that class, but that certain subgroups of students actually showed improvement, specifically closing of achievement gaps, really, in classes that they took after the psychology course. And these were classes that we’re not even in psychology. So you think about that for a minute. How does that happen? Now, nobody knows exactly for sure, because this was kind of an unexpected finding in the study was my impression, but a real possibility is that when students have sat in this class and every single day they have seen the power of taking a quiz… of spacing out their learning… and attacking it in a very active learning approach… Once they’ve seen that happen, they’re more likely to go home and say, “You know what? I can do this in my biology course too. I don’t have to sit there with the teacher’s quiz. But, if I attack it in the same way I might get the same results.” So I think with those things, we can have students walking away with that enduring practice that we want them to have. And it is funny too, because it brings up one of the, I’ll just say it was a really heartwarming teacher moment that I had this last semester when I did bring in these Kahoot quizzes. In the run up to the first exam, I had done my fancy little in-class quiz and was kind of patting myself on the back of what a great leader I was. So I came in to, I think it was the class right before this exam, and I come plowing into the classroom and it’s dead quiet. And there is a student at the front of the classroom. He has commandeered the podium and the computer and what has he done? He’s accessed the Kahoots, which I gave them the links. You know, they’re out there for all the students to see. And he is running them and the rest of the students in the class are taking them just as seriously as when I was administering them. So they’re running through the questions again, giving it another shot. I didn’t tell them to do this at all. It was one of these moments where I just backed out and closed the door after me. And I came back in when they were finished. And so I think those are the kind of moments that we can set ourselves up for when we really do bring this into our classes and we get behind it and a really authentic way.

Rebecca: I think one thing Michelle, in the way that I asked the question, I had also asked it from the perspective of a student hadn’t had the experience of doing a retrieval practice in a class. So, if you’re working with a student, maybe outside of class or as an advisor, are there things that we could do to help those students adopt those practices, even if they haven’t seen it modeled for them?

Michelle: So how do you help students when they haven’t had some of these experiences on their own? And I think this is a part of a bigger package of goals that I think a lot of us should have in supporting our students to really put students in the driver’s seat. To say, “Yeah, you don’t have to wait for a certain style of teaching or certain subject material in order to succeed with it.” I see that isn’t really part of a larger package of growth mindset honestly. So what can students do to make themselves the masters of this? Now, I think that there are some old standard, very traditional, approaches that are worthwhile. And when you look at those approaches, sometimes retrieval practice is at the core. So it may not be a matter of trying to get a very unfamiliar set of terminology or anything for students before, but really getting them to look at some of these approaches in a new way. So things like you’ve ever heard the term SSQ4R or PQ4R, there’s a couple of those that have acronyms and the things they have in common are that they tell students when you sit down to study or read a text or prepare for a test, here’s what you do. You don’t just start reading from the top with no goal in mind other than “Oh, I want to get a good grade later.” What you do is first of all, you set yourself up with questions. Which is, after all, a lot like retrieval practice. Start with a question, say, “What do I want to answer? Can I answer that now?” And if not, why not? To read your text or go through your material very intentionally around this questions, and then there’s always this piece of recite or review. That’s what one of those Rs stand for in some of those traditional systems. And that means closing the book. That means closing the book and saying, “Okay, I’ve sat here with this material for this amount of time. What can I actually say about it at this point?” So, sometimes directing students back into some of those and saying, you need to adopt these strategies, which are completely teacher-independent, they’re fairly discipline-independent as well. That can be good. If you’re doing them for the right reasons and with the right approach in mind, that’s very good. Encouraging students to take advantage of the publishers’ companion sites… Now this is a little bit more of an uphill climb. That is where we run into “Well, if there’s no points for it, Why should I do it?” But encouraging students to say “Look, you already paid for this textbook. To be crass about it, you paid all this money, did you know there’s a website over here and if you interact with those materials in this particular way that’s really, really likely to pay off for you?” So, besides just ensuring that our students have what I consider to be these basic foundational pieces of knowledge about the mind and brain: that we remember through testing, we remember through retrieval and an active engagement. All students should have that, but those are some specific things that we can counsel students to use across all their studies that really should pay off.

John: Do you have any other suggestions for those faculty that are thinking about expanding their use of retrieval practice?

Michelle: You know, just to really encourage and support faculty who are starting this journey, as it sounds like that you all have, to really re-examine how we can bring in this incredibly powerful principle, and to really reassure each other that “Yeah, this is not about really just piling so much more work on yourself.” We can sometimes even just re-examine the assessments and assignments that are already in the course and so that’s kind of one last piece of practice that I think that a lot of us can really stand to bring in. And that was a big thing for me as well to say, “Well, if I’m going to administer these tests… we administer tests, we administer midterms anyway… what else can we do to increase their value as learning experiences… as learning events. So here’s another where I’ve brought in various forms of test discussion activities. Instead of standing in front of the class with that deadly the day-after-the-test class period where I say, “Let’s go over it” to realize, you know what? That probably will not work that well. But one of the amazing things that we have learned from the research on retrieval practice is that it’s not just in the moment in the taking of the test, where this advantage to memory happens. It actually creates a sort of a receptive window for learning and for review when we’ve just taken a test on something. So if you’ve got a midterm in your class already, well, hey, why not carve out the time after that test is up to say give it back to the students… like I photocopy the exams before I’ve graded them. They have no feedback on them whatsoever. And I hand them back to students as a group discussion exercise. I say “Alright, here’s a blank copy of the exam, your group can fill out this blank copy together, just knowing what you know, revisiting all those questions, having those good discussions about what you understood and what you didn’t. And I offer a little bit of extra credit for really good performance on that. And there’s other ways to work that out so it actually takes off as a small group exercise in class. But regardless of the specifics of how you make something like that work, the spirit of it is the same. That when you sat down to do this, to try to drag all this information out of memory, that is not an end unto itself. It should be part of a bigger picture of learning in the class. And it’s sort of an untapped vein of potential we have as teachers and that our students have as well and that we can access it regardless of what the discipline is or how the class is setup.

John: …and students see it’s very relevant. I started doing something very similar in my econometrics class last year following a suggestion from Doug McKee who had been doing that in his class. And it worked remarkably well… and turned what was normally a pretty unproductive class period where we’d go over the test… and the people who did well with just be really happy and pretty much ignore any discussion, and the people who did badly were just sitting there unhappy and not really being very receptive. But when they sit there, and they’re explaining it to each other, it seemed like a really ripe time for them to learn the material much more deeply.

Rebecca: I remember the first time that you implemented that. You sent me a text message with a photograph of his students taking a test and it looked very active. [LAUGHTER}

John: …and they were having fun.

Rebecca: Yeah.

John: One of the nice things that came out as I was wandering around listening to their conversations, and I was hearing people say, “Oh, yeah, now I see where that came from.” Or someone would say, “Well, when did we learn this?” And then someone else would say, “Well, remember when we worked on these problems?” …and it just helped them make connections… and the power of peer instruction is so remarkable. I’m going to do it in as many of my classes I can.

Michelle: That’s perfect. That’s what we want as teachers, and that’s what our students benefit from.

Rebecca: So as you know, we always wrap up our podcast by asking what’s next?

Michelle: Oh, wow. So I am looking forward to kind of rebooting a course that I have not taught in some time. My senior capstone course in technology, mind, and brain. And that is a fun one. But it’s one that is going to take a lot of revision, since it is technology and things change so rapidly, as we all know. So that is going to be a big part of next semester. I also have a crop of research projects in various angles on teaching, learning and educational technology that I’m really excited to be moving forward in the next calendar year. One of those… really foremost among them is a project on virtual reality for learning. We have an incredibly creative and dynamic team looking at virtual reality here at Northern Arizona University. They put together an amazing series of interactive exercises that are part of the organic chemistry course here and teach some of the challenging concepts in that extraordinarily challenging gateway course. And so we now have a whole set of data from students who went through and did this at varying points during the semester. We got their feedback, we’ve got all different kinds of psychometric measures that we gathered from them at the time as well. So, I cannot wait to be tackling that and looking at all kinds of angles on how this part of technology is impacting student learning in this course.

Rebecca: Sounds like some exciting adventures.

John: That sounds like a wonderful research project. …looking forward to seeing what you find.

Rebecca: It was a real pleasure to talk to you again, Michelle. Thanks for spending some time with us.

John: It’s wonderful talking to you. Thank you.

Michelle: Oh, likewise, always a pleasure to talk to your listeners.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.