Category: active learning

109. Active Learning

Posted in active learning, collaborative learning, evidence-based practices, higher education, learning, motivation, pedagogy, peer instruction, STEM, teaching
November 27, 2019

Moving from a familiar instructional format such as lectures to a more active learning environment can be daunting. In this episode, Dr. Patricia Gregg joins us to discuss how she flipped her classes and embraced active learning. Trish is an Assistant Professor of Geophysics at the University of Illinois at Urbana-Champaign.

Show Notes

Transcript

Rebecca: Moving from a familiar instructional format such as lectures to a more active learning environment can be daunting. In this episode, we share the story of one faculty member who fully flipped her classes and embraced active learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Patricia Gregg. Trish is an Assistant Professor of Geophysics at the University of Illinois at Urbana-Champaign. Welcome.

Trish: Thanks for having me.

Rebecca: Today’s teas are:

Trish: I am drinking a peppermint decaf tea.

Rebecca: …in what looks like a very nice handmade mug.

Trish: Yes! This was made last summer at the YMCA of the Rocky Camp in Colorado.

John: My tea today is a Harney and Sons chocolate mint.

Trish: Mhmm.

Rebecca: And I have a Prince of Wales tea today.

John: We’ve invited you here to talk about some of the active learning techniques that you’ve used in your class and also a little bit about how you use a flipped classroom approach. But, before that, could we talk a little bit about your own experience in science classes, and whether active learning was common while you were a student?

Trish: I was thinking about this and it’s interesting because it’s sort of a yes or no type of situation. Geosciences in general is the fun major in that we pull together a lot of different disciplines. So, you have chemistry and physics and math and computer science and you’re using those all in applied ways to understand the structure and evolution of the earth. And so our classes typically have a lecture-based meeting time and then a laboratory that’s associated with it. So, when I was matriculating, most of my classes, there would be three one-hour meetings throughout the week where we’d be lectured at, and then we’d have a three-hour laboratory class at some point during the week or a field experience that would help to apply some of the knowledge that we gained in the passive-learning setting. But, then as you get at higher levels, and things become more theoretical, it really did switch to more of this passive-learning mode. And I don’t want to age myself, but I matriculated a while ago, so I didn’t ever really experience these new active-learning techniques that have become so much more widely adopted nowadays. So, even through graduate school, most of the classes were me sitting passively scribbling furiously to try to take notes as quickly as I could, while a professor lectured and basically tried to stuff as much knowledge into my brain as possible. So, I didn’t really know a lot about the types of things that you can do to engage learners until after I was out of that student mode. But, yeah, geology is cool, though, because you still do have active portions where you get to go on field trips with your professors, and they show you things in the field and you apply that knowledge directly. But, in the classroom, it really was sort of divided, like, “This is your passive lecture that you’re going to sit and listen to, and you may never get called on through the entire semester.” And then “Here’s your lab where you will look at a microscope and look at hand samples or do other types of things that are a little more active.”

Rebecca: What motivated you to do something different in your own classroom?

Trish: As a graduate student, I really didn’t have a chance to do a teaching assistantship. I was on fellowships through most of my PhD time. So, I knew that I was woefully underprepared for entering academia and teaching my own classes. So, as a postdoc, I applied to this call that I saw out by the Center for Astronomy Education. And it was in 2011, they had this course called Improving College General Education and Earth Astronomy and Space Science through Active Engagement. And I saw the ad for this course and I thought, “Oh, this sounds great.” And then I saw that it was three days in Hawaii, and I said, “Oh, man, I must apply to this.”[LAUGHTER] And so I applied and it was mostly astronomy graduate students and postdocs, and the workshop was run by Ed Prather and Gina Brissenden out of the University of Arizona through the Center for Astronomy Education. And they had been doing all this amazing research about how to engage students in 100-level classes, mainly for the idea that they would sort of entrain new majors and new science students. But, it was just a mind blowing experience. I for the first time learned what think-pair-share was, I’d never heard of that before. We did lecture tutorials, I didn’t even know that was a thing. They did all of these, like voting and role playing and these different pedagogical things that I didn’t even know it existed. And they use them on us throughout the workshop. So, we were learning about these techniques through them actively using us as guinea pigs. And then we each had the opportunity to sort of develop a little module. They gave us specific astronomy, like 101-type things, that we would be teaching and we got to teach the other workshop participants and get feedback immediately on things that we didn’t do so well and things that we could improve on. But, it really just blew my mind. I think that was one of the most transformative experiences for me because, up to that point, all the experiences that I had, had been very research focused and how to improve as a scientist and how to improve my research approach, but I’d never had an opportunity to actually learn how to teach and how to teach effectively. So, yeah, I credit that three-day workshop in Hawaii, which was awesome… to be in Hawaii. It’s just sort of changing my entire worldview on how education can be and how I could be a better educator. Had it not been for the Center for Astronomy Education, I don’t know what I would be doing now. So, I think what I took away from it more than anything is that not every student is going to learn simply through lecture… passive engagement… type of situation. And I was fortunate that I seemed to do well in that mode, but it was amazing. I loved that workshop……. It was great.

Rebecca: It sounds really transformative. But, the one takeaway that I hear is next time we want a faculty member to change what they’re doing, we just need to woo them to Hawaii. [LAUGHTER]

Trish: Well, I have to admit that being a postdoc gave me some flexibility in that regard. So, yeah, when I saw that call, I was like, “Oh, I want to do that… three days in Hawaii.” I took my mom with me and she hung out and snorkeled during the day while I was in workshops. It was wonderful.

John: Could you tell us a little bit about some of the techniques that you’ve used in your classes?

Trish: The first semester that I taught, I was given a class that had already been developed and it was sort of easing me into that mode of becoming the head lecturer of a course. So, I didn’t really have a lot of wiggle room to change the curriculum yet, because I was still sort of learning how one gets in front of a class and does things. And so in that first term, I started to use some of the approaches that I had experienced through the Center for Astronomy Education, and sort of trickled them into my class. I use lecture tutorials and think-pair-share a lot during that term. And then I even used some small group activities and jigsawing to try to figure out ways that I could engage the students. And it was sort of a perfect situation to get my feet wet because I had the scaffolding of a well- developed course where I could put in some of my own ideas and try them out and if things weren’t working, I could get immediate feedback from the students and change my trajectory. I was also really fortunate that the students were super kind to me, it was my very first time teaching, I told them straight out. I was very communicative throughout the course. Every time I tried something new, I’d say, “okay, we’re going to try this. I don’t know if it will work, but this is why I’m doing it.” And the students were sort of brought in as collaborators in that process, so they didn’t see me as sort of this professor that was telling them “Oh, you’re going to do this, this and this and just follow along and trust me blindly.” They realized that I was trying to learn how best to teach them and so they were very helpful and when things didn’t work, they’re like, “Yeah that didn’t work.” And then when things went well, they say, “Oh, I really liked that.” And even after that semester, I’d get emails from students. They do a lot of journal reading and science reviews. And one of the students had emailed me over the summer and said “Oh, that really helped. At my first job they asked me to review some literature and I was able to use the template that you provided in class and what we did as groups to do that for my job.” So, I gained a lot of confidence through that process. And then after that first term, I started looking around campus to see if there were faculty development potential to help me to do a better job of developing my next courses. Because while that one had already been developed, I was then sort of slated to develop three new courses, which would be mine and I’d have to start from scratch and really think about how I wanted to develop my teaching as a portfolio. So, one of the things that I really wanted to try was this idea of flipping and mainly it came from a place that I didn’t enjoy lecturing. I would get bored hearing myself talk… like there would be times where I’m up there at the dry erase board writing out things and then suddenly I forget what I was saying, because mentally I’d fallen asleep at that point… like, alright, I’ve been talking so long, I don’t even know what I’m talking about anymore. And I enjoyed the parts where we were actively learning as a group so much more. That was so exciting to me, where the students were doing things hands on, and I could walk around and help them to gain more insight on what they’re working on. So, I contacted the Center for Innovation in Teaching and Learning on campus and looked at the different things that they had available. And one of the facilities that they had advertised on their page was the Illinois iFLEX classroom. So, this is the Illinois flexible learning experience classroom. And I was able to get some training on how to use IFLEX classrooms from Dr. Eva Wolf here on campus. And that also then immediately changed my perspective of how teaching could be because these were classrooms where all of the tables were on wheels. So, you could move them around. They had monitors that students could plug in their computers or laptops, iPads or whatever too so that they could do collaborative learning. And she showed me some of the things that other faculty members were doing in these flexible spaces and it helped me to be inspired to think about what sorts of things I could do. So, as I started to develop my next class, I was like “Alright, I want to be able to use the computing facilities, I want to be able to use these flexible classroom configurations and I really want it to be flipped.” So, the first time I taught a flipped class, I recorded lectures and put them online and naively thought my students are going to watch them and they’re going to do the readings and they’re going to come to class prepared, and I did not have the assessment structured as such that the students had points awarded for doing those things. And boy did I learn quickly that students are not going to be these wonderfully motivated pupils that do all of the things on the list ahead of coming to class. So, I quickly spoke to colleagues around the department and found out about this edtech tool called PlayPosit. I don’t know have you guys had an experience with PlayPosit?

Rebecca: No.

John: That one I haven’t heard of.

Trish: So, PlayPosit is an edtech tool that you can integrate with a learning management system. We use Moodle on our campus. And you take your video lectures, and it embeds questions and prompts within your video lecture. So, students can’t fast forward and they don’t know when these questions are going to pop up. But, it’s a way to assess how they’re doing with the video… with the lecture as they’re watching it. So, sometimes I’ll use multiple-choice questions. In the upper-level classes I mostly use essay questions because I really want them to delve into the topic a little bit more. I also sprinkle in questions from the readings that they’re supposed to do because it’s another way to assess that they’re actually looking at the text or reading the papers that I’ve suggested. And then at the end, it’s great because you can put in some questions about what concepts did you not understand? What do you want to learn more about? Are there sticking points that are kind of confusing you? And this fed directly into learning about the just-in-time teaching method. So, I could have these PlayPosits that the students had to watch before class and I set them for midnight the day before. And I could come in the morning before class, assess how they did on the lecture and immediately I have a lot of information going into the classroom that day for where they’re stuck and I could modify my approach to the learning goals for that day based on how they did on their PlayPosit. And that just changed everything. That made it so much better, and I think from the students perspective, they felt more accountable because there were points that were associated to watching these lectures. And then I would come into class and the first thing I would do is sort of go through the questions and the things that they missed and talk to them about it. And it gave this really nice back and forth. And it sort of broke the ice a bit, because there’s always that little awkward start when you get into classroom, or at least there is for me, and this was an easy way for, say, “Okay, so on the lecture that you guys completed for yesterday, here are some topics that you didn’t really understand. So, let’s go through them together and maybe we can make sure that everyone’s on the same page.” And that sort of changed the game for me for the flipped classroom model.

John: Going back to the PlayPosit, you can also do the same thing with Camtasia and upload the videos as a SCORM package into Blackboard, Canvas, or other things as well.

Trish: Oh, I have to check that out.

John: Once the students arrived in class, you mentioned that you used a just-in-time teaching approach. How did you structure the class? What would your class generally consist of during the class time?

Trish: I originally taught on Monday, Wednesday, Friday for 50 minutes and realized that that was not a long enough time period for us to do what we wanted to do. So, I switched to a Tuesday-Thursday class so I could have a full hour and 20 minutes with my students twice a week. So, typically when students come to class, we have this sort of icebreaker where we go through the lecture material. And sometimes that might take 10 or 15 minutes if there’s a concept that the students really need to get for us to do our activity for the day. And then we usually go straight into a prompt for what the activity is going to be. So, for example, one of the classes that I teach… my favorite classes… junior-level class in volcanology… so, it’s just called volcanoes. And it’s a sneaky class because it’s actually a geophysics class. It’s very math and physics heavy, but I don’t tell the students that when they come, and they do not have an upper-level math requirement to take the course. And this was sort of my sneaky way to entrain students that might not realize that they can absolutely do this. So, I get a lot of diversity in that class: we’ll have communications majors, advertising, education majors, as well as the geology majors. The first time I taught the class, of the 20 students in the class only 4 were geology majors and the other 16 were just spread from throughout the campus. So, it was a really cool opportunity to empower students that “Wes, you can use math and physics and it’s not that intimidating”. So, we go straight into these activities and every exercise is quantitative. They get real geophysical data from deforming volcanoes and active volcanoes around the world, and they analyze it. And there’s a large social sciences component because I want them to think about the societal impacts of those volcanoes and potentials for eruption and how it might impact the communities that are around the volcanoes. And then also that communication thruway of how as we as scientists communicate hazards to local populations. So, they have a lot of different levels of work that they’re doing. Almost every class period is done in a jigsaw manner, where they’re broken into small groups and each small group is going to be working on some component that at the end of the class we’re going to come together and discuss. I typically start the prep of the activity, for example, “Okay, today we’re going to be looking at this type of volcano.” And maybe it’s a stratovolcano. And we’re going to look at Mount St. Helens in the US, we’re going to look at Mount Fuji in Japan, we’re going to look at Ruapehu in New Zealand. But, each group will have a different volcano. And they’re going to look at data directly from that volcano and the surrounding areas and do a small activity that helps them to understand that data set, the type of physical processing, and then at the end of class, each of the groups will come back together and present what they found to the group and then there’s some larger full-classroom group discussion questions that will go over as a class. So, it’s usually the activity I hope will take about 45 minutes, but it really depends. I try to keep them short in my mind, but then oftentimes they go a little bit long if the students get really excited about it. One of the things I think is really good about the flipped class is that I’m able to do so much more than I was able to do in sort of a classical passive learning model. And with that came a lot of grading. And so the first term I did flipped classes, I had not learned about light grading, and was buried in the amount of feedback I was trying to provide to students. So, if we had PlayPosits a couple times a week, we had these activities twice a week, they had additional outside of class things, they had midterms… so, it was unreal, the amount of grading. So, I was very fortunate to find out about light grading, and how to maybe back up the amount of feedback and time I’m spending on student papers and that really helped a lot. So, I think that one of the things that has to be said in conjunction with this particular model is: you need to do some sort of light grading, because there’s no way to stay on top of everything without losing sleep. [LAUGHTER]

Rebecca: Can you talk a little bit about how you adjusted your grading in specific ways?

Trish: Originally, when I did the PlayPosits, again, there are a lot of essay questions. So, I would take a lot of time to really think through the answers to the essay questions and making sure that I’d have a rubric for what I wanted, what points I wanted them to hit on each of these essay questions, and I was very detailed about when they miss things and providing feedback. And so as I shifted into a light grading model, I would do that for the first couple of weeks. And then from then on out, it would just be a quick glance, like, did they hit this? And I wouldn’t spend as much time with the subtleties of “Yes, they wrote out a great answer, and they hit all these points.” And then for just the in-class exercises, the thing I started doing too was originally I had each student turn in their own exercise, even in the groups because I wanted to see their individual contribution. But, I recognize that there was enough individual assessment through the playp osits, and through their midterms, that having that additional individual assessment through the group activity really wasn’t necessary. And it wasn’t really contributing to their success in the class. By having individual assessment on the group assignments, it wasn’t helping students who were falling behind do better. And so after that point, I started allowing the groups to just do their presentations as a Google slideshow, and then I would have their Google slideshow. So, basically, in the jigsaw puzzle, they go into their group, they work on this presentation and when we come back together, each group shows a Google slide or Google doc of what they’ve been working on, and they present it to the entire class. At that point, I just say that’s good enough and I don’t require that each student then hands in the answers to the discussion questions in their own words. So, little things like that made it a lot more tangible for me. Whereas before when I was having each student providing responses for the discussion questions, and then on top of that having the discussion in the class, it was just too much. [LAUGHTER] But, I admit that it came from a perspective that I was concerned that some students would not fully contribute to the group activity, and I wanted to try to hold people accountable, but it really was a little bit too much micromanaging. And I think that the groups ended up holding each other accountable in their own ways without needing me to sit and say, “Okay, everyone needs to answer this question.” The other thing I really like about the small groups is that I’ve noticed that it brings out a lot of discussion from students that otherwise do not participate in the larger group discussions. And one of my favorite things in those small group activities is going around the room… and I typically spend a couple minutes with each group, and I just sort of keep roving around the classroom and it helps me to get to know individual students a lot better. And it also gives them so much more confidence to talk to me. And I feel like it’s made me more approachable as an instructor because they’ve had these smaller group interactions with me where I’ve sat at their table and said, “Oh, that’s a great idea…” or “Oh, have you thought about this” and those just little micro interactions really build up and it creates a student population where they feel more comfortable in the class. And then by the end of the term, I feel like, as a class, it’s much more energetic and engaged. And even in those larger groups discussion, some of the quieter students that you would never have heard from previously are starting to speak out and oftentimes with the encouragement of their group members. That’s another thing I really like about the small group setup.

John: Could you give us some feel for the size of your classes? How large are they typically?

Trish: My typical class size is 20. I usually keep the classes up to 40 students because that’s what the flexible classroom configurations will hold. One of the interesting things about the flipped class is that the first day of class, I do tell the students, these are my expectations. This is a flipped class you’ve signed up for and we go through what that model looks like and I always have students drop after that first day. That’s kind of fascinating for me, maybe it would be nice to follow up and find out…. Did you drop because of the model that was being used in the classroom? Or did you drop because of a schedule conflict? Or were there other things going on? But, I typically end up with 20 students that end up through the entire term. I teach mostly upper-level junior-, senior-level courses. So, I’ve not had the opportunity to try these techniques in the large introductory level classes.

John: I think most of them should scale pretty nicely, except for the grading aspect of it.

Trish: Yeah, I think that were to be done in an introductory class, you probably would want to have some TAs involved as well, just to help. I recognize that other instructors do these amazing small group activities in these large format lecture classes. But, I think having the logistical setup so that you can walk around and interact with groups, maybe not every group every time, but enough so that you can hit most of the groups once in a while, would be imperative because I really think that the students greatly benefit from that almost one-on-one interactivity with the professor.

John: I teach a class typically between three and 420 students in the fall, and I do wander around and I’ve found something similar. I don’t get to sit with each group. But, the students that I do interact with become dramatically more likely to stop by and ask questions, or if they see me in the hallway, to come up and just say hello. So, those individual interactions can make a big difference in practice.

Rebecca: I think it’s just a far more efficient way to give feedback as well. You can disrupt misconceptions and reframe things for small groups. And then if you stop by a couple of groups and hear the same kinds of misconceptions, you can address those more holistically to the whole group. I found that works really well for me, too.

Trish: Yeah, absolutely. And I always get tickled when I see that. I mean, this isn’t necessarily a good thing. But, when there are lots of groups that have the same misconception, because it means that there’s something that… or a piece of information that I have not given the students or something that’s missing in how we’ve set up the activity. And that’s always kind of nice to see and it helps me to redevelop how I’m going to teach it the next time. So, I really do like that. Because otherwise, if I were just lecturing, I would never realize that there was this piece of information that nobody got until the exam comes back and at that point it’s sometimes too late.

John: That’s one of the advantages of a just-in-time teaching approach. It allows you to focus your class time on the things that students are struggling with, and to skip over the things that they already understand. So, it lets you use your class time much more efficiently.

Rebecca: At the end of one of those class periods or even during that class period, I jot down what those things are, so that if it’s a while between each semester when you teach it again you don’t forget what those are because sometimes you can lose track. So, coming up with a system to routinely to check in on those things can be really helpful.

Trish: Yeah, a journaling effort or something. Yeah.

John: And I saw you also do something called Trashcano?

Trish: [LAUGHTER] Yes, Trashcano, Trashcano is an activity that we do late in the term once the weather gets nice. In the class we talked about different styles of eruptions. And one of the styles that we get to later in the class is explosive eruptions andTrashcano is a demonstration that was developed by my colleagues at Colgate: Karen Harpp, Danny Geist, and Alison Koleszar. And they basically developed this experiment where you take a trash can and you fill it up about two-thirds to three-quarters of the way filled with water. And if you submerge a two-liter bottle with liquid nitrogen in it, that bottle represents a pressurized magma chamber and it ends up rupturing because liquid nitrogen is boiling at ambient temperature. And so the two-liter bottle ruptures in that water and creates a column that sprays into the air. So, for this activity, the students do some calculations of plume heights so they can use their iPhones to measure the angle of the trajectory of the water and they can say, “Okay, the plume went this high” and they can do some back calculations to discuss what sort of pressurization caused that amount of uplift to the water. And then we also put styrofoam balls of different sizes and shapes into the trash can. And they can make isopach maps… basically how we actually map explosive eruptions where we take the different grain sizes, and we create a map of how far the different grain sizes spread from the center of the eruption. It’s a fun day outside. This past year, we did it in the rain, which was rather interesting to see how rainfall dampens the amount of distance the styrofoam can spread. I’m not sure that we’d want to do that in the rain again, but it was an interesting experiment. Yeah, we do a lot of little things like that so that the students can take their concepts, the actual equations that we’re working on in class and apply them in a tactile, physical way.

Rebecca: Trish, do you use consistent teams throughout the semester or do you rotate how your groups are formed?

Trish: I’ve done it a couple different ways. I’ve now had the opportunity to teach my flipped classes two to three times each at this point. And some terms I do let them switch around and some terms I keep it consistent. And I’ve found that overall, it works a little better when they’re consistent teams the whole way. I feel like the students build a lot of teamwork and camaraderie with their groups. But, I don’t know… I try to take it by a term-by-term basis because I have had situations where the students are eager to switch around and meet other members of the classroom. We do this a little bit with our jigsaw discussions. So, for example, we do a role playing exercise where each group is a volcano monitoring agency. So, in your monitoring agency, you have a volcanologist, you have a seismologist, you have a geodesist you have a communication specialist, but then all the communication specialists from each group will have to get together and work as a team for one of the activities and all the geodesists will have to get together and work as a team for the activities and then bring them back to their initial group. So, they do get some chances to interact with one another through these, I don’t know, is it a jigsaw puzzle within a jigsaw puzzle? [LAUGHTER] I’m not sure how you describe it, but they do get these opportunities to move around to other groups. But, that’s something that I still am thinking a lot about. I think you had a guest on recently. Dakin Burdick. In his he talked about how sometimes he likes to let the students all which groups all the time because then they get to know everybody in the class, and then sometimes it keeps them all together. It seems like a lot of people do different things with this. I don’t have a great method yet. But, I do tend to go on a sort of term-by-term basis and get a feel for the culture of the class and how people are melding. I do find sometimes when you do the consistent groups, it can happen that the group tends to congeal really well. And then it lifts up all of the students in the group. And so people attend class more regularly, and they’re much more engaged. But, I have seen it happen where groups have sort of fallen apart because one or two members just aren’t attending regularly, and they’re really not committed or engaged. And that becomes difficult. And then you really kind of need to reshuffle a little bit.

John: We talked about that in episode in early October with Kristin Croyle when she was talking about team-based learning where there are persistent teams. And one of the things she suggested is it’s really important to form teams that are constructed to be balanced so that you don’t run into that. But, there are some advantages of having persistent teams. But, if it’s a persistent, dysfunctional team with people missing, then that could be problematic. I think a lot depends on the nature of the activities. If you’re going to have persistent activities like in team-based learning, having well defined teams may be useful, but for other types of activities that vary class to class it may not matter as much.

Rebecca: Yeah, I think there are certainly advantages to both. I’ve had experiences where we’re doing long-term projects. So, doing some preliminary shorter activities with those groups that they’re going to have for their long-term projects can be really helpful. And getting those teams gelled before it really matters. And I’ve also had experience doing persistent teams when I’ve gamified a classroom. And that actually works really well in getting people to hold each other accountable and be competitive. So, I’ve had really good luck when I’ve done that as well. You’ve also received a lot of grants for your research.

Trish: Yeah, I’m in a fortunate position that my primary position is research. So, I don’t actually teach that much. I only teach two classes a year. So, I do try to find ways to integrate all of the exciting research that my group is doing into what we do in the classroom setting, but not just my classes, the other classes in the department too. We teach a 100-level course in oceanography and a lot of my research centers around seagoing expeditions and collecting geophysical data at sea to understand submarine volcanoes. So, we try to bring that experience back into the classroom for our introductory level students. Especially in a landlocked state of Illinois, many of our students have never seen the ocean. They’ve never been to the beach, and they don’t really have a concept for why would scientists be at sea collecting data? And what are scientists doing in our marine setting? So, bringing that into our introductory classes, I think, is really critical. So, the big push there was… I was chief scientist of an expedition we’ve just wrapped up. We had two seagoing missions to the eastern Pacific, it was called the Oasis expedition. And we’re investigating a line of seamounts on the sea floor. So, these are volcanoes that have been active over the past million years, and we were using this submarine to collect data, to collect rock samples from the sea floor, and it started because I have a young daughter and I was going to be gone for about 45 days, and I wanted her to feel connected to me while I was away. We don’t really have great internet at sea, as you can imagine. So, it’s hard to continue to feel connected with loved ones at home. So, I decided with the help of my husband, to create a YouTube channel that would chronicle our life at sea and link back to my daughter’s classroom and some local schools that they could watch what scientists are doing. And then we also ended up using those videos in the introductory courses on campus at a higher level so that students could see a sort of a hands on of what we do when we’re at sea. But, yeah, it started out predominantly as me wanting to stay connected to my then six year old while I was sailing, and became a really great way to provide outreach to a broader learning ecosystem. So, lots of people throughout the community,

Rebecca: I think it might seem more obvious that students in a landlocked state don’t have experience with marine life. But, at the same time, I think that our students don’t have much experience with many professional experiences in what it’s like to be in any kind of industry or research setting. So, I think that that same methodology works in a lot of circumstances to give students exposure to what it might actually be like to be a professional in the field.

Trish: Yeah, and I really like that sort of informal blogging aspect. So, these videos were [LAUGHTER] very informal. I had a blogging camera and I basically just filmed myself doing things. And I remember at one point, my husband sent me this email that basically said, “Wow, you look really tired. Are you doing okay?” Because I just was like, “Alright, it’s 4:30 in the morning, we’re getting ready to do some scientific stuff. I’ll put my camera on me and film what we’re doing.” But, yeah, it’s an exhausting process because it’s a 24/7 operation when you’re out at sea collecting data. You have this facility. For 30, 40 days, whatever it is, and you want to use every second of it to get as much information as possible. But, I think that’s important because a lot of people don’t know like, “Oh, that’s what an ocean scientist….” well, what my particular volcanology centered ocean scientist “…does for research.” And then the other arm of my research program is very much in volcano hazard. So, that feeds directly into the volcano geophysics courses I teach because my group works on developing forecasting mechanisms and algorithms for taking volcano monitoring data and providing monitoring agencies with information about how volcanoes are evolving and we have a lot of monitoring agency partners that we’re working with to try to provide some new quantitative methods for assessing volcanic unrest. So, these are things that we’re thinking about every day, but we certainly can infuse them into classes on volcanology and volcano geophysics,

John: Having that video channel would also let you do some time shifting… where much of the work that you’re doing takes place during breaks when classes wouldn’t be in session. And it still allows you to bring this into your own classes as well. I’ve watched several of your videos, and they’re really good. We’ll share a link to those in the show notes.

Trish: Oh, great… Thanks…. [LAUGHTER] They haven’t been updated for years. But, yeah, I think the asynchronous aspect is really cool. One of the things that we struggled with when we were first doing these expeditions was we were trying to schedule, within reason, because it’s really hard to schedule your ship time because you’re working with all the other scientists that are utilizing the facility, but we’re trying to schedule them such that students could participate synchronously with what we were doing. So, while we’re out at sea we’re sending back Q&As and doing videos. But, what we found was that you could still use all of this information after the fact so students have been benefiting from these videos for the last three, four years, which is really fantastic. And I think that it’s something that a lot of fields scientists could take advantage of. For example, the Antarctic field season is when everyone’s off for holiday. But, perhaps if they’re doing these videos, that they could bring them back and create learning modules for students to see more of what is it like to be a scientist working in Antarctica during the Antarctic summer… and not in a documentary way, I felt like one of the things I really wanted to do was provide that informal feeling for students so that they could look at that and say, “Wow, I actually feel like I could do that. And I could see myself in that role.” Whereas when you have that documentary, shininess, it’s harder to imagine that it’s not this esoteric thing that you could never aspire to be as I wanted to show like, “Yes, we’re up at 4:30 in the morning and we’re tired.” Yeah. I like that.

Rebecca: I think you’re right, that that polish sometimes makes it seem really not approachable to students, or that they don’t belong in the field, or they don’t belong in the discipline. But, if you’re showing that realness in that authentic moment through your own lens, it’s really beneficial to students… and I can imagine this working in just about any context, actually, to help students understand the day in the life that they might be pursuing.

Trish: Yeah, absolutely. It would be so helpful too for K through 12 students, because a lot of times they have no idea. Geology is an interesting discipline in that we’re kind of a found discipline. Students usually come to college thinking, “Oh, I’m going to go do chemistry or I’m going to do physics or engineering…” and geology is not really on students radar, but then they start to see how they can apply chemistry, physics, engineering, and math, all in one discipline, and they sort of gravitate towards us. But, we don’t get a lot of freshmen into our major but maybe if K through 12 students saw what geologists do on a daily basis and what a career looks like, they might say, “Oh, yeah, that’s a major I could be interested in.”

Rebecca: There’s a lot of disciplines and careers that students have no idea exists. The lens in which they see the world is largely through whatever classes they’ve been taking. So, it’s like, world is math, English, science, these really broad categories of things.

John: And they see it from the textbook perspective, as a well defined body of knowledge that they just have to learn or memorize, and not as an active, ongoing endeavor. And those videos you created, and these types of connections that you’re making for students help open up that possibility to them. As part of the OASIS project, you used a variety of social media including Twitter, Reddit, and I believe you did a Reddit Ask Me Anything. Could you tell us a little bit about your use of social media for this project?

Trish: One thing that we learned very quickly is that the internet on the ship was not great. So, the day we did the Reddit Ask Me Anything, it was a day that the Alvin submarine was on the sea floor collecting samples so we knew the ship could stay in one spot. So, it’s sort of like having an aerial antenna on your old TV and you’re trying to like bend it in just the right way so you have a good connection. So, we were able to set the ship in one location and then rotate it [LAUGHTER] so that the satellite was in the right spot so we could get on Reddit, and then we had to like shut down everything using the internet and we all crowded around one computer [LAUGHTER] and did the Ask Me Anything. And I think it was a really good experience. One of the things that cracks me up is that there was a scientist on a sister ship in the northern Pacific that responded to one of our questions and said, “Hey, we’re up here on the RV Armstrong. Hello.” [LAUGHTER] Yeah, I think that they’re Reddit Ask Me Anything was a great experience. It was mostly done by the graduate students on the ship. I was sort of running around doing other things. But, it was a great way, again, to provide outreach. It gives you a demographic that otherwise you may not have interacted with, which I think is important.

Rebecca: Sounds like another sneaky method, like not telling students that your volcano class has math in it. [LAUGHTER]

Trish: Yes. Exactly…[LAUGHTER] Exactly.

John: We always end the podcast by asking, what are you doing next?

Trish: One of the things that we’re working on right now, as I mentioned, we’ve collected a lot of video at sea and on the last expedition, we collected a lot of virtual reality 3D video with GoPro fusion. So, we use GoPro fusions to collect really nice 3D videos on the ship. So, things like how the scientists were cleaning tube worms that were collected from hydrothermal vents and how they’re processing rocks and just the day-to-day life on the ship. So, we collected all this virtual reality video and now we’re working with colleagues and the CITL, the Center for Innovative Teaching and Learning on campus to develop VR modules for introductory classes. And this is been a really crazy learning experience for me because one of our goals is to give these experiences to students that would not necessarily ever have a chance to go to sea or to do this type of work. But, building and structuring the learning goals for these VR experiences is really difficult and I didn’t realize how big of a leap it would be from just the video content and lectures to creating a VR structured activity for students. And there’s some really cool things they’re doing here on campus. The medical school’s been doing a lot with VR techniques for med students and different procedures in VR. And there’s an Archaeology Professor on campus who’s been using it to like simulate an archaeological dig using VR. So, we’re working with some really amazing educators. And hopefully, that will come out with some fascinating modules for our students and upcoming offerings of our oceanography class. But, that’s sort of the big thing that we’re doing right now. I’m kind of excited to see how that will turn out.

John: We talked a little bit by email about you and your husband coming back on later to talk about some of that work. So, for our listeners, we will be revisiting this sometime in the next couple months, I think.

Trish: Yeah, hopefully we have a paper in review right now, where we look at asynchronous linkages to field expeditions and ways that you can collect videos and content while you’re in the field, sort of non-disciplinary-specific, of course we’re looking at marine sciences. But, again, you could use it in other fields and how you can bring that back to produce learning modules for your classrooms.

Rebecca: …sounds really exciting.

John: We’re looking forward to hearing more about that as well.

Trish: Thanks.

Rebecca: Well, thank you so much for joining us. It’s been really interesting.

Trish: Thank you so much for having me. It’s been a pleasure to talk to you guys. And I really appreciated your podcast, it’s been so inspirational to me and my work. I really appreciate it. ‘

John: It’s been a lot of fun for us too. We get to talk to people in depth. Normally, when we gave workshops, we’d hear little bits and snippets of what people on campus were doing, but being able to explore things like this is so much more valuable for us.

Rebecca: And we get to learn about all kinds of different disciplines too which is really exciting. to

Trish: I think what’s so cool is exactly what you said in your hundredth podcast that a lot of times faculty can’t go to those workshops. So, giving them a way to listen to the podcast while they’re commuting or traveling is just awesome. It’s very, very cool. Before going to sea, I always load up my phone and computer with all the podcasts I can get my hands on, because once you’re there, you’re there. That’s pretty much it. [LAUGHTER] Just download the entire catalog.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

102. Team-Based Learning

Posted in active learning, classroom community, classroom management, collaboration, collaborative learning, evidence-based practices, higher education, learning, lecturing, pedagogy, peer instruction, project based learning, team based learning
October 9, 2019

A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, Dr. Kristin Croyle joins us to discuss how she transitioned from  explore using interactive lecture to collaborative learning, and then to team-based learning. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego.

Show Notes

  • Johnson, R. T., & Johnson, D. W. (2008). Active learning: Cooperation in the classroom. The annual report of educational psychology in Japan, 47, 29-30.
  • A discussion by Dan Ariely explaining why asking for shorter lists of positive features in a relationship can engender positive feelings appears in this March 24, 2014 video clip.
  • Michaelsen, L. K., Knight, A. B., & Fink, L. D. (2004). Team-based learning: A transformative use of small groups in college teaching.
  • Team-Based Learning Cooperative
  • Epstein Educational Enterprises, What is the IF-AT?
  • Sweet, M., & Michaelsen, L. K. (2012). Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement. Stylus Publishing, LLC.
  • Croyle, K. L., & Alfaro, E. (2012). Applying team-based learning with Mexican American students in the social science classroom. Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement, 203-220.
  • Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251-19257.
  • 74. Uncoverage – David Voelker – Tea for Teaching podcast episode discussing the uncoverage movement in history, March 27, 2019

Transcript

John: A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, we explore one faculty member’s transition from using interactive lecture to collaborative learning, and then to team-based learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Kristin Croyle. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego. Welcome, Kristin.

Kristin: Thank you.

John: Our teas today are:

Kristin: Earl Grey

Rebecca: I am having Mama’s work tea, because Ada made it this morning and she calls it work tea, which means she pulls the tea bag tag out and puts it in the big cup. Also, it’s just my normal English Afternoon. But, that was a better story.

John: And I’m drinking Spring Cherry green tea.

Rebecca: We invited you here today to talk about collaborative and team-based learning in your classes. But before you do that, you’ve noted that you had a strong preparation in teaching before you got started. Can you talk a little bit about that. We’ve talked a little bit about that on the show before and how a lot of faculty aren’t prepared…

Kristin: Um hmm.

Rebecca: So, could you talk about how your preparation may be informed what you’ve done.

Kristin: My graduate program, I went to the doctoral program in clinical psychology at the University of Montana in beautiful Missoula. And that program takes the preparation of their grad students very seriously, but across several areas not just in clinical work and research, but also knowing that some of them are going to end up in positions in which there will be teaching. So, while I was there, that very first semester I was brought in, they had a structure for teaching their introductory psychology classes where graduate students were assigned our own classes where we were the instructor in the classroom, but we had a supportive network around us. So, the syllabus was already there, the textbook was already there. We collaborated in writing tests. We had a structure of TAs that supported us and they would have recitation sections in which the TAs also received development. And we joined in that so we could see how more hands-on kind of things could be done with students in smaller groups. We even assigned our final grades together. And some of those pieces are pieces that are areas of skill that people don’t often think about developing. So, that first semester, all I had to do was think about working within the structure: How am I going to handle the day-to-day teaching and learning in the classroom? I didn’t have to worry about course design because the course was already designed in front of me. And I also didn’t have to, at the end, think: “When you assign grades, is that rigid? Do you really have to follow the exact, you know, 90/80 that it is in the syllabus? Or what if there are natural breaks around 88 or 89? Is it okay to flex that? What kind of power does an instructor have that is fair to students and evaluation?” I got to do all of that in a collaborative setting with a very experienced faculty member as a guide. There was also a credit-bearing course for teaching psychology that we were encouraged to take… which I really enjoyed. And then I was given opportunities to function more independently. When they needed a stats teacher over the summer, and they knew I was living there over the summer, I got to teach on an adjunct basis, but still with the support of faculty around me. So kind of putting students in the deep end, but with a high level of support around them, I felt very prepared when I was done with the graduate program to enter into an assistant professor position. And I still appreciate the preparation that they gave me.

Rebecca: I think with the preparation like that you’re probably far more willing to experiment and do new things as a faculty member too and to maybe even break away from what faculty around you are doing. Do you find that to be the case? Or were there other faculty doing some of this collaborative work in the department that you were in?

Kristin: Yes, and no. One of the experiences I had at my previous institution, which was the University of Texas – Pan American that then transitioned through a merger to be the University of Texas Rio Grande Valley. I was talking with a colleague in another department about the kinds of things we were doing in the classroom. And I still remember him saying “Oh, I didn’t know you could do that in the classroom and that was like teaching.” He had a very restricted idea of what teaching was, and what would be acceptable to colleagues, which he had never had the opportunity to test with other people around him. And that was something that I arrived from day one… that you talked about your teaching… that you can do many different things in the classroom. And it’s all teaching, as long as you are trying to work with students to create a learning environment and they are learning, then it counts as teaching. So I did come in with a much more flexible idea, then certainly some of my colleagues who hadn’t had an opportunity to ever have those discussions. And of course, some people are hired into departments in which those discussions don’t ever happen, so they may persist with those misconceptions for many years.

John: Or throughout their entire careers at times. [LAUGHTER] The scaffolding that was provided is really nice, because we’ve talked to a few people who’ve been in teaching training program or had some training in graduate programs. But usually, it’s not quite as structured as that and that’s a nice feature.

Rebecca: Yeah, I came from a program like that, but it was like very front loaded. It wasn’t that ongoing…. So I felt a lot more prepared, because I did have a lot of those experiences, but I didn’t have that same kind of supportive network throughout. Which is incredibly valuable.

Rebecca: So, you want to take us through what some of your collaborative experiences have been in the classroom and the ways that you set up some of the team-based learning exercises, maybe starting with what are those?

Kristin: Sure. So kind of the way that I journey through my teaching, particularly when I was an assistant professor, I felt comfortable in the classroom, but I didn’t feel expert. I felt like I was still trying to figure out what was going on, which is a perfectly fine way to be and a good state for learning to occur. So I felt like I was a talented lecturer, like I can engage students. I teach in psychology, I also think psychology is naturally very engaging, but part of that is because I really love the field. So, I felt like I could engage students and that they would listen and that they would be interested. But I started to become dissatisfied that there was always a core of engaged students and I had no idea what was happening with the other students in the class. And then sometimes I would be disappointed when we have tests or homework. Everyone said they had no questions. Clearly that was wrong. I was wondering how do you engage the majority or all of the students in their learning so that they aren’t coasting through class believing that they understand until they really don’t. And then I also felt like I was kind of fooling myself into thinking that students were with me when they were not with me. So I had an opportunity at that time to do some intensive cooperative learning training along the model of Johnson and Johnson collaborative learning. And that model from the University of Minnesota, it focuses on the importance of cooperation in the classroom, and that in cooperative settings, students learn more, develop a stronger sense of self efficacy around their learning; that they together are able to achieve more than they would individually. And it also has impacts on retention… that if students are feeling like they are individually known and valued in the classroom by their peers, they’re more likely to continue showing up to class and to develop relationships outside of the classroom that supports them along the way. So through that training, it was intensive, it was like eight hours a week, one day set for several weeks. The very first day, I could see what a difference I was going to make in my classroom. So, for example, I was using group assignments in class and they had all the same disadvantages that group assignments and most classes have, because I had no idea how to structure the group work so that it would be successful. I was doing group work to save me grading time, honestly.

Rebecca: That’s why a lot of people go to group work.

Kristin: Yes! Without understanding that all I had to do was some structural changes, and then it would actually be effective for learning as well, instead of just saving me grading time. In that cooperative learning training, I learned how to structure intensive group work that could be the length of an entire semester, or it could be the length of a single class day. I learned how to structure less intensive moments of team time. So how do you do a think-pair-share that works versus how do you do a think-pair-share that doesn’t work very well. So, that within the course of that training… actually just within a few days… I suddenly had, instead of 10% of the students in the class engaged on a daily basis, I had 100% of the students engaged on a daily basis. So, that was a huge breakthrough and I continued that way for several years.

John: What were some of the structural changes that you made that did lead to increased engagement?

Kristin: So, the cooperative learning approach of Johnson and Johnson, is kind of theoretically heavy, in the sense that they outline the pieces that are necessary for strong collaboration to occur. And then they turn it over to you as the instructor to say, “How do I build those pieces in?” So, for example, they emphasize positive interdependence as one of the essential components of cooperative learning… that when you create a group and a group activity for them to do, the activity has to be structured in such a way that each person is necessary to contribute. You can’t structure it in such a way that you can have three people talking when one person is only needed, and there are specific recommendations on how do you structure it so that everyone is needed. At the same time you have to build an individual accountability as another required component, so that, even if each person is needed, people can still slack off, say, “Yes, you all can’t do as well without me because you need me, but I don’t really care about what is happening here.” There has to be a level of individual accountability that’s also built in. Along with that, some of the skills that I thought were most important, they build an emphasis on group processing and social skills, so that if you have people consistently working together in class, they may not have developed the social skills to do that effectively, especially over time. You can work with someone for two minutes on a think-pair-share and really be bad at social skills. But, if you have to work with them over an extended period of time on a project and things are going south in terms of group conflict, it’s the instructor’s responsibility to help them to develop the social skills to work together. For example, on the first day of class, when I first start having students talk to each other so that they know that’s going to be a pattern in the class, I give them something quick to talk about. And I say introduce yourself to the person next to you… spend two minutes talking about this. And then I’m going to ask you about what you talked about. And then I run around the class real quick… pair up people who aren’t participating, introduce them to each other so that they understand this is a part of the class. So, then I follow back. So, what pieces are important there? …that I explicitly instruct them, you turn with your body… you actually make eye contact. And I will point out as people first start doing this, look at these two people, they are looking at each other, because many times students won’t do that, and it’s very hard to have a cooperative interaction if you don’t make eye contact… and I will say, “Who was the person you talked to? Tell me their name.” So they understand that I was serious when I said, introduce yourself, tell me something about them and that there’s individual accountability through just random calling on… that they need to participate in the cooperative portion. And then there’s also the self-reinforcing aspect of it that five minutes later, when I say to talk about something else, they realize they already know somebody in class, they have a connection. The next day, when I come in, they’re not quiet, they’re already talking to each other, they’ve created those connections.

John: A nice thing about that, too, is for people who are uncomfortable talking about themselves in class, having one person tell you something about the other person, it’s a little bit less pressure, it’s a little less revelation to the whole group. There’s some evidence that that type of thing is more effective in providing a more comfortable environment.

Rebecca: Kristin, can you also talk a little bit about a specific example of a cooperative activity where all of the members are held accountable, and all have a role? …just to provide an example for people who have less experience.

Kristin: So cooperative learning can be divided into informal and formal cooperative learning. Informal cooperative learning tends to be much shorter activities that can be done kind of on the fly if you already have an idea in your mind of how you might want to do that. Formal cooperative learning tends to be more intensively structured… longer-term activities. So that could be a single class session. If you’re going to do an activity that takes an hour, that would be more formal… or if you’re going to do something that takes an entire semester. The pair-and-share that I just talked about is an example of informal cooperative learning. Something like a jigsaw classroom activity can be structured as a formal cooperative learning activity. And it already shares almost all of the components: there’s individual accountability, because each student is given a specific role. There’s also positive interdependence, because the success of everyone depends on each person doing their role. So there are ones that are already structured with a built in component. The pieces that aren’t built into something like a jigsaw classroom activity, would be the group skills and group processing, and the ways that you can build that in. You can, for example, ask groups to reflect on what went well. I typically emphasize that more than asking them to reflect on things that went poorly, because asking to reflect on what went well tends to maintain a positive atmosphere, but also helps them to cover both bases at the same time anyway.

Rebecca: …or realize that my list for what went well is not very long… [LAUGHTER]

Kristin: Right. So, a common group processing thing I would have students do after their first more lengthy or more formal cooperative learning activity would be: list three things that your team did well together and one thing that you could improve on. And another thing I might ask them to do is to provide positive feedback to each member of the group at the end of the activity. And the kind of feedback that they provide is usually pretty specific, and helps to shape their behavior throughout the rest of the semester. So when they say things like, “I like it when you disagreed, and you said that this other thing would be a better way to go” that provides important feedback, and it helps to encourage better processing going forward. But I will go around and give individual social skill feedback too. But it’s usually things like, “Oh, I see you’re sitting so far away from your group, I’m not sure they can hear you, let’s scoot your chair in so that they can hear you.” Or I might ask, “Oh, do you know this person’s name next to you? What’s her name?” …and we’ll make sure that people maintain the social and cooperative connections that enable to do that kind of good group.

John: Just as an aside, it’s useful if you’re asking about things that went well, to keep the list fairly short. I’m reminded of a study that Dan Ariely talked about where they did a controlled experiment where in one case, they asked people to reflect on three things they liked about their partner and another case to list 10 things I liked about the partner, and then they surveyed them on the quality of the relationship. And those who were asked about three things generally rated their relationship with a partner fairly high. But when they were asked to come up with 10 things, they struggled with that and they rated that relationship lower. So keeping the list short…

Kristin: Right.

John: …is really good so you don’t…

Kristin: There’s kind of analogous thought about keeping things like gratitude lists. If you list too much stuff, it can have a negative effect, because you start to identify things that you really don’t think are that important, and it makes you think the whole thing is less important.

John: And if you want to get the opposite effect, ask people to list 10 things that were bad, and then they’ll struggle beyond the first few. You talked about having continuous relationships or persistent relationships with collaborative learning. Did you try to keep the group relationships consistent for the same groups throughout the term? Or did you vary that?

Kristin: I varied it. There are some good data to suggest that in collaborative learning… they refer to them as base teams… that base teams have a persistent positive effect, particularly on things like student engagement and retention throughout the semester and throughout the year…. that you have a team that is expecting you every day. But when I was doing cooperative learning, I didn’t restructure my courses. I restructured the day. Does that make sense?

Rebecca: Um hmm.

Kristin: So I didn’t have a reason for base groups. And I felt strange imposing them on the students without a reason. Besides, they would maybe be socially a good idea. I had to completely rebuild my courses from the ground up before I started using base teams. And that’s when I transitioned to team-based learning.

John: …and in team-based learning, persistent teams are recommended as part of the process.

Kristin: Absolutely.

John: Could you tell us a little bit about this transition to team-based learning. What prompted you to introduce that? …and how it worked?

Kristin: So I was happy with how courses were going. People were interested and engaged. I had students telling me, “I know every single person in this classroom.” and when you’re teaching a class of 30, or 40, or 50, that’s unusual. “I know everyone in here, I feel really supported.” I feel like things were going well. But I was unsatisfied with what I was teaching. I wasn’t clear, in my own mind, about what persistent learning outcomes I wanted for my students. I had not sat down and really thought through if I were to follow up with a student in a year or five years, what would I want them to recall from this class? What would I want them to be putting into use in their lives or in their careers? I had never thought that through. And I was fortunate enough to run into team-based learning at that time, right as I was primed to start thinking about this questions. Team-based learning originated by Larry Michaelsen. He was coming from the perspective of enrollment increases. He had been assigning some pretty challenging work. He was a faculty member in business. And as his course enrollments increased, he started to wonder how can you maintain the same kind of interesting, really challenging in class… by case work, for example… with a large enrollment. So he developed team-based learning to address that piece, but it also requires you to completely rethink the design of the course. And to start from the course outcomes: “What do you want the persistent outcomes to be?” …and then structure the course forward in that way. So in team-based learning, after you make a decision about your course outcomes, and what you really want students to be able to do, then you structure the course in a modular fashion. And each module has certain steps. So the beginning is student preparation, then when they come into class, you test. You say, “okay” …and it’s called the readiness assurance process. So you want to know what students are ready to do after they’ve individually prepared, and what they’re not ready to do. So they prepare, they test. And then, since it’s a team focus, they also test as a team. After that you have a good idea as an instructor, what are they ready to do? What are still the fuzzy areas? What do they really not get at all? What are their competencies as a team already, even if every individual student doesn’t have it, and then you can do some corrective lecturing, basically, so many lectures that fill in some of the gaps. And that’s all part of the readiness process, because you’re getting them ready to do some interesting application work in class. And the rationale for that is… and actually what I had been doing prior to that, was giving interesting application material to work on at home individually, while doing lecture and cooperative learning in class. But the interesting application material was actually the heart of the course, and the much more challenging piece. So it was better to bring the hard piece where they needed support into the classroom. And the piece they were ready to do, which was to do their own self study back into their own lives. So you do this readiness assurance process to make sure they are ready for interesting application, and then the majority at the time for the module you spent on application. Doing that after I had already worked with cooperative learning was really helpful, because all of that application work is done in a team setting. So when you already have some experience with how to build teams, how to maintain and develop their social skills, that’s really, really, really helpful. That’s a short version.

John: One of my colleagues, Bill Goffe, who was on one of our very early podcasts, noted that when he gave the group test, the performance always went up significantly, so that they could see the benefits of the peer discussion that was part of that. And he was really impressed with it. And he noted that, oftentimes, if a student didn’t show up for class one day, they get a hard time from their classmates from the group because they let the group down. And he said his attendance had never been better than when he was using a team-based learning approach.

Kristin: Absolutely. And a lot of people who do team-based learning, use the same methodology for doing the team testing, which is honestly really cute. It’s a scratch-off form. And the scratch-off form is used so that the team gets immediate feedback on each option. So on any particular item in a multiple choice test, if they want to select “B” they scratch off “B.” If it’s not there, then they continue to scratch until they get the right answer. For one thing, they love it. But also they are getting immediate team feedback. If this person is not speaking up, if they say I think it’s “B” and then they stop advocating and then it turns out to be “B” later than the team immediately knows, by the time they get to the next question. “Okay, we need to incorporate more feedback from all of our team members, wait a minute, this person who’s not speaking up actually has a lot to say.” In the course of just a few multiple choice questions, it brings their team development forward leaps and bounds. And they kind of have fun with a scratch off, which is also a bonus.

John: And it also gives them incentives to come prepared and to listen to other people in ways that they might not otherwise.

Kristin: Yeah, and their team will give them grief, if they say “Oh, I don’t know, because I didn’t read,” their team members will be like, “But we are depending on you, you need to read, we all read.”

John: And it also gives them a little bit, perhaps, of improvement in metacognition because they’re getting that immediate feedback, and it’s being coupled with the reactions of the peers. So if someone was insistent on a wrong answer, and they dominated that discussion, they might be a little more careful in the future and more willing to listen to the other people and reflect.

Kristin: Exactly, and it doesn’t have to wait till next week, it can happen right away. Right on the next question. The team application activities are also structured in a particular way. In team-based learning, they talk about the four S’s for the application activity, the first one is that you have to select a significant problem. So what they’re working on is something that will be important to them, something that they will identify with, or that they recognize is worth their time in thinking about and trying to think through. The second one is that they need to be working on this same problem. You can’t say teams one and two are working on this, three and four are working on this, five and six are working on this. Third one is that they structure in so that they make a specific choice as the outcome. Because it’s easier to solicit team feedback if everyone is making a specific choice rather than having kind of an open-ended narrative response. And it helps to stimulate whole group discussion as you’re moving. Now it can sound like it’s limiting to say that you have to make a specific choice, but you can do in a very broad way. And the fourth one is simultaneous reporting. So all the teams are asked to report at the same time on what the choice was that they made, so that they can’t piggyback off another team who’s putting in effort. So, as an example, one of the courses that I taught in the psychology major in Texas was the tests and measurements course in psychology, and test and measurements starts with a stats review. They’ve all had statistics, it usually comes prior to tests and measurement. But it’s the first time that they have an opportunity to work with statistics in kind of a decision-making way. So you start with a stats review. So one of the activities that I would do, I gave them two hypothetical first-grade teachers with how many questions 10 of there students got right on a spelling test. And the two distributions had the same mean, but one was fairly normal, and one was highly skewed. So they had to do their quick statistics review… Do the mean, median, mode and standard deviation describe the shape of the distribution. But the question I was asking them was, “If you were the principal, which teacher would you offer an after-school tutoring program to for extra pay? And which teacher would you potentially nominate for a teaching award?” They found that question to be a really interesting question. For one thing, students think a lot about what good teaching is, and what constitutes a good teacher. So they already come in with very strong opinions. And they also understand the complexity of, you know, if everybody’s passing but people aren’t excelling, is that good teaching? Whereas if most people are failing, but a few people are getting an “A” is that good teaching? …and how the data contributes to good decision making, but can also be kind of manipulated to contribute to decision making in not such a good way. So instead of just saying, “Let’s review the stats, here they are,” it was a question with a specific choice that they simultaneously reported on. And then we could discuss together. And of course, their answers are different. There’s different rationales in both ways. So then we could discuss together what their rationale was, if they want to debate they can debate a little. It generates a lot of student enthusiasm, and everybody’s doing it instead of just 10% of the class.

John: And once they’ve committed to an answer, they have a stake in and they really want to know, that’s something we’ve seen a lot of things we’ve talked about in the past, too.

Kristin: Absolutely.

Rebecca: When you were doing the team-based learning, were you sticking specifically to problems that were on a class-by-class basis still, like you were discussing in the co-operative setting, or were you doing some longer term activities that went across multiple class periods?

Kristin: I had the… what I consider gift… to often be teaching in a three-hour time slot, which is my very favorite time slot. So I would have activities that would extend two or three hours, but typically not between classes, I found that to be more of a sweet spot, at least for me. At my previous institution we had a very high commuter population. And I promised, in both models, that I would never ask them to do something out of class with their teams, that was one of my rules… that it was just simply too burdensome for students who have multiple outside of school commitments… family and work, or living potentially 150 miles apart, which was not unheard of. I promised them no out of class stuff. I structured that intentionally so that the individual preparation that they were doing, they could do anywhere on their own time schedule, but they were expected to be there. And their team expected them to be there to be able to engage in class. And it was also one of the ways that you talk people into it, when they say “I worked with other groups who were all slackers and we would always set times and they wouldn’t show up.” And I said “That’s not going to happen in here. We already have a time we’re all going to show up together.”

John: And the philosophy that’s very similar to the flipped classroom approach where you let students do the easy stuff outside and then give them assistance with or have them work in a framework where they’re getting more assistance with the more challenging issues.

Kristin: Absolutely. I think TBL [team-based learning] is definitely a flipped classroom approach.

Rebecca: I think the other thing that helps too with that model… of making sure you’re not working outside of class… really helps students with really different backgrounds start working together, because you might have students who are more traditional who are on campus. And so for them to meet outside of class is often not such a big deal. But then if you have students who are working or have families, and there’s a disconnect in the class, even, between those two populations, that helps make that more obvious and work a little bit better,

Kristin: Right. Absolutely. Yeah. And I didn’t want to set up anything where people were made to feel like unvalued team members, because they couldn’t do what was asked of them because of other commitments. Since that was in my control, I wanted to make sure that people felt welcome.

Rebecca: I’ve tried to even do that with long-term projects. In the field that I’m in, we tend to do things that go across class periods, but there’s always the “Are we going to do this outside of class or are we going to do this inside of class, and I try to have them do anything that needs to be collaborative, and decision making, in class, and then things that can be done on their own, even if that means doing some creative work, or whatever, outside of class. But those are independent things that can be done for the same reasons. And I find that students will try to manipulate that system, so that they’re gonna: “Oh, we’ll just do it outside of class, because we don’t want her to know whether or not we’re on top of something,” or whatever. But I call them out on it, because it’s really devaluing some of that exact thing. People have other commitments and things.

John: You mentioned, you started to use a backwards design approach where you started with the things you want them to remember five years later. Did you have to cut back on the breadth of the coverage in the class, to some extent, by doing that?

Kristin: Yes, I did. When I was going with the straight up cooperative learning approach, I did not have to cut back on the content at all. Without the full redesign, I found I could cover the same amount of material in straight lecture versus in a cooperative setting. But it was all coverage. It was just a different kind of coverage. When I approached it from a backward design perspective, and I really was able to focus on the objectives that I thought were important, I did have to reduce the amount of things that we were covering. I have no regrets about that, of course, because I completely recognize that covering material isn’t just covering it. What are students going to do with something I covered in class? They didn’t cover it, I was the one who was learning it and talking about it. So I’m much happier with an approach in which I am consistently hitting on the objectives that I really want them to recall, and that they are working hard to apply those throughout the semester.

John: If they’re not going to remember it passed the final exam, covering more material isn’t terribly useful.

Kristin: No.

John: We talked about that in a previous podcast with David Voelker, who talking about the coverage approach in History…

Kristin: Right.

John: …which is the same logic.

Kristin: Exactly. And I actually now consider that to be a complete waste of time. So why am I spending class time on something that I actually don’t really care if they remember, it’s not the most important thing to me, and they really don’t care if they remember.

Rebecca: You have some compelling arguments for why team-based learning and collaborative learning are good options. If one wanted to start moving in that direction, what would you suggest their first steps be?

Kristin: For team-based learning, there are a couple of great books that are very easy to approach. There are several great resources for team-based learning. Larry Michaelsen published a book in 2008, for example, that covers that from front to back. It gives examples of applications in different disciplines. There’s also a book published a few years later on team-based learning in the social sciences and humanities. That also covers the basics, but has applications that are more specific to social sciences and humanities. Team-based learning has really caught on in medical education and in business education. So in the original book, there are more application examples that are in MD preparation or in business schools. So if you’re looking for other examples, the second book might be a good choice as well. And that one is edited by Michael Sweet and Larry Michaelsen.

John: And in fact, I read your article, or

Kristin: Oh, did you?

John: …your chapter in there as background.

Kristin: I’m glad someone read it.

John: Now I have to read all the others. But, I, at least, did read that. It was very good. So for faculty who are moving to this, what are some pitfalls that they might run into? Or what sort of problems might they encounter?

Kristin: Team-based learning as being a much more structured approach… Michaelsen does a really nice job of laying out the pieces that he thinks are critical. And I agree they are critical. So, for example, he talks about explaining, and testing the model with students on the first class day, and you cannot skip it. So the very first class day, I give students an example individual application test, like they would get for their readiness assurance. It includes basic psychology knowledge that may or may not be present in the culture. So they have some chance of getting some of them right and some not. And then I have them do it as a team. And the team scores, of course, are always dramatically higher than the individual scores. And the team testing process is so much better. [LAIGHTER] It’s more pleasant and interesting and collaborative than they expect it to be. That simply going through that, it allays many of their fears about what a team is going to be like to work with. Plus, when they see that the team has tripled their individual score, they’re like, “Hey, maybe I could depend on other people to help me learn, and maybe this will pay off for me.” So going through an explanation of what the rationale is, having them experience it a little is really, really critical in helping them stay open minded while they experience it. And then regularly throughout the semester, I will keep reinforcing them with those messages. I’ll say, look at this amazing thing you guys did. You used all the intellectual resources around you, and you analyzed this difficult problem and came up with some great solutions. I’ll remind them how much they’re learning and what kinds of challenging tasks they’re able to do as a team when they have the preparation to do it, which helps as they’re starting to think “Well, wouldn’t it just be easier if I could do this by myself?” It helps them to kind of remember, ”Well, yes, but you wouldn’t be doing this, you would be doing something not as challenging, not as integrative.”

John: and probably not learning quite as much either&hellp;

Kristin: Yes. He also emphasizes an aspect that is also emphasized in cooperative learning… of helping the teams develop and giving them feedback, helping them give each other feedback. That’s also really critical, especially very early in the semester, as they’re starting to develop group norms and bond together to make sure that you don’t short the time in class for them to have some group processing time and to build their team skills. So, for example, when I taught last spring, I had a student who came to me after I think it was the second week. So it’s very early in the semester, and she said, “I really need to reassigned teams. My team hates me, they won’t make eye contact with me.” She was really upset. And I’m reluctant to reassign people teams, because often what they’re experiencing, they take with them. It’s not always a function of that team process. So we talked some, and I tried to get a handle on what she was experiencing. I knew where she sat, I had an idea of the team composition. And I asked her to try one more day, just one more day. And then we would talk about reassigning her teams. And that day, I was sure to build in plenty of time for group processing, where they talked about what they were doing well as a team and something to improve. Their team turned around immediately. She was a relatively assertive person, which I already knew. I knew that she could handle this. So she went back to the team. She was able to talk with her team about not feeling heard. They immediately turned around in the way that they were with her. And by the very next class day, they were a relatively high functioning team. They did well all semester. They brought doughnuts for each other. I mean, it was a really nice supportive group. What they needed was the time in class to do some processing. And if I, as the instructor, had been moving too fast, and not giving them time to do that, and not giving them a prompt to do that, it would have been a really negative experience for her. So, also building in time for the team to develop and prompts for them to do that.

Rebecca: So you mentioned liking to have a three-hour teaching slot.

Kristin: That’s my favorite. It’s not required.

Rebecca: So, in that amount of time, how much time would you designate towards this group processing, for example, to give people an idea of what that proportion or the amount of time to dedicate so that you don’t shortcut it and you don’t rush through it?

Kristin: If I were to do an activity that might take an hour, I might spend 10 minutes for group process, it doesn’t have to be very long, or even five. And you don’t have to do it every time, you could do 10 minutes after the first one or two more intensive activities, and then not do it for another few times… and another five minutes just every so often to help them resolve their underlying dissatisfactions and to recognize that what they’re doing is not just application activity, it’s also group interaction. So please take time to do both. Another really important required component that I didn’t mention is peer evaluation, I always incorporate peer evaluation as part of the grade.

John: How did you form the teams in these classes?

Kristin: They’re heterogeneous, first, with a very open process so students can see it happening and know there are no shenanigans… that this is all very open… talking about the rationale that people of different backgrounds bring different strengths. So you want a group that has people of different backgrounds, so you can have a larger kind of learning base between you. So usually, I’ll pick a few characteristics that might be important in that kind of background. And I will line them up around the room based on those characteristics. And if it’s 200 people, it’s a really long line. And then we count off. So when I teach introductory psychology, students who have had a high school psychology class usually are starting a big leg up on the other students. So I’ll include that as a characteristic. Sometimes I’ll include the distance that people are coming from, because then they have different experiences, depending on what class I might also include if their student athletes, just because if you put too many together in a team, then they’re all gone on the same day. They have interesting backgrounds, but they also have patterns of attendance and of absence that need to be adjusted around. And we’ll count off all the way around so people can see how the teams are made. But heterogeneous teams are really, really critical. Having students with pre-existing relationships will throw off the team process in a way that automatically excludes people that don’t have pre-existing relationships… plus they tend to be lower performing teams. And I don’t want to set that up on purpose.

John: One of my colleagues once did this in a class of, I think it was about 350 students, but he just sorted them alphabetically. So he had them organize himself that way, and it was a fairly long process. But, it was kind of amusing for those of us wandering by and just seeing…

Kristin: …this huge line… Yeah.

John: He didn’t do it that way In the future, he used other criteria.

Kristin: I’ve had colleagues that I’ve talked with that think that this is a long process. It’s not. You can sort 200 people in 10 minutes, and then you’re done for the whole semester,

John: Doing it alphabetically…

Kristin: takes a lot longer.

John: …can be more challenging, because they were self forming that… it didn’t convert rapidly.

Kristin: The other thing I never do is I don’t put the students who didn’t come the first day into a team, because there are characteristics about why they didn’t come the first day. If you put them all together in one team, they share some of those characteristics… It tends not to be a very high performing team. So I make sure they’re sorted out among the other teams. But that was one of the things that I learned in cooperative learning. That, before I did cooperative learning training, and I was assigning group work, I would assign people based on if you didn’t come the day we did the assignments, you were in another group. And that group typically did not do very well. And as an instructor, it’s my responsibility to create a learning environment in which students can excel, it’s on them whether they do their part. But if I’m setting up a team in ignorance, with predictable characteristics, so that they’re going to have a failure experience, that’s on me to correct. And it’s not on them. So afterwards, I felt guilty when I had come to a new realization. But, yeah, it’s my responsibility to set up an environment in which those students can be successful in their teams.

John: In your chapter in that book, you mentioned that when you switched over, it did affect your course evaluations a little bit. Can you tell us a little bit about that?

Kristin: Just a little bit. But yes, it did. So when I was doing straight lecture, I was shooting for engaged lecture. And in psychology, you can build in little experiences, especially in introductory psychology, where the topics are changing frequently, you can always build things in that are kind of interesting. You can do a little optical illusion here and a little bit of memory trick there. And there’s these ways to build it in, but it is still basically straight lecture. And I got high evaluations for that. I was careful about trying to build those in every day, you know, every few minutes. And when I went to cooperative learning, where it was essentially the same approach, but in in a much more engaged and cooperative fashion, those evaluations stayed very high. Students knew each other, they were happy in class. When I went to team-based learning and I was actually asking every student to participate all the time, and be prepared in class in a way that their contributions were much more obvious than mine. My evaluations did drop just a little bit, not a lot, but a little. And I am grateful that I was teaching in a context where I knew that my department wouldn’t care. They were more interested that I was doing good teaching. And they understood the many factors that influence student evaluations. But I also recognize that it’s incumbent on me to help students understand how they are learning, what kinds of things encourage learning and retention, and then you kind of let the student evaluations fall where they may.

John: When I read that, it reminded me of that study that came out a few weeks ago from Harvard in their physics program, where they found that students in active learning classes did demonstrably better on tests, but they perceived their learning as being lower. So there was a pretty strong inverse relationship between their perception of learning and actual learning. That seems to be fairly common, there have been a number of other studies where what students think to be most effective, is often not what most enhances their learning.

Kristin: Right.

John: Do you have any other advice for our listeners, who might think about using either collaborative or team-based learning in their classes,

Kristin: The one thing I would say is that teaching a cooperative learning or a team-based learning structure class is a lot more fun. You have to be willing to give up control, because when you’re lecturing, you have absolute control… meaning even that students can’t ask you weird things, because you haven’t opened the door for that to happen. But when you structure the learning experience, and then you give up the control to the students, it is an exciting environment to be in. I wasn’t as tired when I was coming out of class. I was energized, you could feel the difference in the room just walking into class… they were excited and talking with each other. When I would circulate around before class started, they’re talking about the class instead of talking about other stuff. It completely changes the environment in the classroom in a way that I think really matches what I expect out of a university education for students, it creates a environment of intellectual enthusiasm around the topic that you’re teaching.

Rebecca: So we always wrap up by asking what’s next?

Kristin: That’s a great question. So right now I’m 100% administrative. And since I’m in a new position, in a new institution, I’m gonna spend some time figuring out all the newness pieces. But I’d like to go back to the classroom, at least for a course here and there when I can. There’s nothing different about students than there is about people. So I also think often about how what we do in the classroom, what we understand works and what we understand doesn’t work, how that applies in administrative settings as well. We know for example, that people tend to try and find the shortest path. So if they’re trying to learn something, they want to put in the least effort to learn it. If you ask a faculty member to do a task for the department, they are obviously going to choose the easiest path to do that… not necessarily the best path. So how do I take the experiences of learning and teaching, that in some ways are better understood to an environment of administration that in some ways is not as well understood? What kinds of lessons can I apply there as well?

Rebecca: Well, thank you so much. It’s been a really interesting conversation. I’m sure it gives a lot of people things to think about as they move forward in this semester and future semesters.

Kristin: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

94. Open Reflection

Posted in active learning, authentic learning, collaboration, commons, higher education, learning, OER, open pedagogy, pedagogy, peer instruction, project based learning, reflective practice, teaching, writing
August 14, 2019

Students can provide useful feedback on instructional practices and class design when they are asked, In this episode, three students from John’s spring economics capstone class join us to provide their reflections on the class’s experiment in developing an open pedagogy project. Our guests in today’s episode are Maria Aldrich, Victoria Heist and Charlie Tararzona.

Show Notes

Transcript

Rebecca: Students can provide useful feedback on instructional practices and class design when they are asked. In this episode, students join us to provide an open reflection on one class’s experiment in developing an open pedagogy project.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Maria Aldrich, Victoria Heist and Charlie Tarazona, three students who participated in the creation of an open pedagogy project in one of my economics classes this spring semester. Welcome.

Victoria: Thanks for having us.

Maria: Thank you.

Charlie: Yep, excited to be here.

Rebecca: Today’s teas are…

John: None of you are drinking tea, are you?

Maria: No.

Victoria: No tea.

Charlie: No tea today.

Rebecca: How regretful. [LAUGHTER]

John: It happens with many of our guests. I’m having ginger peach black tea.

Rebecca: And I’m drinking Lady Grey. The issue is our tea selection is no longer close to our recording studio. It’s a problem. It’s an epidemic now with our tea choices.

John: …now that we’re recording in this little closet in a building next door, where at least we don’t have toilets flushing every 30 seconds or so that we have to edit out.

Rebecca: So John, can you start first by explaining what open pedagogy is, to kind of frame our discussion?

John: Going back a step further. Last year, I saw a presentation by Robin DeRosa who presented on this at the SUNY Conference on Instruction and Technology and she made a really compelling case for some of the advantages that open pedagogy projects have. And open pedagogy just involves having students create content that is open and shared publicly with the world.

Rebecca: So what class did you do an open pedagogy project in?

John: This class is a capstone course in the economics department here. It’s taken mostly by seniors and a few juniors. And it’s a seminar course in economic theory and policy. It’s one of our smaller classes. And we had only 27 students in this capstone, this semester.

Rebecca: So why this class?

John: Because the students were ready for it. The course builds on all the courses that they’ve had up to this point and it allows them to pull together material they’ve learned in all of their prior courses, as well as the cognate classes in statistics and math and so on.

Rebecca: So what kind of project exactly did you propose to these students?

John: I originally proposed two options. One was to do something on behavioral economics, because past classes have found that to be a lot of fun, and another one I suggested was they could just pick current topics and work in small groups and create papers on that. Turned out that they really didn’t like any of those ideas and given the nature of open pedagogy, I left it up to the class to decide what their topics would be. And I think it was actually Charlie, who came up with the idea. And would you like to tell us what that was?

Charlie: I know you had mentioned in the beginning of the class the idea of open pedagogy. And I found that pretty interesting because it seemed like a good opportunity for us as seniors and juniors to really put what we had learned out there. And also, in terms of topic selection, you gave us the opportunity to really choose which topics we wanted to talk about. We ended up choosing the topic of intergenerational mobility and economic inequality. We focused more on the economic inequality aspect of it in the end. But yeah, like I said, it was just a good opportunity for everybody to really finish their college careers with something that they can show.

Rebecca: Dr. Kane is going to close his ears now and you guys are going to tell us exactly what you thought when he said, “Hey, you’re going to write a book.”

Victoria: I was hesitant at first, just because group projects are kind of daunting, especially in economics. However, a collaborative group project was exciting to do… to see all of our work put together. As economic students it isn’t something you really see, it’s usually individual work.

Maria: Oh, yeah, I would agree with Victoria. I was kind of hesitant at first, especially because it was something new for our class so I figured there are probably a lot of kinks that needed to be fixed. So I was a little worried about not having everything fully figured out at first. I thought it was something interesting. It appealed to me because I like the thought that other people could read what we had written and we could have control of what we would want to talk about.

Charlie: And the topic and the idea of a book project really intrigued me… that it just let us put out there what we had learned over these past few years and gave us something that we can show in the end of it.

Rebecca: Were any of you scared?

Victoria: Not scared. I wouldn’t use that word.

John: Were you concerned?

Victoria: A bit concerned, just because I like doing my individual work. I feel stronger in that.

Maria: Yeah, I would agree. I think group work can sometimes be difficult to have for every class because everyone has a different writing style and everyone works on their projects at different times. So I think at first, you’re a little bit worried that not everyone will be able to work well together. But I found that in my group, we were able to work very well and we’re able to meet once a week to go over what we needed to work on for the week.

Charlie: Yeah, I found something similar to that experience. Whereas my group, after the first few weeks, figured out what we wanted to do, and when we could meet, and what was the most effective way for us to put the book project together? And I think it turned out really well.

Rebecca: So you’ve all mentioned groups, can you talk a little bit about what the groups were, how they were determined, and how that worked?

Victoria: The groups were groups of three from the class, because there’s 27 people. And then we’re able to email Professor Kane and ask if we wanted to work with anyone specific, like if we had friends in the class, we could work with them. But if not, or if we didn’t want to work with someone we knew, we’re able to randomize it.

Maria: I was put into a group of two other random people that I didn’t know, but we were able to set up a group chat immediately and communicate very well through that.

Charlie: I actually emailed Professor Kane about being a group in Victoria and we also included another student in that. I think it worked out pretty well and I was happy with how it turned out.

John: Before the groups were formed, though, the class decided on what the topics would be. So we had kind of a free-form planning session where we narrowed it down to nine topics you wanted to address. And then at that point, we knew how big the groups were going to be. And it worked out nicely with three people per group.

Rebecca: How did each group get assigned a topic?

Charlie: So the way we assigned topics was, we had created a list of the nine topics, and then each individual group could choose their top three, and then we divided them that way based on everybody’s top choice. If they didn’t happen to get their top choice, they usually got their second or third, I think that only happened for maybe two groups, and they seem to be fine with what they ended up with.

John: And going back a little bit further, it was a weighted voting scheme that you didn’t just rank them… that you assigned points, if I remember was it 10 points I gave you? And so if you really wanted to chapter you could bid all 10 points on that. And if you were indifferent, you could have assigned weight to your top three preferences and so forth. And it did work out really nicely where I think most groups got their top choice, but two of them ended up with their second or third choice, but it seemed to work.

Rebecca: How did you find collaborating in the end?

Charlie: I found that it worked really well meeting every week. We also had presentations every week that we gave on specific topics that we’re talking about during that week. So that set the initial schedule for us to meet every week and talk about what we were doing and what was going on. Also with the book project at the time, in terms of organization, I found it very laid out and simple.

Rebecca: That sounds like you had a writing group that met that frequently, but it also would be more of like a study group as well?

Charlie: Yeah, I would definitely say it was a mix between a writing group… a study group. Your group members ended up being the way, if you wanted to succeed in the class, like that was the way to do it was to work cohesively with your group members.

John: And it should be noted that they had other tasks in the class as well, where they selected topics that were presented each week and each group was responsible for presenting an article or a research paper on a topic, some of which were related to the book and others were completely different. The groups were persistent across all the assignments and involved more than just writing the book.

Rebecca: How’d you get feedback to make sure whatever you’re putting out in the public was good enough?

Maria: Well, we mainly used Google Docs. At first, we tried to use hypothesis. But that wasn’t really working out well. So we ended up just going back to Google Docs. And each group would be given a couple of chapters to review each week. And they would write a couple comments in that chapter as well as some comments made by our professor and we used that and we also used each other’s feedback to make those edits.

John: How did that work? Where the comments helpful?

Victoria: No. [LAUGHTER] Just because I’m very protective over my work, which I know I should be open to criticisms. However, I got some comments sometimes I was just like questioning, like instead of “what about this article that you might want to look at” it would be “change this word,” where I know we emphasized that often in class many, many times, but still people would persistently do that.

John: I hope that wasn’t from me. Was it from me?

Victoria: Oh, no. Well, if you wanted to do that, you’re the professor. You can do that. But you also give us feedback that’s helpful. Whereas, students I feel like if they’re rushed or doing it, like 20 minutes before the class, they’re not going to look at me like, “Oh, what about this topic that might be interesting to consider.” Instead, they’re like, “Switch this word.” That just might be the students in the class too, just because we did have a lot of work in the class. And I found that because our group would meet to practice our presentation before the class, a lot of groups are doing their final work 15 minutes before the class.

John: Yes, the quality of the work did vary a little bit across the groups and across the individuals within the groups. Overall, there was some really high quality work, and all three of you did really well. But the quality of the feedback varied quite a bit.

Rebecca: So the feedback was generally done outside of class? Like not during class time?

John: Primarily, except for the presentations on the work where there was some feedback during the presentations.

Victoria: Yes, but I found that your feedback was most helpful, rather than the students.

Maria: Yeah, I would say I paid a lot more attention to Professor Kane’s feedback than some of my fellow students. Luckily, we didn’t have that experience. We had a lot of people give a sincere, really constructive feedback, but sometimes I’d be hesitant to take that feedback because I didn’t know that was the direction that I should be going in. But I definitely think our experience was a little better and our comments were more substantial, I would say.

Victoria: And I think next time maybe switching the groups that review the feedback might be helpful, because if you have one group that gives worse feedback, and you keep getting that, it’s not as helpful.

John: The way it was structured was each group reviewed and provided comments on three other groups, and we did that on three stages. And the class decided to maintain persistent groups there. I did give them the option, but I think it does make much more sense to vary it so you’re getting a wider range of feedback.

Charlie: I think the idea to keep persistent groups stem from the fact that we wanted to have somebody read the paper and then continue to read the paper throughout the weeks when we were supposed to be improving it or making it better. So then they could also see the changes we were making. And I agree with my classmates where I think we can say that it didn’t work out too well. There’s some groups just didn’t happen to give feedback that was too good.

Victoria: I also think part of it was the length of the papers because each of us had to review three full papers for the weeks that we did that, and three 20-page papers is a lot of reading to do on student written economics. And I think maybe in the beginning it was helpful to read all three, but maybe as time went on to scale that back a little bit, so we don’t get burnt out.

John: More detailed feedback on a smaller number of papers.

Victoria: Yeah because at first, I find myself doing it too. Like the first paper, I’ll take the time to read every single word and provide helpful feedback. But I can see myself not doing as much on the third.

John: I gave feedback in three different ways. The first time I gave video feedback, and while I’ve heard that that can be really efficient, I was taking about two hours or so per paper. And that was really slow and people really didn’t like the feedback that much because some of the feedback was fairly long in terms of the suggestions. So, I probably gave a little too much feedback. The second was with comments embedded in Adobe. And the third time I just basically went along with everyone else and provided the feedback directly in Google Docs. And the nice thing about that is I was able to see some other suggestions and sometimes I’d say, “Well, maybe that’s not such a good idea.” Because in many cases, the original draft actually made more sense than the feedback.

Rebecca: What way did you all like feedback better?

Charlie: I think the best feedback I received was actually in class feedback when I would go to Professor Kane and ask him, “Hey, you know, this is what’s going on with my paper. Is there something else I could look at? Is there another source I can find?” I found that to be the most effective in helping me write the paper. I was also a fan of the comments in Google Docs, they were pretty helpful.

Maria: Yeah, I think the most helpful feedback were the comments from Google Docs because, for that last draft, I was able to go through and resolve any comments that I had made the changes to and that just helped motivate me to make my draft a really good copy. And then I would say that I was really against the video feedback because I personally like to review feedback multiple times. I like to go through it and make changes to it. And I found myself just typing up his comments at the end of the doc so that other teammates could see it as well. So I was definitely against the video feedback and prefer the Google Doc comments.

Victoria: Yeah, I prefer Google Docs as well, just because I could see where exactly you wanted the changes done. It gave more specific feedback and then it also gave the students validity I guess, like this should be changed. Like I made a comment in one paper and said, “I think you mean a different word.” And they just resolved it and moved on. But then Professor Kane came through and said, “Yes, I agree.” And I think that you agreed, and they now are aware that yes, those changes need to be made.

Rebecca: I think sometimes when a faculty member responds to student comments in a way that it also helps students know how to make better comments. So it would be interesting to see how another round of that would have gone after Dr. Kane had responded to some of those right? To see if the comments were better the next time around.

John: Yeah, I think I should have done that from the beginning. And I’m sorry, I didn’t. But in the future, I’ll probably use Hypothesis. Now that we have Hypothesis in Blackboard it will be much easier. Among the problems we had is that people had some trouble making comments on Google Docs because they also had edit access to those and they couldn’t mark up specific text. And with PDFs, that was a bit of a problem given the way the browsers were set up that they had to change a program in order to make comments on PDF documents. So now that we have that in our learning management system, it’s going to be much easier to do that and the comments will be a little more persistent, because one of the issues was people were, as you mentioned, resolving comments sometimes before anyone else had a chance to see them. And the strategy was to have the draft documents with the comments copied over to another folder, and they were only supposed to make changes in their working document, not in the documents used for comments. But there were three or four people who through three drafts, just didn’t quite get that notion and I’d see the email saying that comments were resolved, and I would go back in and unresolve them. But in any case, there were some problems with those. That’s an issue that I think has to be worked out a little bit more efficiently.

Rebecca: Beside some of the technical issues that we mentioned, what were some of the biggest challenges of working on a project like this?

Charlie: I think one of the bigger challenges was keeping the cohesive idea behind the whole book where the topic we had chose was income inequality and we also had talked about intergenerational mobility. But as the book progressed, we kind of saw that portion of the book fall off a little bit where chapters were really focusing on the income and economic inequality topic.

Rebecca: So is that something you discussed in class to keep everybody on track?

Charlie: I think we mentioned it at one point towards the end, we’re just like, “Okay, are we going to keep this? Are we going to not keep this?” And I think we agreed, we could talk about it but we won’t make it a major portion of the book.

John: There was also some scaffolding on the project… that it didn’t just start with people starting to write, groups were first asked to put together a bibliography, and then an annotated bibliography, and then an outline of the chapter, and then the actual writing started after they had feedback on each of those steps.

Maria: I would agree with Charlie, I was definitely worried about the cohesiveness of the entire book. But for my group, specifically, we did a very broad topic, the global trends of economic inequality, and for myself, it was really hard to find relevant subjects to talk about because it was just such a broad topic. It was really hard for each of us to find something that we could spend a large amount of time writing about. So I’m not sure how the other groups felt. But for us, it was definitely hard narrowing down what we specifically wanted to talk about, and then to find resources that were recent enough to include.

Victoria: Yeah, I agree with you on that. I think one change I would make after we figure out the specific topics, you can go deeper in that because it’s hard as a group to form a thesis statement or very cohesive argument because we ended up doing more of a timeline than like an argumentative paper because you have to split it up.

John: Your topic specifically was on what?

Victoria: Tax-structure and income inequality. So basically, we looked at early 20th century, later 20th century, and the 21st century, and how the changing tax structures led to increasing income inequality over time. So that’s kind of how we split it up. But I think if I was to do it again, I would take a different approach to it, because I did the first section and finding information on World War One income inequality is much harder than it seems. So I struggled a lot with that too.

Charlie: Yeah, in terms of how we wrote our chapter of the book, I’m usually a fan of writing papers that follow a timeline as an explanation but that’s just a personal preference. It doesn’t work for everybody. So I can definitely see how making the cohesive argument along with following that timeline can be pretty difficult.

John: In your chapter, I think the timeline made a bit of sense. We were talking about the evolution of it and the transitions in your chapter were pretty smooth. I don’t think that worked as well in all the chapters, quite often it looked like they were three essays…

Victoria: Yes.

John: …chopped and pasted together.

Victoria: There was this one paper with a bunch of sub topics, but it wasn’t cohesive. And I was reading it and it just did not make any sense to me how it was organized. So that was one of the suggestions I made… maybe taking a step further in class and presenting maybe our papers a little earlier.

John: In more stages…

VICTORIA. I was just trying to read it and I just could not make sense of the organization of it, where maybe if we caught that earlier we maybe could have made better paper.

John: I was giving them feedback in several groups… that sort of feedback… that they need to smooth out the transitions and have a more logical structure. But some groups responded really well and did a nice job with that, other groups were a little more reluctant to do that.

Rebecca: Perhaps some groups will respond really well to some peer pressure. [LAUGHTER]

John: And having the presentations in class would have helped do that. When people in the class were saying, “This is just too disorganized.” And most of them got better by the end, but it was a stretch getting there.

Rebecca: So you’ve talked a little bit about some of the challenges, but what was really rewarding about working on this project?

Victoria: I found it really helpful to work with the group. I had Charlie and then another student, Junweii, in my group and we all read each other’s parts. I know I went through the document and made comments for my own group too. And we were all able to bring it together, make comments for each other, ask each other questions about like what sources to use especially too. And it was easier in that regard than an individual paper. Because if you make a mistake and you don’t realize it, no one’s there to help you, it’s just you. But here we have people to help each other.

Charlie: Yeah, I always find it beneficial to complete a task with other students also trying to complete the same task as you. It just makes the learning more interesting. And you’re more willing to go and spend the extra hour looking at the document to just make sure you understand what you’re writing, but also that it fits with whomever else you’re working with. I found it really beneficial or satisfying just the fact that we, like I said, could create something that any ordinary person could probably read and understand what was happening.

Maria: Yeah, I think the most rewarding part for me was just seeing that finished product and getting you know, positive feedback from Professor Kane and from my other group members. I think working in that group setting helps to motivate me to do the best of my ability. And I think it was just rewarding at the end to see everything come together well.

Victoria: I think it gave us all a deeper understanding of the material too because, instead of writing it yourself… because you can write something and not understand it. I’ve done it many times. [LAUGHTER] But when you’re sitting in a group, getting a presentation ready, you each need to understand the material. So you’re explaining what you learn to each other. And that’s something you don’t get by yourself. I found that really rewarding.

John: What about the public nature of the project? The fact that this will be out there, it will have your names on it, and it could be out there indefinitely.

Charlie: I found that portion of the project pretty intriguing and exciting. Just like I said, you can go out there, and obviously we’re college students, we’re looking for employment after this. So just showing an employer, “Hey, I’ve written something that’s been published. It’s out there, you can go read it for yourself and see what you think.” It gives something for the students to show.

Victoria: Yeah, it made it exciting because we knew what was at the end of the project. Rather than just a finished paper, we actually had something to like prove ourselves, like we did this.

Maria: I think for me, it was cool to know because at the end of the semester, I’m able to go to my family and say, “Oh, here you go. This is something that I worked on all semester long. Here’s something that you can read and you can better understand what I’ve studied for the past four years.” So I think it was helpful that I was able to show my family I’ve worked hard on this. This is something that is to show for that.

Charlie: I would definitely concur with that. Economics as a topic isn’t really discussed when you’re talking just with family members, so many of them don’t understand what you’re talking about. And you’ll try, but it’s hard sometimes. So to put something together that they’d be able to read and understand, I found that pretty satisfying.

John: And how did the class select the audience for this? What level was it written for?

Victoria: Students with a background in economics I think we decided on. But we came together as a class and decided on that. But you need economic background to understand some of the things we wrote.

John: But at an introductory level, so it wasn’t written at an advanced level. It was written for people who’ve had an economics course somewhere along the way.

Victoria: Or just no background. You don’t have to go to college to read the book.

Rebecca: How would you change this project in the future? We touched on a couple of things here and there, but do you have any other key things that, if the same exact project were presented to another group of students, how would you change the structure? Or the way it’s organized? Or the way that it’s presented the first day?

Victoria: Thinking about the class as a whole rather than just the book project, we did weekly presentations which was a lot of work in itself. So I would probably minimize those and focus on the book. Because we were sitting there reading 20+ page economic journals every week and making a presentation on it and doing the book project. So I think having more time dedicated to the book project and presenting on that material, rather than just economic journals that people have written, like it gives background, which is helpful, but maybe a little less, or maybe shorter ones, or ones that are just easier to understand. Because I know a lot of times you would say, like, “I know you guys don’t understand this, it’s challenging. But we still need to know it.” Like you would explain it in class, which would be helpful, but reading something you don’t understand is really difficult for students… in economics specifically. That’s challenging.

Maria: Yeah, I would agree. I think, at the beginning of the semester, it was a lot of work to have to juggle both the presentations and the book at the same time. So I kind of like the idea, I’m not sure if it was you Victoria, who mentioned it in class, of doing the presentation one week and then the next week working on the book and having class time devoted to the book in the week after. I think that would have been very helpful too because we did meet as groups, but if we were able to meet in a class setting than I think other classmates will be able to make comments on your chapter and offer advice. I think it would just help overall with the workload that we have.

Charlie: I also agree with that. I think the improvement can be made where we’d work on maybe a random topic every other week, and do a presentation on that, and then also incorporate the book project into that. I think it would help with the cohesiveness of the book along with just feedback and all the other problems that we had discussed.

John: One of the things I had suggested at the very beginning, you may recall, is I suggested one option is to spend the whole class focused on this. Another option is just to do it the way it was done in the past, or something else. And the class actually voted for the something else. Now having had the experience, the something else didn’t work quite as well, and that more class time should have been devoted, I think, to this and I saw that too.

Victoria: I think we’re just looking for something exciting. Like yeah, it’s a book project like we know what we’re going to do with that. But the presentations just added something else, but if I went back to a book project because then we could have taken the steps at a slower pace too, like the annotated bibliography, like the topics, we could have taken way more time with that than we did. Because once we did that very quickly, and then went into presentations, and then we just had due dates instead of meetings in class.

Maria: Yeah, I think for us, what appealed to us with this combination of the book and the presentations was that the presentations offered structure for us when we knew what we were getting with those presentations. We knew each week that we’d come in with the presentation. And I think with the book, we were excited because it was something new and different and I think we were a little too hesitant to go fully and choose the book, because we weren’t sure what we would be doing in class. We weren’t sure how we would be tested on that. So I think the combination of fields lost because we were able to have that structure, but we were also able to try something new.

Charlie: I know for some of the students in the class they had mentioned to me… they were hesitant to get rid of the presentations weekly because they were a fan of learning something new every week and learning a different topic, not just focusing on the book project. They really wanted to increase their knowledge base by just learning about multiple fields of economics. So I think that’s why we ended up going with what we went with in the end. But I think we all could all agree that if we had done that every other week, it would have been more efficient.

John: I agree. And I think some combination might be good for the reason you mentioned, but more class time devoted to it would be helpful.

Victoria: Maybe at first too, do a presentation. Like the first presentation, I don’t know what week that was, but maybe keep that one because when our group really met each other, we worked together, and then we planned a time every week where we would meet.

John: And if this is done again, and that will be if the class wants to do this in the future, perhaps that first topic for the readings could be related to whatever they choose to do so they’re actually doing some scaffolding with the presentations then.

Rebecca: I had something similar in my classes before where a team formed early on. We did something small, low stakes, to figure out how to work with each other and what doesn’t go well. So that when we did something a little more high stakes, you already knew what the wrinkles were going to be so that you could plan for that moving on. So it sounds like your presentations served that purpose, whether or not you intended that to happen or not.

John: But it became a lot of work when it was done every week, in addition to writing a book.

Victoria: That was difficult.

Maria: Yeah, I think it just helped to make us all more comfortable with each other and more comfortable speaking in front of the class.

Rebecca: So the big question is, of course, should other faculty do this?

Victoria: Yes, I’m working on my honors thesis right now, which is kind of what you would do in a traditional seminar. And it’s very difficult. So just having people there… write it with you… know what you’re talking about… You can ask them questions. In our group chat, we often ask, “What would you recommend for this part of the paper? Or what articles do you think are appropriate for this?” If you’re doing it by yourself, it’s very difficult. And the overarching topic… I feel like in a lot of seminars, they have that. It’s a topic for the seminar, but it doesn’t really filter through as well as the book project does, because we are all cohesive, all of us together working as a class of 27 people, which you never see. So, I found it really helpful and I liked it a lot. And it wasn’t like a crazy amount of work. You did the work, and you study, you did the presentations, and you wrote a paper, but it didn’t take you hours every day to work on. I feel like I learned more in this class than I have in other classes that I write individual papers for.

Maria: Well, I think I would partially agree and partially disagree with that. I think as a class, we all appreciated that Professor Kane was willing to change like the class structure and was willing to try something new. And I think that was definitely intriguing for us and provided something different as our last economic course. But I think if I had done my own topic paper, I think I probably would have learned a little bit more, I think just I would preferred that. But I think it was still important to get this experience and try something new.

Charlie: I think I would definitely suggest it to some other faculty members to maybe try this out. Like Victoria was saying, working with a group is pretty beneficial. And I feel like, from a personal standpoint, I learn more when I’m working with other people who I can ask questions to, get feedback from. Really, it helps your understanding of the class. In terms of incentive, I find that I wanted to work on the book project because you had that end goal of, “This is something that I can put out there and show to somebody.”

Victoria: Yeah, but at the same time, group work can sometimes be the worst thing that ever happens to you. Like we got really, really lucky because I know Charlie, we’re friends so we were like, “Okay, let’s work together. We’ll just get one random person.” Junwei was like such a blessing. We just work together so beautifully, but I feel like if we had someone that didn’t want to do the work… wasn’t willing to put in the work… didn’t show up to meetings… that would ruin the project for us. So I don’t know how you could fix that. But just if there’s a good group, it works. If there isn’t, I feel like it wouldn’t work as well.

Rebecca: So good to write one book during your time here, but maybe not many books. [LAUGHTER]

John: But there could be other things. For example, they could have been podcasts that were created. They could be collections of essays.They could be video projects that are put together by groups. So there’s a lot of different things that could be done.

VICTORIAL: Yeah, I would throw that out there. If you did this again with another book, like, yeah, you can write a book, but you can also do that… a different kind of form of the same kind of structure. That would be interesting.

Maria: I would be interested in doing some type of podcast because I know some of my friends in their classes have been required to do podcasts. And I feel like you have to prepare really well for that. So I think maybe that would have forced your teammates, if they weren’t doing the work, to do the work so that they wouldn’t get to the studio and not have anything to say. So I think that would have been another really cool option.

Charlie: I think it would be a good option for capstone classes, just because I know for a lot of majors, you hear what the capstone is about for the three years before you even get there. And I know personally for me, I’m also trying to get a political science degree, my capstone is next semester, like I’m already dreading the 25-page paper I’m gonna have to write. So to switch it up and have the students maybe not know exactly what they’re in for, I think it gives a little bit of an intrigue and like, “Okay, this isn’t just the I’m going to go and write a paper all year. It’s something else that I’m going to do.”

Victoria: Yeah, it’s more fun. I’m more willing to write a paper that my group members are in. Like we can all see each other too in the Google doc and talk to each other in the chat… be like, “What do you think about this part?” Or like Charlie can watch me while I’m writing my part of the paper and say, “This is good. Maybe change this. Or bring this sentence up.” You don’t do that in individual papers and even if you write an individual paper and have peer feedback, it’s not the same as having it right there, real time, or just people caring more because it’s theirs too.

John: We did have some issues with that early on though, in the first draft or two, because there were some people who really didn’t want to try using Google docs for writing. And were any of you involved in that?

Charlie: So, I’m not opposed to Google Docs. [LAUGHTER] I had just always used Word documents before. So it took a little bit of getting used to but once you commit to it, it’s a really nice thing to have in your repertoire. Google Docs, I feel like, is used by countless numbers of people, companies, places, businesses, the college. So honestly, as a student, you should just take the incentive to try to get to learn it. And once you learn it, it’s really beneficial to you.

John: One of the problems was that some people were writing in Word and then uploading it to the drive and that made it really hard for other people to edit. And eventually everyone switched over, but it did take a few iterations with some people.

Maria: So yeah, I think there were a couple of challenges with having different drafts because people made comments on separate drafts. So I think just sending out a reminder email would be helpful and letting people know because I know I think I made my changes on the wrong draft the first time and we had to send an email right away to have him fix that. So I think just having it set up all before the due dates like before you mention it in class would be really helpful too.

John: Yeah, there were some rough spots. This was new for me too.

Rebecca: So we always wrap up our podcast by asking what’s next for each of you?

Charlie: This December, I’m looking to graduate from Oswego, which is exciting for me. And after that, I’m not really sure what’s going to go on. We’ll see.

Maria: Well, I’m graduating this Saturday, and I’m going to be moving down to Florida for a little bit and doing an internship there.

Victoria: I’m also graduating Saturday and I’ll be working at HSBC this July in their graduate development program.

Rebecca: Sounds like exciting futures for each of you.

John: What are you doing in Florida? An internship where?

Maria: I’m doing the college program, the Disney College Program.

John: Oh, wonderful. Maybe I’ll see you there at the OLC conference. Well, thank you. It’s been a pleasure working with you all semester. And thank you for joining us.

Charlie: Thank you for having us.

Victoria: Thank you.

Maria: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

89. Teaching About Race

Posted in active learning, classroom community, diversity, equity, evidence-based practices, growth mindset, higher education, inclusion, instructional communication, learning, pedagogy, race, social justice, stereotype threat, teaching
July 10, 2019

Class discussions of race and racism can be difficult for all participants. In this episode, Dr. Cyndi Kernahan joins us to discuss ways of building a classroom climate in which these issues may be productively explored.

Cyndi is a psychology professor and Assistant Dean for Teaching and Learning at the University of Wisconsin – River Falls. She’s the author of Teaching about Race and Racism in the College Class: Notes from a White Professor, which will be available from West Virginia University Press in Fall 2019. The book will be part of the Teaching and Learning in Higher Education Series edited by James Lang.

Show Notes

Transcript

John: Class discussions of race and racism can be difficult for all participants. In this episode, we discuss ways of building a classroom climate in which these issues may be productively explored.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Cyndi Kernahan, a psychology professor and Assistant Dean for Teaching and Learning at the University of Wisconsin – River Falls. She’s also the author of Teaching about Race and Racism in the College Class: Notes from a White Professor, which will be available from West Virginia University Press in Fall 2019. The book will be part of the Teaching and Learning in Higher Education Series edited by James Lang. Welcome, Cyndi.

Cyndi: Thanks.

John: Welcome. Our teas today are:

Cyndi: I actually I just have water although I am a big tea drinker usually.

Rebecca: I’m drinking golden-tipped English Breakfast tea.

John: That’s a new one.

Rebecca: I know I’m branching out! [LAUGHTER]

John: And I have blueberry green tea today.

Rebecca: We’ve invited you here today to discuss your forthcoming book, Teaching about Race and Racism in the College Class: Notes from a White Professor. Can you tell us a little bit about the book?

Cyndi: Yeah, the book is essentially my answer to a question to my earlier self. So when I started teaching about the psychology of racism about 20 years ago, when I first started here, I felt desperately in need of help, because I’d always wanted to teach about the psychology of racism but it was much more difficult than I anticipated, as most teaching often is. And I was very young and new and I wanted a guidebook and there really wasn’t one. And so I kind of have had it in my mind for a long time and about five years ago, I started thinking seriously about how to do it. So the book is meant to be sort of a guidebook. It’s got both my own experiences, but also a lot of evidence in it. I’m a social psychologist, so there’s a lot of evidence from my field that I think is very easily translatable to the classroom in terms of how to learn and how to think about these issues because they’re hard to teach. It’s hard to teach about racism, I think. There’s a lot of difficulty in it. There’s a lot of evidence and also just sort of my overall philosophy about how we can teach it in compassionate but very honest ways. And so that’s my overall thinking…making sure that you tell the truth but that you tell the truth in a way that doesn’t alienate your students and keeps them engaged, which I think is kind of a can be a difficult line to walk. So, that’s kind of what it’s about. It covers a lot of different things, student resistance, creating a good climate, how to take care of yourself as an instructor when you teach this sort of stuff. But, those are some of the basic ideas.

John: A few years ago, with the election of Obama, there was some people who claimed that we had moved to a post-racial society. I think evidence since then has shown that that hasn’t quite been the case.

Cyndi: Yeah.

John: And I think the book is particularly well timed because these issues are in the forefront with the news all the time. How do you begin to address issues of race in your classes?

Cyndi: First of all, I think that idea of the post-race thing is really interesting, and I see it a lot in students. I mean, I’m just finishing up teaching this class now. We’re in our last week of classes now, it’s finals next week. And when most of my students, most of whom are white, came to the class a lot of them just have this colorblind idea, which is similar to the idea of post-race, like we’re done…sorted that out in the 60s, it’s all good. And it’s obviously not…and so they believe that we’re in this equal playing field, which we’re not really in obviously. And so that’s kind of a starting point. I talk about that in the book, this colorblind ideology that most Americans share. The first order of business is sort of getting through that. And so there’s, I think, two main things that most students and most people (especially white people) need to understand. One is that colorblindness isn’t really possible, even though we think it should be the norm, it’s not really the ideal. So that’s one thing, but then also that there is this larger structure of what people in my field would call institutional racism or structural racism. And that’s the piece that I think most white people, most students don’t really get: that racism is not as people said, individual acts of meanness, it’s also these bigger things that affect us that we don’t think about. That’s usually where I start. We talked about what race is and what it’s not, what institutional racism is, and what it’s not. So I think that mostly answers your question as to where we start.

John: And that feeling of colorblindness is that more unique to white students, perhaps than students of color?

Cyndi: I think it’s more unique, but it’s not exclusive to white students. Students of color can often struggle with that understanding of institutional racism, and structural and cultural racism, as well. If you look at attitude surveys, it’s not unusual for people of color to say that they don’t necessarily see it in institutional or structural terms. Or you’ll see surveys, they’ll ask, “What’s more important? Individual behavior or institutional laws and policies?” And almost all Americans with the exception of really recent immigrants and Native Americans, I think, say that the individual behavior is more important. And as a social psychologist, I would say, actually they’re both important. But as far as what impacts your life more, it’s those big, broad institutional, cultural stuff. So I think white students are more likely, but not only.

John: How do you make students more aware of those issues? How can you help get them past that notion of color blindness?

Cyndi: One answer is a lot of evidence, but it’s how you deliver that evidence. My usual way to try to get these things across is to combine a lot of statistical evidence, a lot of broad evidence, with stories and examples that are representative. So I try really hard in my content, like I don’t just cover a bunch of psychology experiments, and I don’t just cover statistics. I try to have that together with individual stories of people’s experiences. And I also think discussion is really key. So I don’t lecture in this course, really much at all. I’ll do some mini-lectures. But, that’s never the main thing that I’m doing because I think it’s really important for them to read, and then come to class and process all that stuff. Because the number one thing that happens, again going back to the misconceptions they come in with, is that they realize that there’s all this stuff that they didn’t know. So we cover a lot of history, for example, and there’s all this history of how we got to the racial categories that we have now that they’re just like, “No one told me this.” And they need to hear other students say that too. That’s part of also creating that climate is like, “Oh, I’m not weird or stupid for not knowing this. All these other people didn’t know it either.” And we talked a lot about, “It makes sense that you wouldn’t know because we don’t really teach it in our K-12 system very well for most students.” So, I think it’s a bunch of things. I think it’s what the content looks like, it’s how the class is structured, it’s how the evidence is presented. I think all those things matter.

Rebecca: Many faculty members avoid talking about race, especially in classes that are not about race specifically.

Cyndi: Yeah.

Rebecca: So can you address maybe why faculty do that, and how to help faculty overcome that fear?

Cyndi: I think fear is the main reason. And there’s different types of fear. One is: I don’t want to be the bad guy and I don’t want to be confrontational, which is understandable. Many years ago, I was talking to a friend of mine who taught in our English department, and she was teaching something called ethnic film and literature. And at the time, I was coordinating ethnic studies, and I really wanted her to teach that class again so that I could get it back into the rotation. And she just told me…we were at a party and she said, “I’m not teaching that anymore.” I said, “Why?” And she said, “Because I have to fight with them about whether or not racism is a real thing. And I don’t want to do that anymore.” She didn’t want to put up with the resistance, essentially. And she didn’t want to have to be what she felt like was the bad guy to deal with that resistance. And so I think that’s a big piece of it. If I try to tell students the truth about this stuff, they’re just going to resist and then I’m going to have to deal with that discomfort. And that’s a real fear, particularly for instructors of color. I mean, they’re all these national examples, right? …of people getting called out by their universities for basically just telling the truth in their classes and trying to teach institutional racism. The most famous example was Shannon Gibney over at Minneapolis Community and Technical College near where I live, and she was officially reprimanded by her University, basically for teaching what I teach, because she was getting pushback from white students, essentially. So I think that’s part of it. Also, it creates a lot of dissonance in students which was related to the resistance, so knowing how to deal with that dissonance can help students feel okay about themselves even as they recognize that they hold a lot of these beliefs and they haven’t really been very critical about it. So I think that all those things, all those types of fears play into why you don’t see people covering it.

Rebecca: How do you suggest maybe faculty get over that or feel prepared for that resistance or can actually deal with that in the classroom and not feel shocked or distressed or overwhelmed.

Cyndi: The big key, I think, is being prepared and feeling like you know how to talk about it. I’ve also heard a lot of instructors say, “I don’t feel like I have enough knowledge.” For white instructors, they feel like they don’t have the right or the credibility to talk about it. So that’s sort of an issue, and for instructors of color, there’s a whole other set of things. If you look at the research for them, there’s just a ton of microaggressions that they often have to deal with. They’re also seen as not being credible purveyors of this information. So that’s an issue. So I think just a couple things: one being as prepared as possible. So knowing your subject really well…being clear that when you teach the class, you’re gonna be clear with the students like, this is the evidence we’re going to use. So you’re not coming at it, like it’s all people’s personal experiences or opinions. That I think is where it gets especially hard. But if you know that, you’re going to come back to this scholarly base of evidence that usually makes most of us more comfortable, because that’s how we roll. …and people had this misunderstanding of teaching about race: “Well, it’s all just opinion.” I once had a student say, “How could you possibly give us a test on this? I mean, it’s just all people’s opinions.” I was like, “No, not quite. [LAUGHTER] There’s definitely evidence there. There’s psychology, there’s sociology, and there’s history and we’re going to use all of those things.” So I think that part of it is knowing that you have this common base of information. And also knowing what to expect…how to deal with the resistance, knowing what the resistance looks like, I have a whole chapter on what resistance looks like and how it manifests in white students versus students of color and how to think about it and how to deal with it. So I think that can be helpful too.

Rebecca: Can you talk a little bit about maybe one or two ways that we tend to see resistance and a couple of strategies that we might be able to use to overcome that?

Cyndi: Yeah, I think, in general, what you see with the research is that white students tend to be more resistant than students of color in general. They tend to be more vocal about it than students of color. One sort of broad distinction you sometimes see is that students of color are more likely to leave if they’re the minority in the classroom. So, if you have mostly white students (which is a lot of what I deal with, and maybe I have a few students of color), you might get more passive resistance on the part of the students of color, they sort of withdraw a little bit more, because they don’t want to be the one student saying things in the class and then being really looked at or targeted in that way. And for white students, they just tend to be more comfortable speaking up about this, particularly in the primarily white environments. So I think the ways to get around resistance are: 1. always sort of coming back to the evidence that you’re using. And if you set the table at the beginning of the class that can usually work so you can come back to: “Okay, here’s what we’re going to focus on.” So for example, I’ve had students say in class, when we talk about wealth disparities or something, they might bring up a particular person that they know who doesn’t fit that. And so what I found is useful as I don’t fight with them about whether or not their story is true. I don’t know the wealth of this individual family of color. So if I tell you that the wealth disparity is like 10 times… white families have on average 10 times the wealth that black families have…and they say, “No, no, there’s this one family I know. And they’re, they’re really, really rich…,” you know, as sort of an anecdotal thing. That’s a way you see resistance manifest, right? And so instead of arguing back and forth about whether or not that family is rich, which is useless, you can just undercut that and talk about the general wealth disparity. So, that’s one way to do it. That’s obviously pretty mild resistance. If you have consistently students questioning evidence, which doesn’t happen as much as you might think, bt it can happen, but it can happen, then I sometimes will take that out of class. For students of color, the way I try to work with the resistance you often see there, which is that sort of passive withdrawal because again, they don’t want to be targeted within a predominantly white classroom, one technique I use is to send an email really early in the semester that says something along the lines of, “We’re going to be talking about racism all semester. Your classmates may look to you to be a representative, just know that I know that you don’t need to do that. That’s not your job. And I’m not going to expect that of you, because I want to try to make those students especially feel as safe as possible. And I also recognize too, I reinforce to them: “I’m a scholarly expert on this. But this is your lived experience and I recognize that. Not saying that one is better than the other, but just I see you and I see that your experience is going to be different than the white students in the class. And so that’s the way I try to short circuit that withdrawal from class as much as possible. And most students, at least so far, have appreciated that…and also just acknowledging way up front that this is going to be new to you. You’re going to be uncomfortable, I put it in the syllabus: “You’re going to feel uncomfortable, this is not comfortable stuff to talk about, and so just know that it’s coming.” So those are a few things.

John: Do you recommend having the class come up with rules of engagement or discussion on these issues.

Cyndi: I definitely do. I mean, I have some that I think are important, but I let them drive that discussion. And then I add them in if I feel like maybe they’ve missed them, or something. And I usually have them. Actually, I think I learned this from an earlier version of your podcasts, someone was on talking about having the students working groups to develop their rules of engagement. And so I did a little bit of that on the first day this semester, let them talk about it in small groups before we talked about it in large groups. And then I just take a picture of what those agreed upon discussion groups are and I post them up on Canvas, so that that way they can come back to them. Yeah, we have a whole discussion about discussion: about what it should look like in our class, how we want to engage. One big question we always talk about is do you want to raise your hands or just talk? How do you want to handle somebody upset? They always think it’s going to be more contentious than it actually is, at least so far in my experience. I haven’t had a whole bunch of anger or confrontation. So that’s been so far that’s worked well.

Rebecca: What are some of the consequences of not dealing with race specifically in classes or subject matter that doesn’t directly indicate that race is going to be a part of the conversation? So we often hear this from faculty in math or science, or I would even say in my area of design where it’s not obvious that race might be something that should be discussed. What are the consequences of completely avoiding it?

Cyndi: Well, obviously it marginalizes it. So, it turns it into something that only some people can do. Only some people can cover these topics, only some people are allowed to on some level. And it’s interesting you ask that because this has come up on my campus in the last couple of years in a couple of different ways. Because we have, I don’t know if you all have it, a diversity requirement, but we do. So we have one course, American Cultural Diversity, that students are supposed to take. And there’s been sort of a…fight is a strong word, but…argument over who should be teaching those courses. And I don’t think just anyone should be teaching them. But I do think people can develop an expertise within their own field. So for example, I have a colleague (and we have a very big agriculture college on my campus), and she teaches soil science and crop production and all that sort of thing. So that’s an area where you think, where’s the race going to be that but it’s deeply in it because she’s teaching these future farmers and folks working in that industry. And there’s so many immigrants that work in farming. So she developed and she developed the expertise on this. And she developed a class on immigration…migrant farm workers, essentially. She and I talked about that course many times when she was in the process of developing and starting to teach it. She teaches it regularly. And I think the advantage of that is that, for the students who are majoring in that college, that’s going to feel very relevant for them. And so again, I think people have to develop some expertise to do it, and then they need some tools for how to teach it. But we all need tools for how to teach it because it’s hard. So, I think it’s important because it allows people to be more motivated to see how it relates to their specific field. I also have a colleague in math, who has talked a lot about trying to develop some courses around race and whiteness and math. You may recall, there was a math professor, I think, in Illinois, who was writing about white privilege and math and got a lot of haters online, essentially. Basically a really good scholarly analysis about the way white privilege works in math. And that stuff is really useful for students. It helps them connect in ways that are relevant to them.

John: You mentioned using narrative. While evidence is useful, it doesn’t always reach all of the students. I’ve been teaching about discrimination in my labor economics classes for decades. And it’s remarkable how resistant people are to facts. Because they rely on the sort of narratives: “My third cousin twice removed is this exceptional case. And it means all this evidence is wrong.” But you mentioned using narrative perhaps in a constructive way to help students understand that. Could you give us some examples of that?

Cyndi: Yeah, sure. Like the example that pops to mind right away was a really powerful piece that I used this semester on black maternal death rates, you may know that they’re wildly disparate, right? There’s a huge health disparities when it comes to race and healthcare. And so there was a really nice piece that was actually in the New York Times Magazine. That was this really good combination of a personal story of one woman and her experience. She lost a baby and then she was pregnant again and dealing with that… and there have been all these famous cases. Serena Williams most famously talked about her complications during birth and not being believed by doctors. So this piece was really great because they both had her story but then woven into it, it was a really well written piece. It had all the statistics, the differences…And the students…I gave it to them over a two-day period, because it was pretty long, and we discussed it for two days. And I just used it as this larger example of implicit bias in the healthcare industry. And these larger statistics on the disparities, the wealth gap…it illustrated a bunch of things that we had already covered. And the students loved it. It was hard for them to read, but they were very engaged with it. And some of the questions I got were excellent. And I’ll just say, too, I think one important thing is that when you come into a discussion, it’s really important to have a base to discuss with. And so, every day they have to read and post questions. I don’t give like reading quizzes, but they have to post questions so that I know that they read or at least mostly read, and I grade them. Because if you don’t do that, then your discussion is not good at all. If people don’t have a base of evidence to work from, you’re not going to get anywhere. And so those types of materials…I’m always looking for them…where they have that combo of the broad statistics and also the individual stories…and yeah, I know economics, it’s really tough to get those pieces. It’s the same for psychology can be really hard to find, but they’re useful.

John: Well, certainly in immigration and discrimination in labor markets, there’s a lot of examples out there and lots of good content.

Cyndi: Yeah, there definitely is. You just got to always be on the lookout.

Rebecca: So, what happens when it’s a conversation where the discussion point wasn’t going to be race, but then it becomes race. There isn’t a piece that you’re going to discuss ahead of time. But it pops up in conversation, it needs to be addressed. Do you have any strategies for handling those more impromptu situations that occur?

Cyndi: Well, for me, I think I always go back to then, either being honest that I don’t know enough to comment on it. What that question makes me think of are when students will bring up current examples, either that I don’t know or that I don’t know enough about. And so I will always try if I can to find some relevant psychological data or evidence or sociological data or evidence that I can bring it back to. And there have been times when if current events are happening, and I feel like I don’t know enough, I’ll just say, I don’t think we know enough. So if we have time, maybe we all spend a few minutes on our phones trying to find some information and you can do that right in class sometimes. But I always see myself as sort of a guide in terms of helping them sort through what the larger patterns are in terms of that evidence. And I think if you create a decent enough climate where they trust you, that usually can work. But yeah, the impromptu stuff can be tough, particularly if it’s not your area. I think that’s where, again, being prepared is hard. And I think you can just say, if this isn’t your area, like, “Let me go and find some more. Let me go find some stuff out and then we’ll come back to it.”

John: On dealing with things in the moment, how would you recommend people respond if someone makes a comment that somewhat racist without realizing the impact of that?

Cyndi: A really good example of that happened actually in a colleague’s classroom on campus this semester, and he called me after it. I forget what the term was, but a student had used a term in class that he found offensive and he was pretty sure other students found offensive and he didn’t really deal with it in the moment. And so he called me and asked what I thought. And I think one of the strongest ways to deal with that is, oftentimes, students don’t realize or people don’t realize that a turn might be offensive, or it might seem offensive. And so a lot of times, and what I advised him (and he said, it worked pretty well) is to go into those conversations, discussing it in terms of kind of a growth mindset idea. You know, here’s this term that was used…it’s offensive for some folks, but some folks might not understand why and then maybe talking about why that is. The word “colored” gets used a lot… “colored people.” And it makes sense in some ways that white students now who are very young wouldn’t know that that was an offensive term in the 50s and 60s. And so they use that term because they hear people say, people of color and then so colored people seems like a normal permutation of that, right? But it’s really not. It has this very unique history and so you could talk about how “Here’s this history that you may not have understood. And here’s this term that people didn’t use to describe themselves, it was used about them. And so that’s part of what makes it offensive.” And it’s normal that language changes and it evolves. And there’s plenty of examples you can bring up around that, like we talked in my classes about the word queer and the way that shifted over time…and language evolves…and so just sort of accepting that you’re going to make mistakes, you’re probably going to say things that might be offensive, but what’s the mindset that you bring to that? Do you bring the mindset that it’s normal, and you’ll figure it out, and you have to make your classroom safe for that too. So that it’s not like people are being called out and told that they’re saying the wrong word. We talked a lot in class about the difference between willful ignorance, like, “I know it’s wrong, but I’m gonna say it anyway” and just ignorance…like just really not knowing and coming out that and I give examples of my own, like times I screwed up…things I’ve said that were wrong, as a way to help them see that you’re never finished. I’ve been doing this for like 20 years, and I still make mistakes. I think that helps.

John: When you’re setting the classroom discussion rules, would that be a good time to bring that up?

Cyndi: Yeah, I think so. I think that you could, you could talk about expecting people to make mistakes. In the rule setting phase, you can talk about not expecting perfection, and how people will make mistakes and that’s all right, and ways to sort of come back from that, and gently talk about it rather than calling people out.

Rebecca: How do you handle microaggressions or other behaviors that might happen in class, that aren’t just like a word or whatever, but it’s something that’s happening or you see a pattern of behavior with a particular student. And maybe it’s something that you feel like you need to handle one on one. How do you usually handle those kinds of conversations?

Cyndi: Dealing with those. It’s usually much better one on one, because again, just like anybody, if you if you were to call someone out in class, then you’re likely to just get defensiveness and nobody’s going to be able to hear it. So what I’ve done in the past has been to talk to students one on one rather than to frame it as “You’re a bad person for doing this,” it’s like, “This is what I’m seeing. This is the pattern that I’m seeing. This is how I think it could be perceived” …and then just listening to what you hear. And you have to have a fair amount of trust with a student to be able to do that. But in general, I think whenever it comes to talking about someone’s racist behavior, it’s always better to focus on the behavior rather than the person. This is why when you hear national conversations about is that person a racist, I always want to throw my radio or my phone or whatever because it’s so frustrating to hear it framed in that way. “Is someone a racist?” is not a useful question. And I never quite sure what that means, because the goalposts always move, in terms of like how we think about what that word means. So, instead, focusing on the behavior, this is the pattern I’m seeing this is a problem. And I think if you’re in a moment where (this would be less with students, but more with colleagues) where you’re seeing this happen, and it’s directed at a person who has a lot less power in that moment… so like, a person of color, for example…you could step in and say, “This is what I’m seeing, and this is how I think about it.” So you’re not putting it on the person who was maybe the target of it. But you are saying in that moment, I see this and I see that this is a problem. That can be harder to do and less with students, I think more with colleagues. But, in general, it’s just sort of noting that it’s happening and being honest about it without necessarily saying you’re a bad person for doing this.

John: Last fall, we had a reading group addressing some of these issues. And one of the issues that came up in a lot of discussions is how to address these issues with colleagues, particularly those who are evaluating you for retention, promotion, and similar issues…

Rebecca: or hiring…

Cyndi: That is so hard when someone is in a position of power. Because if you’re the job candidate, there’s just no way that you’re going to be able, in that moment, to be able to do that.

John: What if you’re a junior faculty member on, for example, a recruitment committee and you observe comments or behavior that seems to be biased in some way. What would you suggest to a faculty member in that position?

Cyndi: I think you could go back to the sort of something called micro resistance. And there’s been a little bit written about this. In terms of how to deal with it, again, not making it about the person but just saying like, this is what I’m hearing, this is what I’m seeing. This is how I feel about it. And so you make it more about yourself. In extreme situations, and I’ve certainly been in them and seen them, you could go to other people that you trust on the committee and say, “This is what I’m seeing. This is what I’m hearing.” This is slightly different, but I had a slightly different but I had a student come to me last week and say that she’s in another course. And she’s hearing this from an instructor. And so then I was able to go to that department chair and say, what’s happening? So, I think using your mentors, using your colleagues, if you’re in that lower-power position; and if you’re in a higher-power position in those same spaces, try not to make the target responsible for that. If you’re a man, and you’re seeing sexism, it’s useful to just call that out. And again, not calling the person out, but just saying, “This is what I’m seeing. This is the pattern.” We talked about this actually, there’s an interesting anecdote in my class this last week. We were talking about this micro resistance thing and one of the students is a softball player and she’s on the softball team. According to her, there’s one black softball player on the team and everybody else is white. And, according to my student, whenever racist things will come up, like, they’ll all look at the black student to ask her “Is this okay?” And we talked in class about like, maybe that’s not fair to put that on the student of color. This white student feels like she really wants to be an ally, like she really wants to be an advocate. So we talked about, well, maybe you just say what you think about it, rather than asking her “Is it okay?” or going to her afterwards and saying, “Do you feel okay about this?” Because what is she going to say in that moment? I mean, she’s in the minority…the black student is, and so I think that can be a useful way to think about it too, because a lot of times we want the person who’s lower in power to like, excuse it and make it okay. And that’s really not fair. And I think it happens just because people don’t think about the power dynamics at all. They just don’t think about it. It doesn’t occur to them. And so trying to be more intentional about what is the power in this situation and trying to be more fair.

John: One of the issues if there’s a small number of minorities in a class, one potential issue might be stereotype threat. What are the consequences of that? And how can we address that perhaps by making it a more supportive environment?

Cyndi: Yeah, stereotype threat is really interesting. I know a lot about this. Actually, I’ve given lots of workshops on this, in addition to like implicit bias and stuff, too. And it’s a real problem. The consequences are…they’re sort of short term and long term. So the short-term consequences of stereotype threat is that you have students who underperform. So in a test situation or on a writing assignment, where you have a student who is feeling stereotype threat as a result of race or gender or social class. And so then it just create that extra layer of anxiety and stress, essentially. And it’s not always apparent. And you don’t necessarily know that that’s what you’re experiencing. But we know from the neuroscience research that, you just have less working memory in those moments because of stereotype threat. And so the short-term threat is that you underperform. The long-term consequence is that students disengage from the area altogether. So this is why we hear.…I’ve heard it so many times from my female advisees…“I’m not a math person. I’m not a science person. And I think it happens in art as well. I don’t know about design specifically…

Rebecca: um hmm.

Cyndi: …but you’ll get like, I’m not an art person. I’m not creative.

Rebecca: I can’t draw.

CUNDI: I can’t draw, yeah, that’s it. That one’s, I think, less about race, maybe a little bit more about gender, but it’s a very similar thing of like, “I don’t feel like I can do this. I’m not creative.” And so I’m just going to withdraw from it altogether. And so you see what Claude Steele calls dis-identification. So I’m just going to dis-identify with that field. It’s just not my thing. I’m going to go get my self esteem somewhere else. And obviously, that has serious consequences if the thing you’re dis-identifying with is school altogether. And so that’s why we see this underperformance over time with students of color and with women in math and science. The ways to get around that…there are a few. There’s a whole set of interventions that social psychologists have developed that can be really powerful. I guess I would send listeners to the mindset network web page. I don’t know if y’all have ever seen that. It’s mindsetscholarsnetwork.org. But it’s a bunch of social psychologists who have gotten together to create these really pretty low-cost interventions around increasing belonging…using values affirmations… Utility value is another one…growth mindset. There’s a bunch of them and there’s a little tweaks that you can do in your classes to help that. The other big intervention, and you can sort of call that active pedagogy. So there’s really good research that the more active your class is, that’s going to be good for everybody. But, it’s especially good for your students of color your first-generation students and your students who are women in math and science and engineering courses, where they’re more likely to feel that thread. That as an intervention itself is really great. There’s a ton of really nice discussions of that, and studies of that that you can find in terms of active pedagogy being an inclusive pedagogy. Because, in general, you want students to have a sense of belonging and you want them to feel included, and that’s going to help to undercut that, because really all stereotype threat is about is about a lack of trust. So everybody thinks it’s a lack of confidence in the student. It’s not. They don’t trust the environment to be fair, and so that’s why they disengage and they pull back. And so you want to you want to do everything you can to keep that trust.

John: …and they build more of a sense of community with their fellow students.

Cyndi: Absolutely.

John: I’m going to our conference in a few weeks, and one of the activities there is something called “sip and paint.” A friend of mine tried to convince me to do that. And my reaction was “No, the last time I painted I think I was seven years old.” [LAUGHTER] So, there’s a gender issue perhaps with the artwork thing.

Cyndi: Creativity. Yeah.

John: You mentioned implicit bias. My labor classes are online and one of the things I do is I have them take some of the Implicit Association tests, and then discuss them. And they tend to be pretty comfortable discussing many of them, but they tend to be much less comfortable discussing race.

Cyndi: Oh yeah.

John: But one of the things that led to some really good discussions are the associations between gender and careers.

Cyndi: Yeah.

John: And a lot of female students remark on how surprised they are that they associate women with home activities and men with careers. But, one of the things I note from the students who tend to perhaps have the more resistant attitudes towards facts in general, from other discussions, is that they tend to question the tests themselves and say, it’s clearly set up to demonstrate a bias when that bias really doesn’t exist. And those students are really hard to reach and we can keep giving them facts. But I’ve never been completely successful in getting through that barrier, at least in any one course. Any suggestions?

Cyndi: It’s really tough. As a social psychologist, I feel pretty comfortable talking about the Implicit Association test, but it is really hard to describe well, so that’s one problem with it, because you try to explain “No, no, like 25 years of research…” When I still had paper versions of the literature, I gotta bring in my big giant folder and I just sort of slap it on the desk and be like, “They’ve been studying this since 1995. But, like you said, the facts don’t always help. One thing I think that helps with them understanding implicit associations, is to depersonalize i… and I have some great podcast and book suggestions and article suggestions on how to help them understand what implicit associations are. But really, it’s not about them as a bad person. And that is one way I found to get at it. There’s a phrase that gets used by Mahzarin Banaji, who was one of the test co-creators and she talks about implicit associations as the thumbprint of the culture, which is really accurate, you know. So it’s not you’re a bad person, you have implicit bias… like, we all have it and it’s the thumbprint of the culture. You’ve been learning since you were a baby, what’s associated? what’s good and what’s bad? I mean, it really is that crude. It is your brain saying, “This group is bad. This group is good” over and over and over again, you get those messages. So if you can de-personalize it, I think that can help a lot. I have found that using the podcasts that I have on it, and some of the more newsy articles and they cite the researchers, that can be really helpful, too. But yeah, it’s they want to criticize the test all day long. I’ve gotten to where I don’t have them take the test until after they have a decent grounding in the science because they’re very resistant to the idea. They think the test just sucks.

John: At least those who have their preconceptions not confirmed in the way they’d like them to.

Cyndi: Yeah, because again, they think this means I’m a bad person. They think it’s the racism test. There’s a King of the Hill episode. I don’t know if y’all have ever seen that show, but I used to love that show. And there’s an episode where Hank has to go take the racism test, because he’s worried that his dog is racist or something. I can’t remember the full thing of the story. But, that episode is one of my favorites because it’s like, “Okay, let’s see if he’s racist.” But, that’s not the way it works, folks. I’ll have students sometimes say like, “We should just have all cops and all teachers and all judges take this test. And then we’d know who to hire…” and I’m like, “There’d be nobody left. There wouldn’t be enough people left to do all these jobs.” And I think if you talk about it in that way, it can make it so that it’s not a moral failing, which is, I think, why they’re so resistant.

Rebecca: I’ve done something as a follow up to doing some of the tests in my classes where I had students look at their portfolio of design work, and just see who was represented in the materials that they made. And what they usually do is discover that either it’s a lot of people that are just like them, or that it’s white and young….

Cyndi: Yeah.

Rebecca: …which some of the people in the class may not fit that particular group, but that’s what they’ve still represented. And that helps a lot, because we talk about, “Well, it’s easier to design for a group of people that you’re around all the time, perhaps”

Cyndi: Yeah.

Rebecca: Or, You know, know what, like this particular population, maybe preferences, if that’s a group that you’re a member of. And that sometimes helps too because it kind of breaks down some of the total ownership or blaming a student for something. It becomes more of that cultural identity piece.

Cyndi: Yeah, you can ask them, like, “Who’s most of your friends? Who’s in your environment?” I have them write journal entries all semester. I don’t say “Go find the racism and tell me about it” I just say like, ”Just tell me what you observe in terms of both race and gender.” Just like “What do you see? Who’s doing what jobs? Who’s in what space?” and that helps them too to start to see the stuff that they just sort of take for granted, because it’s the water that we all swim in. We’re all very segregated. And so I think it’s good for students to recognize that and then how that plays itself out in who you select to design, for example, and who comes into your consciousness. So again, thumbprint of the culture rather than moral failing…bad person.

Rebecca: I also do an activity in my capstone class where I ask students like, who are there five designers that inspire them, and then I end up with a pretty small list when we aggregate all of them together. And then I say, I’m going to ask this question again later in the semester, and I expect these lists to be really different. [LAUGHTER]

Cyndi: Yeah, that’s good…makes them explicitly think about it.

John: We always end with the question, what are you doing next?

Cyndi: I want to write more about these issues. What I really want to do is run some workshops for faculty. I’ve done a couple. Most of my workshops have been on stereotype threat and
implicit bias. So I would like to run more workshops on this topic in particular, like how do you teach about racism rather than teaching inclusively. That’s fun to talk about too. But how do you how do you teach about race and racism? I would love to do more of that. And I would also like to write more about these issues. Because I think it’s hard to do. And so I would like to just have more conversation. I’m also hoping eventually to maybe write a different book about inclusive pedagogy. We’ll see. I’m not sure. it’s a ways off.

John: And when is your book coming out?

Cyndi: It’s supposed to be November, I believe, November or December.

Rebecca: Well, I know that will probably have a line of people now that really want to make sure they get their hands on your book, because…

Cyndi: I hope so.

Rebecca: …there’s a lot of books that deal with these issues conceptually, but not in a practical way.

Cyndi: I could not find a lot on teaching about it. Like I said, I wanted the guide that I wish I had for myself 20 years ago, but I there’s just there’s not a ton. There’s a lot of good chapters on it in some edited books, but there wasn’t a lot that had sort of an overarching idea. So that’s what I wanted to try to do.

John: We were looking for that just last year. So, we will have it on pre-order very soon.

Rebecca: Yeah, Definitely.

Cyndi: Cool. Thanks.

Rebecca: Well, thank you so much for joining us. This was really great.

Cyndi: Yeah, thanks so much for asking.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

85. Small-Group Discussions

Posted in active learning, discussions, evidence-based practices, higher education, learning, online learning, pedagogy, peer instruction, role-play, teaching
June 12, 2019

Small-group discussion activities provide all students, even the quiet ones, with an opportunity to actively engage with course material. In this episode, Dr. Dakin Burdick joins us to explore a variety of small-group discussion activities that can be productively integrated into our classes. Dakin is the Director of the Institute for College Teaching at SUNY Cortland. He has been active in professional development for almost 20 years, and has served on the Board of Directors for both the Professional and Organizational Development Network in higher education (the POD network) and the New England Faculty Development Consortium, where he was a president for four years.

Show Notes

  • Burdick, Dakin (2019). Small Group Discussion Protocols
  • Joan Middendorf — Teaching Resource Center Director at Indiana University
  • Middendorf, J., & Shopkow, L. (2017). Overcoming Student Learning Bottlenecks: Decode the Critical Thinking of Your Discipline. Stylus Publishing.
  • IUPUI — Indiana University – Purdue University Indianapolis
  • Angelo, T. A., & Cross, K. P. (2012). Classroom Assessment Techniques. Jossey Bass Wiley.
  • 84. Barriers to Active Learning — Tea for Teaching podcast (with Lindsay Wheeler and Hannah Sturtevant)
  • Larry Michaelsen — Professor of Management at the University of Central Missouri, pioneer of Team-Based Learning
  • Michaelsen, L., Knight, A., & Fink, L. (2005). Team-Based Learning: A Transformative Use of Small Groups in College Teaching. Stylus Publishing.
  • Team-Based Learning Collaborative
  • Elliot Aronson — Inventor of the Jigsaw classroom technique
  • Aronson, E. (1978). The Jigsaw Classroom. Sage. Chicago.
  • Zoom — iOS, Android
  • Eric Mazur — Balkanski Professor of Physics and Applied Physics and Area Chair of Applied Physics at Harvard University
  • Teaching Professor Conference
  • David Pace — Professor Emeritus of History at Indiana University, Bloomington.
  • Pace, D. (2017). The Decoding the Disciplines Paradigm: Seven Steps to Increased Student Learning. Indiana University Press.
  • Pace, D. and Middendorf, J. (2004). Decoding the Disciplines: Helping Students Learn Disciplinary Ways of Thinking. Jossey-Bass.

Transcript

John: Small-group discussion activities provide all students, even the quiet ones, with an opportunity to actively engage with course material. In this episode, we explore how a variety of small-group discussion activities can be productively integrated into our classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Dakin Burdick, the Director of the Institute for College Teaching at SUNY Cortland. Dakin has been active in professional development for almost 20 years, and has served on the Board of Directors for both the Professional and Organizational Development Network in Higher Education (the POD network) and the New England Faculty Development Consortium, where he was President for four years. Welcome.

Dakin: Hi. Good to be here.

Rebecca: Today’s teas are…

Dakin: Yes, I am drinking Sleepytime Vanilla today.

Rebecca: That sounds yummy.

John: And a great way to start the day. [LAUGHTER] And I have Ginger tea.

Rebecca: And I have something different today. I have Strawberry Grapefruit Xue Long Flavored Green tea.

John: Okay.

Dakin: Nice.

John: We’ve invited you here to talk about effective ways of engaging in small-group discussions. You’ve done quite a few workshops on that. Could you tell us is about your shift to small group discussion protocols in your own classes and how you get started with using small group discussions?

Dakin: Sure. My method of instruction was lecture primarily. I started off in history and I did a lot of lectures. Some of them were good—and I’m proud of those few—but I was teaching a class at Indiana University Purdue University in Indianapolis at Columbus and it was a U.S. history survey, first-year survey. It was once a week, three hours long, and some of the students had traveled for more than an hour to reach the class after a full day of work. Some of them came in with their dinners, and I knew that lecture class was not going to get me through that class. Nobody would survive it; not even me. [LAUGHTER] So I worked with the Teaching Resource Center Director at Indiana University, Joan Middendorf. I selected several small group discussion methods: jigsaw discussions, role playing. I modified the debate system to create a evidence-based debate protocol. I used the just-in-time teaching method from IUPUI, and the classroom assessment techniques of Angelo and Cross, which I kind of regard as my Bible. The combination of those worked really well. The students remained active throughout the class, we were often surprised to find the class was over, everybody was still energized, I still had a protocol or two to go. Everybody learned each other’s names because we did random groups, and the class as a whole was tremendously successful. I was really happy with the results and I’ve used it ever since.

John: Whole-class discussions are often used but what are the advantages of small-group discussions relative to a whole-group discussion?

Dakin: I actually advocate both of those. I advocate lecture, small-group, and whole-group discussions for different purposes. The large group for me is one where I would often find faculty having the usual suspects were the only ones talking. You had three or four students in a class of 20 to 30, no one else was talking, and the faculty member would usually come to my office and ask, “Well, how can I change this? How can I get more students involved?” and often, that’s where the diagnostics began. What I found was, first of all, they needed to have preparatory homework, the students needed to do it, it needed to be graded, and if it was graded and it’s frequent assessment in order to reduce faculty load, they had to grade this lightly, and place the effort onto the students and not onto themselves. And if they did that, usually things improved. The advantage of the small groups is that if they’ve done all that work, then the students wants to talk. If you get to a large-group conversation and you’re not talking, it’s pretty boring for the students that aren’t talking. For the faculty member, it sounds like it’s a really good conversation because they’re the center of the wagon wheel, they’re the center of the hub, and so they’re constantly talking with those three or four students, but they’re wondering what’s happening with the rest of them. And that’s a good thing to wonder about, frankly. If you use small groups, you have the advantage that more people are talking at the same time. So instead of having one person talk at a time, you can have six people in the room talking at a time, so there’s a lot more conversation taking place and hopefully, more change in learning, which is important. Students in small groups feel more free to talk, there’s less risk in a small group, they can gain confidence from that talking, and they’re more active in the classroom at the same time because there’s more people talking. It also gives them the chance to practice disciplinary skills that the faculty member has put into that assignment. So the assignment shouldn’t be about declarative knowledge or facts, it should be about how can you do something in the field? How can you emulate the skills of an expert? The other piece of this is that the small groups have been demonstrated to be effective. Students in the 1920s said they preferred this sort of discussion—at least large-group discussion at that point—and then by the 30s and 40s, there was research and social psychology showing that small groups were more effective in promoting change and student learning. And from there it went on and since the 50s and 60s, a lot of different types of protocols have been invented and developed. There’s just a lot of advantages.

John: Could you tell us a little bit more about how you ensure that students come prepared to actively participate in those small group discussions?

Dakin: The main thing is grade it. The rule is that students—according to the Carnegie unit—are supposed to study for two hours outside the class for every one hour in. Well, the Study Study at Indiana University showed they weren’t doing that. They were studying one hour outside of class for every hour in, and it was usually on Sundays. That means that we kind of have to take that knowledge and make use of it. We know they’re going to do their work on Sundays. So okay, the assignments going to be due Sunday night, 11 o’clock, 11:55. But you make sure that they work harder. You don’t feel guilty about putting more work on them because more work means more practice. All of them could use more work reading, all of them could use more work writing, so that’s what I have them do. And then I make sure they turn their assignments in on Sunday night, I grade them Monday morning—which is a principle of just-in-time teaching—it’s preparatory homework, I read that, and then I modify my class based on what the students bring to it. And so I can see—first of all—their weaknesses. I can see their misconceptions. I also can see their strengths. Occasionally they have real strengths they bring to the class that would be totally invisible if I hadn’t done this work. And my example for that is I had a class where I was teaching the My Lai Massacre—about Vietnam—and I had in the class two very strong students. One was a G.I. who had fought in Vietnam, and one was a First Lieutenant who had taught the rules of land warfare at Fort Benning for three years. Both of these guys were A, smart; B, aggressive; and C, constantly fighting with each other because it was enlisted versus officer. So they dominated the class very often. And when I got back their feedback, they told me their knowledge and I hadn’t seen that knowledge before. I did not know that these were their backgrounds. So I spent an extra two to three hours reading about the rules of land warfare, came to class prepared, and instead of a class that would be a trainwreck for me, what happened is they came in and the Lieutenant said, “Everybody knows the rules of land warfare. So they’re all guilty, and they are all responsible,” and the G.I. says, “No G.I. is going to read a 100-page field manual on the rules of land warfare.” And the Lieutenant says, “Well, there is no 100-page manual on the rules of land warfare,” and I said, “Well, actually, there’s three. [LAUGHTER] There’s the 1956, 1965 and the 1973 (revised on the basis of My Lai).” Okay, so that stops that conversation. Then I turned to the GI and I say, “Okay, but every G.I. has those little plastic helmet liners, right? …with the 10 rules of land warfare on them.” “Yeah.” “Okay. So we’re agreed; they knew the rules of land warfare. Some follow them, some didn’t. Now, let’s talk about why,” and at that point, the conversation became really useful. First of all, all the other students could participate, because they now had the background, and the two people that were real experts in the room could help us kind of determine why people followed or did not follow those rules. And again, if I had not done that preparation—just two extra hours—that class would have been ruination.

Rebecca: What are the kinds of questions that you have students respond to that maybe elicited some of the information that helped you? What are the keys to asking good questions for that preparatory work?

Dakin: I think the keys to that are knowing your subject. So, everybody that is a content master—every faculty member —has their own expertise, and it’s pretty impossible for me to name the prompts that they might use effectively. But they probably know them, they’ve probably seen them in their graduate work. They know that these are the elements that made up their dissertation exams, their qualifying exams, and they’re probably pretty smart about what are those major issues in their field that need to be discussed, and to be prepared, and the students need to be prepared. The big thing is making sure that we’re talking at a high level of cognition. So in Bloom’s taxonomy, talking about analysis, talking about evaluation, those are the levels you want to get at. And those are almost impossible to get at with multiple-choice questions or tests, so that means there has to be conversation, there has to be writing. Those are elements that are important.

John: Earlier you mentioned that you use random assignments for small-group activities. Do you do change the groups on a daily basis or do you have more persistent groups?

Dakin: Occasionally, if I’m doing long-term group work, I will do some sort of pre-test, find out what the strengths and skills of the students are, and then place them mindfully into those groups, so that they construct useful groups because they’re going to be in those for half a semester. I’ll do a swap halfway through, but that’s a long time to be in a single group. For the random groups, I definitely do that and I do that on a daily basis. The students originally complain about that, but they get used to it pretty quickly and they’re ready for it. And the advantage is that they get to meet everybody in the classroom. They get to be in a group with everybody else and that builds trust and it builds community. And that allows them, by halftime through the semester, they know everybody, they’re comfortable with everybody, they trust that other people have their best intentions at heart, and then the conversation just escalates from there because everybody’s now willing to talk.

Rebecca: In our previous episode that we just released last week, we discussed some of the issues that can come up when you’re using evidence-based practices for the first time. A lot of people know or buy into the idea of small-group discussions and might just go for it without necessarily having a good plan in place, and things might go awry. Can you talk a little bit about ways to be prepared for trying something new, the kinds of things that might go wrong, and how we might adjust ourselves a bit as faculty members as we’re trying new things?

Dakin: The thing that I find usually is that people just don’t give the new techniques a chance. It’s scary. Now there was a study back in the late 90s out at Brigham Young and they asked faculty two questions. They asked them first, “What do you think are the most effective teaching methods?” and then “What do you do?” and they were diametrically opposed. [LAUGHTER] And the reason was time management, people are very busy and the stuff that’s effective takes a lot of time to do—or they think so. So I view my job as an instructional designer when I’m helping them to reduce that amount of time and make sure that they can do that. So first, make it time manageable, so that you can do the task and you can feel comfortable. Secondly, trust the system. Trust the change you’ve made. You made this change for a reason, trust it. And third, trust your students. Your students want to succeed, they want to learn. Trust that and have them help you make this successful. Tell them what it is you’re doing in the classroom, why you’re making this change, why you think this is going to help them learn better, and then also use feedback from them to get it. So I typically will use something called a stop, start, continue—What do you want me to stop doing? What do you want me to start doing? What do you want me to continue doing?—and use that student feedback to then modify the class. So it’s kind of like a mid-semester evaluation, but I feel like doing it whenever I do… it is just fine.

John: Now earlier you mentioned that whole-group discussions have a place. In what sort of sequence might you use or in what combinations would you use small-group discussions and then whole-group activities?

Dakin: My process is basically four-part. One, preparatory homework. There has to be preparatory homework and it has to be graded—lightly graded—and it should be moderately challenging. Next, they come to class, there’s a brief lecture and the lecture introduces the material, frames the questions we’re going to talk about today, maybe corrects some of the errors that were made in that preparatory homework, also celebrate successes from that preparatory homework. Once that lecture is done, maybe 10, 15 minutes, then move them into small group work. Small group work can be anywhere from one to two minutes in a lecture hall to 40, 50 minutes—and you might do a whole session on the rest of that piece, maybe a debate or some large-scale exercise—usually though, about 10 to 15 minutes in small group. Then when you hear the sounds rising, that means they’re talking about things they enjoy, which means their social life, [LAUGHTER] and so it’s time to stop them. You’ll also see sometimes that there will be a student—maybe all the A students somehow got at the same table—and they’re done three to four minutes before everybody else. Well, the point of putting them into small groups is to build energy and confidence and you don’t want your A students to be bored. So if you have a group that’s done first, you appraise how much of everybody else got through, “Can I stop this now?”—usually you can—and you bring them back to the large-group discussion. And in that large-group discussion of 10 to 15 minutes, you do debriefing and you find out what they think they know, maybe use a classroom assessment techniques from Angelo and Cross, and you evaluate and you build feedback that you can use later. And then once you’ve got that, then you move back to the lecture, and you clean up the misconceptions, you explain and reframe the next issue, and then it’s just a cycle. So it’s lecture, small-group discussion, large-group discussion, and continually like that.

Rebecca: What are some strategies that you use in small groups to make sure that everyone participates or is engaged and stays on task?

Dakin: First, make sure they’ve done the homework. Secondly, randomize so that I’ve got some good students and some poorer students in the same groups, so that we have people that can interact—also, so that people can learn about each other. To keep them on task and walk the room: first of all, be engaged with them. Listen to what they’re saying and if it’s on task, you just congratulate them and move on, if it’s off task, okay, now start working with them—and there’s going to be one group that’s off task, certainly. Other pieces are… that you might encounter a small group where there’s a number of dominant individuals. So there’s a couple of people that are really assertive, and they’re talking all the time, and they are just dominating the whole piece, and the other people aren’t getting heard. And so in that point, then you start introducing other discussion protocols that will allow more inclusivity: so things like expense account, talking stick, things where other people’s voices are valued. Another one would be Larry Michaelsen’s Team-Based Learning that also does that.

Rebecca: Can you talk through each of those, for those that aren’t familiar with each of those?

Dakin: Sure, let’s start with talking stick. Talking stick is very simple. You have an object—usually a pencil or something—and one person gets that stick, and is able to talk for one minute without being interrupted, or any comments from anybody else. And then you pass it to the next person, for one minute they get to talk, and it goes around the room that way. And then once it’s gone around once, everybody can talk at once and kind of work out what it was that they heard said, but everybody’s voice is listened to and heard during that time. That’s a rather formal way. Another less formal way is expense account, which is maybe you give them three—or however many pennies you want—three tokens. And they pass those tokens in each time they talk. So the assertive ones are going to spend their pennies very quickly. [LAUGHTER] And the less assertives are going to then have a chance to spend their pennies. And when everybody has spent their pennies, you all get your pennies back and now you can start again. But again, that’s a way to give people a chance to speak. But people can choose when they want to speak, rather than having this turn where it’s coming around. And it’s very set. Larry Michaelsen’s Team-Based Learning is much more complex. Larry started this in a lecture hall. And so he has basically an IRAT and GRAT. And the IRAT is an Individual Readiness Assessment Test and that GRAT is a Group Readiness Assessment Test. So he has them take an IRAT first, and as an individual give their answers, turn that in—that’s a grade—and then he has them do the GRAT. So as a group, they now turn in their group grade. And he also uses the scratch off cards, the if/at cards, that sort of thing. Initially, the assertive ones—again—are giving the answers. But as they discover that they don’t have the right answers all the time, then the quieter ones in the group suddenly become more important to the group because their grade is dependent on this. So they’ll start asking, “What did you get for this? You seem to get A’s all the time. Can you please help us?” And so that’s his method of doing that inclusivity.

John: And in that approach though, I believe he recommends persistent teams over the course of the semester, so that they develop that sort of team dynamic.

Dakin: Yeah, very much so.

John: Are there any other small group activities that you like to use?

Dakin: I have a lot of protocols that I’ve gathered over the years—probably got 40, 50 protocols—and the ones that I select are the ones that are low risk. So I’ve kind of classed them as low risk, medium risk, and high risk in terms of how much risk does the student feel when they’re in the classroom doing these… and I like low-risk things which are usually small group where they’re by themselves and they’re talking, and it’s not in front of the faculty member, and they are not having to answer to the whole class in front of a large group. So, some of the pieces I like are jigsaw—which comes out of Elliot Aronson’s work in the 70s—the idea that you break up an assignment into five pieces, and each of the students in a group will do one of those five pieces, and then they will talk in class and share out what they’ve learned from each of those five pieces. So it’s a great way to synthesize a lot of data that maybe you don’t want all students doing. So when I was teaching my Middle Eastern history class, each student was responsible for a different country and they had to do a lot of reading on that country. But if I had all the students do all that reading, it would have been far too much. So instead, I can have these various countries sit at a table and then have a conversation, and the student representing Israel can talk about Israel’s point of view, and the student doing Jordan can talk about Jordan’s point of view, that sort of thing. So that’s one method I love and I even do a double jigsaw. But I only do a double jigsaw maybe twice a semester and they’re at moments where there’s so much content, that there’s absolutely no way we can cover it. And the best example of that is U.S. history survey, first day, which is the dawn of time to 1492, which I think is horribly disrespectful to everybody that was in North America before 1492. So we do a double jigsaw, which is where you have a jigsaw that creates experts at each table and then those experts then are now experts in five different topics. And those people then go off to create super jigsaws. And that works well, but it takes a lot of time. The other one I love is role playing. Role playing… just because it’s my age… I grew up with role playing, but I’ve done a lot of different styles of role play. The one that I think I use the most in first-year history is Articles of Confederation. Everybody takes a representative to the Confederation and talks about what it was that person was like, and why they voted the way they did, and what were their goals. And then we skip ahead to the Constitutional Convention and we talk about who’s still there, who’s not there, why are they not there? If they’re still there, do they still have the same opinions? Are they still voting the same way? Why are the results different at the Constitutional Convention as opposed to with the Articles of Confederation? So that’s a good one. Other classics are the Oregon Trail… everybody loves the Oregon Trail. And unlike the computer game, what you learn is not many people died on the Oregon Trail. People who died most were the Native Americans who are along the trail, everybody else pretty much made where they were going, but that had to do with who those people were. The other one we did was Cuban Missile Crisis. Did the Cuban Missile Crisis and role played the various operatives in the Cuban Missile Crisis. And then next week we talked about Watergate and again looked at those same operatives and where they were now in the Watergate plumbers. So that was also useful …those kind of things. There’s lots of different ways to use that though. I’ve seen people use that with theorists. So in psychology, different theorists are represented by the students and they argue their different theories and try to figure out how these things go together. Role playing is obviously one I love.

Rebecca: Can you talk a little bit about how you set the role play up? You mentioned what some of the topics are and when you use it, but can you talk a little bit about the logistics of setting that up and how you have students prepare for that?

Dakin: And those vary a lot. When we’re doing the Articles of Confederation, I just have a list of representatives and I asked them to choose one and then we go from there. So my prep on that is zero, they are the ones responsible for that prep. On some of the others, there’s a lot of prep. With Oregon Trail I worked out, I took a K-12 game that had been done for Oregon Trail, and then I made it much more complex and they had to purchase their gear so I had a full list of gear, I had a list of where they were going to stop, how they were going to stop, and what the mortality rates were. So I basically created this whole game around it and then they played through that. With the Watergate and Cuban Missile Crisis, it was kind of halfway between there. I made cards with each of the people they would role play and on the back—like a Clue card—it tells you who this person is and what their role is and then I gave those to them. And then from there, they again generated most of the data.

John: You mentioned you have these organized by levels of risk. How would you recommend using the different levels? Would it make sense to start with low-risk activities, and then as more trust is built, build the higher ones, or would that be affected by the level of the class that you’re teaching, whether it’s introductory or more advanced?

Dakin: Yeah, it definitely depends upon the purpose of the class. I tend to teach introductory classes so I build a lot of trust, I use a lot of low-risk pieces, and I’ll move to maybe medium risk by the middle of the semester—or maybe I won’t ever use medium risk, it’ll all be low risk—because I’m trying to get them to get used to college and figure out what that’s involved in. If I am teaching a class that’s kind of a gateway or portal class that’s going to lead on and it’s supposed to cull out people, it’s supposed to find out who the best people are, then maybe it makes sense to start doing some of those high-risk pieces, but I probably wouldn’t do that until at least the third year. Build a lot of confidence, a lot of trust, and there’s a lot of learning that has to take place before that, before you get to that point. Traditionally of course, if you look at law school or medical school, they have a lot a lot of high-risk protocols, because there’s a lot at stake and people have to do well. And I remember when I started working with the med school that I read academic medicine, and one of the articles was, “We should abuse our students less.” [LAUGHTER] Not, “We shouldn’t abuse our students,” but, “We should abuse them less.” So, that kind of gave you a sense of what we were dealing with.

Rebecca: Can you talk a little bit about some of the small-group techniques that you use in online environments?

Dakin: Sure. Online environments—actually, the small groups—I usually leave that up to the groups themselves. So if there are groups of students that I’m having work on particular topics, I’ll have those students organize, say, a synchronous conference where they get together on Zoom, and they talk about these things. So they get to pick whatever tool they want—I’ll recommend Zoom because it’s free and you can have up to 40 minutes free, and you can record it. It’s very simple—but they will do that work by themselves. If we’re dealing with, say, a discussion forum, what I’ll generally deal with is ways that students can interact kind of more of at a large-group level, because there’s really no need for a small group when you’re on the discussion forum. But what I do want to do is get rid of the old “post one, reply two” because after you’ve taken two or three online courses, you’re pretty bored with that protocol. And so giving them different ways to think about it and moving the jigsaw into it, moving a debate into it, moving role playing into it, those are all really useful.

Rebecca: Can you pick one of those more complex ways of using a discussion board and talk through how you set that up or organize that?

Dakin: Sure. I think the main thing I do is really—it’s not so much about the organization of the board when I’m doing it—but building student activity. I do a big sales job in terms of talking about what is the value you get from an online course. Now, if it’s just teacher to student, I think that’s a really limited amount of value because there’s a lot of good books out, you can read, you can train, you can look at YouTube, there’s all sorts of great ways to learn. But a real value from an online course for me is who is in that class with you and finding out what their strengths are and what they can bring to it, and that’s where a large part of the education comes. So I don’t use this “post one, reply two” but I do want them to make sure that they are responding weekly to their colleagues, but at a level they feel is appropriate. So don’t say something if you think it’s totally pointless. But if you have a comment and you feel it’s worthwhile, say it, because we need to hear it. That’s the largest part of this. In terms of the organization, the only pieces I’ve done in terms of organization have been very slight. So, with an assignment, you turn in your first post Wednesday, and you turn in your final post on the piece, on Friday, Saturday, or Sunday. So the initial post is what your response is—out of a think-pair-share, this would be the think part—that you’re doing your initial writing. And then the second part of the week is simply the sharing part. So now you’re responding to those students. Now, if you’re in Canvas, you can do this with setting up the initial one as your due date, and the second piece as your until date, so you can do it within one assignment. Unfortunately with Blackboard, you can’t do that, you have to have two separate assignments. But that’s the only real difference.

John: What about larger classes? What techniques do you recommend there?

Dakin: In large classes, I often talk about Eric Mazur and his peer instruction, simply because I can send them to the videos he’s got on YouTube, and he’s got a lot of videos there. He’s got a lot of publications. So that’s great, I have a lot of resources I can send them to that they can start working on. But Eric’s technique is largely one that applies to an interactive lecture. It’s not really small-group work per se, it’s a way to maintain activity by the students and also makes sure that you’re getting feedback back on what they’re understanding. But since Eric uses multiple-choice questions, he’s really not getting beyond that understanding- or application-level question. So, the issue really with small-group work and large classes is really not about the size of the class, it’s about the furniture in the class. So, you can do small-group work with a very large group as long as you’ve got movable tables and chairs. So, I did this at the Teaching Professor Conference a few years ago. I had 110 people in the room and we did value line, and we did jigsaw, and we did all these different things, and it’s very easy to do as long as you’ve got the furniture that allows you to do it. The hard part about a “large” class is really it’s about the lecture hall and the furniture in it. So if you’ve got furniture that’s fixed, if you’ve got a table that’s fixed, if you’ve got chairs that are fixed, it’s hard to have more than two to three people working together at any one time because they can’t turn around—they can’t do anything else. Also, since you’re in a large lecture hall, there’s a lot of noise. So again, you don’t want to get more than three people because you won’t be able to hear the others. If in a lecture where students can turn around, then you can have a larger group of say four to six. So you have two to three in the front row, two or three in the back row, and they’re talking together in that small group. And I’ve seen small-group work in lecture halls with as many as 160 people, so I know it works. It does take some effort in terms of arranging it. Usually they don’t do random small groups every day—because that would be chaos—but they do long-term teamwork and the faculty member who did this was David Pace at Indiana University—he was very good at this, he’s the one who taught me how to do much of this—he does a pretest, he organizes the students, he puts them into these long-term teams, and then in those teams in the lecture hall, they have the seating arrangement where they’re sitting. And then when he wants to do small-group work, he’ll do his lecture, and he’ll do small-group work, then he’ll do a debrief, same sort of pattern.

Rebecca: What do you find your role is, as an instructor during small-group work? You want to put a lot of the onus on the students, but what’s your role during all of that and how does that scale up to a big class?

Dakin: My role is—as an instructor small-group work—is essentially challenging, adding to, and supporting. Making sure that they know they’re encouraged and they’re doing a great job and going around doing that sort of thing as I walk the room. A lot of the work I do is really the preparation. Making sure that those things are well thought out, that I have a lot of idea of which directions they can go, and to, after the class, make sure I’ve done my reflection: I’ve written down all the weird places they went so that I know that those are possibilities and I can be ready for those, or maybe I just work towards those. Maybe those were better ideas than the ones I came up with—which is actually one of the big advantages of small-group work because you are paying more attention to the students, the students have a bigger role in the class, and your life isn’t as boring. If I was doing the same lecture 20, 30 years later, I would be bored to tears. But as it is, since I’m using these, every semester is different because every group of students is different. So my life is constantly interesting. And it’s almost like doing improv, really, in a way. You have to be a little brave about it, you give them opportunities, but there’s a lot of trust, you trust the students are there to help you. And everything goes well, even with apathetic classes that when I’ve walked in, the class has just been dead, they don’t want to do anything. After a week or two of this, they start getting into it. And by the end of the semester, they’re the same as every other class, and it’s going very well. So it’s highly enjoyable. So I think that’s it, make the class fun, get them to trust and encourage them to do their best work.

John: And in large classes if you have TAs, you could have TAs going around and doing the same thing, just so that you get more of the room covered.

Dakin: That, or if you’ve got a tight space to deal with, you could also have a backchannel going, so people in the groups are reporting out and the TAs are looking at that backchannel through Twitter or something else and kind of getting those ideas and feeding those back to either the students or the instructor.

John: Earlier you mentioned that light grading be used. Could you elaborate on that a little bit?

Dakin: I think that’s the hardest thing for faculty to do is light grading. Faculty members really want to mark everything. If they see something wrong, they will mark it. And I must admit, myself, when I’m posting to Facebook and somebody writes something and spells it incorrectly, I have to respond. It’s annoying, but I have to do it, and it’s the same way with grading. People will try to grade everything and they will eat up their lives giving these huge responses back that the students really aren’t going to listen to. Nobody has time to make all those corrections. So the smartest guy I ever saw was Bob Ferrell, who was a professor of history. And Bob was highly published—he had 50 plus books—and he still had a line out the door of students that he talked to every day and that was highly admirable as far as I was concerned. And so I wanted to find out how he did this, and what he did is… I took a readings class with him and I handed in a paper a week, and we worked through that. And every paper, the first time he got it, he marked it up pretty heavily to show, “You need to work on your grammar and I’m watching you.” But after that, every week, it was three things. He’d mark… circle one, flip a couple pages, circle another, flip a couple pages, circle another, “There you go,” out the door, you’re done. And so for me it felt like, “Oh, I only have three things to change. This is great, I’m really close to getting that top grade.” And next week it would be another three. And next week it would be another three, and so on. So, he was doing light grading, he was giving me feedback—feedback that was useful to me—feedback that was moderately challenging. I didn’t feel at sea, I felt I could do it. Great. And so I would do it. And that’s the way I come to this. The way I implement it is, say if I’m in a freshman class, I will have the students writing say 1000 words response every week, which for a freshman class seems like a lot, but I want them to work and I want to hear their voice. I will tell them not to use any quotations, I want to hear their voice, I don’t want to hear somebody else’s. I want to hear them thinking, and if they don’t agree with the text, argue with it—that’s fine. If you don’t agree with me, argue. That’s what you should be doing. You’re trying to construct your ability to speak and write. So, when they do that, they then turn these pieces in, and I grade them but I grade them lightly, which means I’ve got now 40,000 word essays I’m supposed to be grading, that will take me about 40 minutes. I spend about a minute on each. I just kind of flip through it, I can tell if somebody’s done the reading or not, I can tell if there’s a major issue or not, and then I write down my responses but I don’t give them to the students. I just give the students grades. And when I get to the class, I’ll do a group grade. So at the beginning of the class, I will then do a couple things. One, I will celebrate some people, and I’ll talk about that in a minute, but I also make corrections. I’ll do grammatical corrections, will say “Here’s the five grammatical errors of the week,” and by about mid semester, I’m still showing some of those grammatical errors up on the board and the girl who’s done it says, “Oh my God, it’s me again!” So they get it and they’re trying to reduce them and that’s fine. The other thing is I talk about misconceptions. Say somebody has a misconception about a particular piece, I’ll say, “A couple people had a misconception about X.” Now it’s not a couple people, it’s Joe. It’s always Joe. Joe’s in the back room, Joe never does the reading, Joe’s having trouble. But Joe knows he’s not doing the reading and Joe knows he’s not getting a good grade, he doesn’t need to have his name called out in front of the class. So I say, “A couple people had this issue.” I talk about that and address it. And then the next part, the celebration. So, in order to make them feel better about what just happened, I then say “Now then, I wanted to talk about some of the great things that were done this week. So first of all, Jenny had this fantastic response, it was just so meaningful. I want to share it with you, because I think it’s really worthwhile listening to. And Bob said something that no one has ever said in this class before and so I think it’s important to address that.” And then maybe I talked about Jim, “Jim really did a very deep reading of the text, he brought up some serious issues that I did not bring up myself, and I think we need to explore those today. So that’s part of our discussion today, it will be based on what Jim has talked about.” So that’s the celebration, but every week, it’s a different three. It’s never the same three, it’s never always the A students. Over the course of semester, I find a way to celebrate each and every student in that class, including Joe. And Joe is hard. Joe’s really hard. So I’m always watching every week to see what Joe is talking about and if Joe says something good it’s like, “Yes, I can now celebrate Joe! Good. Check.” I’m celebrating. And that’s the way light grading works for me, it allows me to spend more time interacting with students, less time interacting with their work.

Rebecca: I’m sure we can all take advice on reducing grading, right? [LAUGHTER]

John: When we talk to faculty about using group discussions in class, one thing they often raise is a question of when students are teaching each other—in general with peer discussion or peer activities—there’s a concern that perhaps it may reinforce misinformation. How can you be sure that that doesn’t happen with small-group activities?

Dakin: Well, I don’t think you can be sure it doesn’t happen, but you can certainly set up a system to check for that and make sure it’s not happening or that if it’s happening, you’re correcting it. So the way to do that would be use some ungraded assessments, those classroom assessment techniques from Angelo and Cross. In the large group during the debriefing, some of those may come to light again, and you can then use lecture to correct those misconceptions once they become apparent. I agree that small-group work builds confidence, but it can really be like the blind leading the blind sometimes, especially if the students haven’t been doing the preparatory homework, and especially if the groups aren’t randomized.

John: And if you do that sort of pretesting, where you’re getting the initial feedback, you can tell what those misperceptions are, so that you can be prepared to address them during the class, which should help reduce that issue.

Dakin: Sometimes… yeah. I have to know which questions to ask, and often I don’t. So it’s that ungraded assessment where they toss back an answer that completely takes me by surprise. Oh, I am so surprised. Now I know what your misconception is. But I couldn’t have guessed at it.

Rebecca: I think that’s important to remember too, that [when] you’ve been teaching for a long time, the misperceptions that you might have come across five years ago are really different than the misperceptions that you might experience this year, because the experiences of our students change and the group of students change, and all of that influences prior knowledge and prior experience that influences how they might interpret material.

Dakin: That is so important. Over the 30 some odd years I’ve been teaching, my students have changed a lot, not only in their content knowledge and what they know and what they’ve experienced, but also how they think and how they behave. And again, that’s the strength of using small group work, because you get to see how they think and how they behave. And they’re not just sitting there in rows in front of you and you imagine that’s the same class you were teaching in 1987. [LAUGHTER]

Rebecca: So we always wrap up by asking, what’s next?

Dakin: I’m collecting all these protocols for my own use and also to help others, so it’d be great if people listening to this podcast could send me some new ideas, send me some more protocols, so I can share those out. The way to do that is to send them at teaching@cortland.edu. That’s our email address. In the meantime, we’ve got a new Institute for College Teaching down here, we finished up a faculty needs survey. We’ve got our advisory committee in place, and we’re just about to start setting up priorities for next year. So, there’s a lot happening, I just don’t know what it is yet.

John: Because you’ve just taken over that position fairly recently, right?

Dakin: Two months ago.

Rebecca: Oh, the surprises you might find, right? [LAUGHTER]

Dakin: I have been very pleasantly surprised so far. I have found a lot of really skilled and dedicated faculty, and I’ve just really been enjoying talking to them. I know I enjoy this because it’s a challenge, and I love a challenge. And they are so well-educated already. It makes me work very hard.

Rebecca: Which means you’ll never be bored, right?

Dakin: Exactly, and that that’s why it’s so important to me.

Rebecca: Well thank you so much for joining us today. It’s been really interesting, and I think will help faculty as they plan for their next teaching adventure.

John: Thank you and we will share some of the resources that you’ve provided in the show notes as well.

Dakin: Thank you. I’ve really enjoyed our conversation. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

74. Uncoverage

Posted in active learning, higher education, learning, mental models, pedagogy, teaching, teaching centers, uncoverage
March 27, 2019

Introductory textbooks in most college disciplines tend to become thicker over time as new topics are steadily added while old topics remain. Classes designed to “cover” all of these topics necessarily sacrifice depth of coverage. In this episode, Dr. David Voelker joins us to examine how some faculty are changing their focus from “coverage” to providing students with an opportunity to actively engage in the discipline and uncover its power to help explain their world.

David is an Associate Professor of History at the University of Wisconsin Green Bay. He is also the Co-Director of the Wisconsin Teaching Fellows and Scholars Program and the co-author with Joel Sipress of “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” which was published in the Journal of American History in March 2011.

Show Notes

  • Voelker, D. J. (2008). Assessing student understanding in introductory courses: A sample strategy. The History Teacher, 41(4), 505-518
  • Sipress, J. M., & Voelker, D. J. (2009). From learning history to doing history. Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind, 19-35.
  • Sipress, J. M., & Voelker, D. J. (2011). The end of the history survey course: The rise and fall of the coverage model. The Journal of American History, 97(4), 1050-1066.
  • Voelker, D. J., & Armstrong, A. (2013). Designing a question-driven US history course. OAH Magazine of History, 27(3), 19-24.
  • Wiggins, G., Wiggins, G. P., & McTighe, J. (2005). Understanding by design. Ascd.
  • Gordon Wood – Author and History Professor
  • Gary Nash – American historian
  • Angela Bauer – Professor and Chair of Biology at High Point University
  • Ryan Martin – Psychology Professor and Associate Dean of Recruitment, Outreach, and Communications at University of Wisconsin at Green Bay
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • 61. A Motivational Syllabus. Tea for Teaching podcast (with Christine Harrington)
  • Christine Harrington – Associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing
  • Lendol Calder – Professor of History at Augustana College
  • University of Wisconsin’s Teaching Fellows and Scholars Program
  • UW Faculty College
  • Spring Conference on Teaching and Learning
  • Regan Gurung – Professor of Human Development and Psychology at UW Green Bay
  • Scholarship of Teaching and Learning
  • Gurung, R. A., Chick, N. L., & Haynie, A. (2009). Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind. Stylus Publishing, LLC.
  • Ciccone, A. A. (2012). Exploring more signature pedagogies: Approaches to teaching disciplinary habits of mind. Stylus Publishing, LLC..
  • Gurung, R. A., & Voelker, D. J. (Eds. Gurung, R. A., & Landrum, R. E. (2013). Assessment and the scholarship of teaching and learning..). (2017). Big Picture Pedagogy: Finding Interdisciplinary Solutions to Common Learning Problems: New Directions for Teaching and Learning, Number 151. John Wiley & Sons.
  • 54. SOTL. Tea for Teaching podcast
  • International Society for the Scholarship of Teaching and Learning
  • American Historical Association
  • Tuning Project

Transcript

John: Introductory textbooks in most college disciplines tend to become thicker over time as new topics are steadily added while old topics remain. Classes designed to “cover” all of these topics necessarily sacrifice depth of coverage. In this episode, we examine how some faculty are changing their focus from “coverage” to providing students with an opportunity to actively engage in the discipline and uncover its power to help explain their world.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Dr. David J. Voelker, an Associate Professor of History at the University of Wisconsin Green Bay. He is also the Co-Director of the Wisconsin Teaching Fellows and Scholars Program. He is the co-author with Joel Sipress of “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” which was published in the Journal of American History in March 2011. Welcome, David.

David: It’s nice to join you.

Rebecca: Today’s teas are…

John:…are you drinking tea?

David: Oh of course, always. Yes. I’m having a Moroccan Mint Green tea.

Rebecca: So you’re a tea drinker so you can come anytime. [LAUGHTER]

John: I’m drinking Black Raspberry Green tea.

David: Oh that sounds nice.

Rebecca: And I have Bombay Chai today.

John: We’ve invited you here primarily to talk about your work on redesigning the introductory history course. Could you tell us a little bit about the problems that you observed in the traditional approach to teaching the survey course?

David: I started teaching history at UW-Green Bay in 2003 and I was trying to create a class that would be very engaging for students. We would look at primary documents, I would really encourage them to think critically about the materials, and so forth. But I was still doing a lot of coverage, I went through the standard list of topics that you would follow to teach American History and I had a textbook. Well, after doing that a couple of years, I had two really wonderful students in my office. These were very engaged, hardworking, curious students, and they were talking to me about an upcoming exam and I found that they were just trying to memorize the content as I had taught it. Now, they were clearly capable of so much more, and they were doing more than that. But when it came time to the hard work they were putting in to study, they saw it as a memorization exercise. And I realized, “Wow, if these are the most engaged students in the class, then something’s not going the way I want it to go,” because that’s where their attention was…on memorization. That’s when I began a long journey to rebuild the way I was teaching—or reconceived how I was teaching—the intro history course. I ended up actually developing not only a new way of teaching the course, but entirely new ways of assessing student learning. So that when students were studying, when they were really putting in the effort and so forth, they would be focusing on the things that I really wanted them to learn, which was: how to interpret a primary source, how to find the argument made by another historian, how to put all that together into historical analysis. So that’s a quick summary of how I got on the path of reworking my introductory history course.

John: I don’t think this is unique to history. I see the same thing in economics. And in general, when students come to college, they often learn somewhere along the way that memorizing lists of things is what they’re expected to be able to do in college, and what you’re addressing is a problem that we see in all of our disciplines to some extent. So what would you recommend as an alternative? How have you approached this issue, or how have you tried to resolve this?

David: Well, first I should say I was really inspired by the book Understanding by Design. And I just looked at my copy and apparently I acquired that and started reading it in 2006. So it’s been a while ago, and it’s quite marked up. But the basic premise there is, instead of starting with, “Here’s a list of all the things I want to cover,” or “Here’s a list of all the things I want students to read,” you really start with “What is it that I want students to be able to do as a result of taking this class?” Now, students taking an intro history class, I’m not going to say, “Well, I want them to be able to build a house,” or something like that. What they’re going to be able to do is think in particular ways. You know, not necessarily the kind of skills that you would be able to observe in some product other than their thought. So I really tried to develop a class that would help students develop their ability to think historically. Now I realize they’re doing that for an intro course at a beginner kind of level, but the emphasis is still on historical thinking.

Rebecca: When you’re talking about the ways that we want students to think and work, we often get frustrated as faculty when students aren’t doing that but we don’t always take the time to articulate that, so I think it’s important to highlight that as a good starting point.

David: One thing I tried to do here is… well, just to be really intentional. What is it that I want them to learn? And then I think the crux of it is to make sure that I’m actually assessing them on the things that I really want them to learn. And I would guess that the overwhelming majority of college level history instructors really do want their students to come away with the ability to think historically, or at least to have some growth in that area with the ability to know what to do with a primary source, you know, historical documents, or perhaps an artifact or something like that. So I would say our hearts are in the right place but the question is, how do we actually get there? How do we align our highest learning outcomes with the actual assessments that we’re doing? In Understanding by Design—I don’t think I mentioned the authors earlier but Grant Wiggins and Jay McTighe—they really put a lot of emphasis on assessment. Now I know in higher education there are folks who are wary of assessment and they’re thinking of institutional assessment, which I would say is important, but here we’re thinking about how are you grading the students? How do you know what they’re learning? So I decided to rethink my assessment and make sure that my exams were really requiring students to do some historical thinking on the spot, also drawing on and demonstrating that they have knowledge about the history we’ve been studying. So rather than seeing if they memorize some content, and you know they can somehow show that on an objective exam, what I do is on exams, they encounter a historical claim. So a really simple one would be “Christopher Columbus discovered a new world in 1492,” and I ask them to think about that critically and to think about that the way a historian thinks about it. And I asked them to argue both for and against the statement like that. So they’re arguing, they’re not simply regurgitating information. That means they’re using evidence and, in order to use evidence, you have to have content knowledge, right? You have to have knowledge of historical context, you have to know some of the sources. And so I see all that stuff that we “cover,” so to speak, really is the raw materials that students are working with as they develop their ability to think historically.

John: You’ve mentioned using in general, a backwards design approach, and what you’ve just described is part of it, where you start with the ultimate learning objective and then you design assessments that would measure that. How do you prepare students to reach that level?

David: Well, I guess that’s another big question, right? One thing, I try to be very transparent about this. I mean it’s interesting how sometimes in higher education we’re not very transparent. And that means that I talk with students about what it means to think historically, what that looks like. I try to model that for them, of course, but I do it in a way that’s explicit. I mean, here again, I would say most history faculty across the nation model historical thinking all the time, but they may not be very explicit about what they’re doing. So I tr y to be explicit about what it means to think historically and then also to give students a chance to practice that in class. So it happens my classes are 80 minutes long and well, I’m very grateful that I’m not lecturing for 80 minutes straight. I would just find that very difficult to believe I’m engaging anybody… I’m covering stuff for 80 minutes. So, I really try to break the class session up and have students not only talking about primary documents—which is pretty common for history classes—but we also look at different perspectives from historians. Now, I don’t introduce them to the vast historiography on any topic that we’re looking at, but I do make sure that they know about at least two different perspectives on major issues of the colonization of North America, or the American Revolution, or the coming of the American Civil War. So they can see that there’s actually a debate out there, rather than assuming that, “Well, history is pretty cut and dry. Like, we know what happened, we’re just describing it.” So, I really try to introduce to them the ways in which the study of history is an ongoing dialogue among historians, that we’re always turning up new sources, and we have new ways of thinking about old sources. And so I give them examples of how historical thinking is working in the field and really ask them to wrestle with some big questions. Again, it’s at an introductory level, so they’re not going off and reading everything that Gordon Wood wrote about the American Revolution and then comparing that to Gary Nash’s interpretation or something like that, but they are getting little snippets of that sort of thing so they can really get a sense of the debate and they talk about that in class. I don’t actually hold formal debates but I’ll have small groups discussing a particular issue and then I make a sort of matrix on the whiteboard and they go up to the whiteboard and sort of vote for where they think this historian stands, and where they think a different historian stands on the issue, and then where their group is, and so they’re really playing around with those different positions. And then, of course, on the exams, they’re actually making their own individual arguments using the primary sources and the secondary sources.

Rebecca: One of the things that you said was “modeling, but making that explicit.” Could you talk a little bit about how you shift from just modeling modeling explicitly or being explicit about doing that?

David: Yeah, sure. Well, one thing I do is I have a kind of graphic that I show that I created that tries to describe in a basic way what historical thinking looks like because I do think a lot of students come in with a really simplistic understanding of history. Again, this notion that it’s pretty cut and dry, it’s a description of what happened in the past. And so I try to help them understand both, through my own comments but then through practice that, “Okay, yeah, we have descriptions about the past, but those are based on evidence that has to be interpreted.” And those descriptions, we assemble those into a narrative and built into those narratives is an actual explanation and analysis of what happened. So we’re really thinking about why things turned out the way they did, how different things are interconnected, and then finally—and maybe the most importantly—is significance. Historians are also wrestling with the significance of the past, different events, and developments. Not only the significance in a narrow historical sense like, “Okay, this event was significant because it led to another event,” but also a longer term significance, and then I think even a kind of moral significance. What can we learn from the past? How can we use the past to understand American national identity? Those sorts of questions. So we really kind of build up, there’s multiple levels of complexity there. And I try to get my students at least beginning to climb that ladder to some of those higher levels, even if they’re just doing it in a really rudimentary way. But then I hope they can really see the value of history that goes beyond just kind of rote recollection of something that happened in the past.

John: Do students sometimes resist moving into this more appropriate view of history? They’ve learned through elementary and secondary schools what history is. How do you break their expectations and get them into this more active role?

David: I’m kind of smiling, or maybe even grimacing right now, because I usually am able to win students over. I mean, there’s so much about the past that’s exciting, that’s interesting, that’s unexpected. When we start to dig deeply into some of these topics, they’re interested and they’re curious and sometimes they’re also pretty frustrated or even angry because they feel like they’ve been misled in the past. In other words, that maybe the basic information they had from high school—of course the quality of that instruction varies—but I do have a lot of students who come in who are upset. They feel like they’ve been misled. And I want to rush to add that I have many students who have had excellent teachers in the past and are already really interested in history in part for that reason. So I think there’s a lot for students to be curious about. Occasionally I do run into a student who is very resistant. Most students find that, “Hey, turns out this is a lot more interesting and engaging,” than the history they had been exposed to in the past, or it’s deepening and complicating the pretty good understanding that they already had.

Rebecca: Do you find that they’re surprised by the first assessment or exam because they have these prior expectations and then your methodology in your class is different from their past experience?

David: Well that’s something that I anticipated and so we actually have a practice exam, so to speak, very early in the semester. In fact, that example I gave a moment ago of Christopher Columbus discovered a new world and 1492 is a sort of maybe second week of class practice exam prompt. And so I have them do that outside of class and then bring it in and we essentially go over that together. They share their thoughts with their classmates, we talk it over, I give them a number of examples. So for a student who’s coming to class and whose keeping up, there aren’t really going to be any big surprises and I give them some samples of what the exams are going to look like from other classes. I used to teach a lot more modern American history and so I can easily just share some sample exams that students told me it was fine if I shared and I think that’s really helpful, especially if there’s a big change in expectations, then I think to be fair to students, you should really be clear with them about how the expectations might be different than what they’ve encountered in the past. And a lot of students actually like the assessments I’m using. I think they find them more interesting. They’re perhaps a little harder to study for in the sense that it’s a little less clear cut but they have a pretty good idea of what the major topics are and I give them some suggestions for what they should be reviewing in terms of sources. So they really come in thinking, “Well, I’m going to need to refer to least a couple of these primary documents and the perspective or argument of this historian.” We practice all that in class because again, that’s what I want them to be able to do. So I make sure that we spend a lot of class time essentially practicing the very thing that they’re going to be assessed on.

John: If someone were to come into your class, what would it typically look like? Are the students broken up into really small groups? Bigger groups? How do you have them engage with the material that way?

David: Well I typically have about 65 students in my introductory US history class. I have another class that I teach in a similar fashion that’s actually a writing e mphasis class though it’s a little smaller at 45. So an intro level class and environmental history. But in both of those classes, we will typically start out with the whole group and I make some introductory comments. Sometimes I give a brief lecture on background context. I mean, historians really do value context. This is part of how you make sense of the primary documents and so forth. So I do all of that and you would expect to see that in a history classroom, but then before too long, I’m giving the students some kind of a series of questions and they’re working with primary documents or they’re working with some short essays by historians and talking those through in their small groups and then we have various ways of reporting out. Like I said, sometimes I will send them to the board so they’re kind of recording their analysis on the whiteboard. Other times it’s just more like I’ll choose particular groups to respond and sometimes I just open it up. So, I think that’s all pretty typical except that I’m constantly going back and forth from maybe a brief lecture to small group discussions to a large group discussion. I suspect there’s more back and forth than you would typically see in a more traditional coverage oriented history classroom. Again, many instructors do use primary documents and they would have students talking about them. I think maybe where I’m going further is just that they’re going to have to write about those sources on an exam.

Rebecca: I think sometimes demonstrating that you really value a particular practice by spending so much class time on it really helps students understand how that’s a part of the discipline or a way a discipline works.

David: Yeah I think so, and many students really seem to appreciate having quite a bit of discussion time. I mean they find it productive, they have a clear task, they have something they’re working with. So at the end of the semester I do have a lot of students who say, “Wow that was so valuable to hear different perspectives.” And I think especially for first-year students, many of their classes are very large, many of their classes are lecture based, and so they don’t actually have very many opportunities to talk with other students in a meaningful way about the course material. And I realized that maybe at a small liberal arts college it would be a whole different story, but I’m at a regional comprehensive public university and I know that a lot of my students are in that kind of situation.

Rebecca: I think sometimes there’s a mismatch between the assessments that we assign and what our real objectives are…because it’s easier to assess other things than what we really want to know, and that tends to lead faculty to be resistant because of workload concerns. Can you talk a little bit about the grading and how you might manage that?

David: I have to agree with everything you’re saying there. That’s a real conundrum. The most meaningful assessments are also usually the most labor intensive and so I’ve had to be very deliberate in my case about what compromises I’m going to make. And one thing that I’ve had to do with the for and against essays, students write maybe six of these essays during the semester and they’re fairly brief. I really emphasize with the students that there’s not any room for BS in these essays. So there aren’t a lot of preliminaries and so forth, they just dive right in and make their arguments for and against. So that’s one thing that I’m doing. It’s not a formal essay because I don’t have time to go over all that extra content when I’m grading. I think the other thing that I do is most of the feedback I give on those essays I do with the whole class. So I give a kind of collective feedback where I show some examples. I actually fabricate a kind of weak response [LAUGHTER] and put that up on the screen and ask the students to assess it and then I show them a stronger response and then we talk about that. So they’re getting some examples of work that could use improvement and that’s generally a composite. I mean, I don’t ever show bad student work. But then often the strong examples…I’ve asked the students, “Hey can I share this with the class?” and we’ll take a look at that as well. So what I’m getting at here is one way I save time is a lot of the feedback I’m giving is collective feedback. If I were writing extensive comments on every one of these essays for 65 students, it just wouldn’t be feasible. I would have to do something else. So I’ve decided that’s a compromise that’s worth it.

John: When they’re looking at someone else’s work, the work that you fabricated, it’s probably a lot easier for them to recognize problems than when they try to diagnose it in their own, because they’ve taken ownership of theirs and they become committed. But when you prompt them by giving them this type of thing, it’s a whole lot easier for them to see mistakes and recognize them and perhaps avoid them in their own work.

David: Yeah, I think so, or at least that’s my hope. And, you know, they talk with their classmates about it. So they’re getting multiple perspectives on the shortcomings of a weak historical interpretation.

Rebecca: Have you or your colleagues seen a difference in the upper-level classes that build upon this like introductory class where the move has gone from coverage to uncoverage? Have they been more successful at the upper level or do you have any evidence related to that?

David: That’s a really good question and I certainly don’t have systematically collected evidence to respond to that. Now I do see a lot of these students again in upper-level classes and the students I’ve worked with before do seem well prepared to jump into deeper conversations, whether that’s using more primary documents or more extensive secondary sources. So I find that they’re well prepared for upper-level classes as far as what I’m able to observe among my own students. Now, we do still have courses that are closer to the coverage model on my campus, for sure. I mean, different colleagues of mine, they all have their own approaches. And just to be clear, I would never say my approach is the one right way to do anything. It’s something that works for me and that I think really serves the learning outcomes that are important in my discipline and in my department, but also to me. But I think there’s a lot of different ways to get to that. I guess I would say I don’t have any regrets when it comes to upper level classes. I don’t feel like, “Well, these students aren’t well prepared because we didn’t cover everything at the intro level.” I am always reminding myself, “Well, just because I’ve covered something doesn’t mean that anyone has learned it.”

Rebecca: We’ve been talking about this a lot. We have these first-year signature classes on our campus that are for first-year students that may or may not be continuing in a particular discipline but are really meant to help students integrate into our college and acting like a college student, et cetera. So they don’t have as many coverage concerns in terms of topics that we’re historically thinking about. So we’ve been talking a lot about how that frees you up a lot to really focus on some of these ways of thinking. So we’ve been having this conversation a lot, we’ve had a couple of recent podcast episodes related to that but we’ve also had these conversations on campus that have really made me, and I think some of my colleagues, think about, what is it that we really want students to learn even at the upper levels? We think about, “Oh we got to make sure we get through the whole textbook,” or whatever but really at the end of the day, I think sometimes we place value on things that we don’t actually value.

David: Well what you said really resonates for me. I’ve been teaching a first-year seminar for I think five years now and that’s actually in the Environmental Humanities where there is no canon. There is no textbook. And I’ve certainly had to build on scholarship that’s out there. I mean, I’m not just creating this out of thin air, but I didn’t have to worry about the pressure to cover some specific body of material. Nobody was counting on me to be sure to cover these 10 things. And I learned a lot from teaching in that first-year seminar context where the pressure for coverage is largely eliminated and I think it’s made me a better teacher in all of my classes. I really do believe that, because I’ve taken some of that spirit into some more traditional history courses.

John: I think this is an issue that comes up in many disciplines. I know in economics there have been discussions for years. The economics textbooks—just like history textbooks—keep getting bigger, there’s new chapters added. Since I was in the history class, there’s been a whole lot of time included, [LAUGHTER] you know, in terms of the chronological study. But more generally, in economics, there’s a lot of discussion about how difficult it is to cover all of this and how little students seem to come out learning. But there’s a trade-off when sometimes an introductory economics class is the only thing required for perhaps all upper-level courses, so there is some pressure to make sure you cover it. It’s a very active dialogue right now and there’s many attempts to reach a consensus, but I think this is happening in many disciplines, isn’t it?

David: I think so. Several years ago, I was reading some workshops on this very topic for the UW system and when I was doing that, I interviewed some colleagues in other disciplines. I talked with Angela Bauer, who’s now at High Point University in biology and I talked with my colleague Brian Martin in psychology here at UW Green Bay. And what I found was that they were really trying to go down this road in disciplines where the knowledge is more structured. As you go through you really are building on stuff that you learned earlier, maybe in more linear fashion. And in the history major, we’re looking at the history of so many different times and places that no one person can master all of that. So you don’t have quite the same expectation. But in any case, what they were doing was starting out with some big questions and this is something that Wiggins and McTighe recommend in Understanding by Design as well. So you can structure a course around big questions. And those questions should be interesting to most students who are engaging in the discipline particularly if they want to go on. And I find that really helpful. So I do that in my history courses but I think that you can do that in any discipline. Like what are the big issues here? What are the big questions in the discipline? And to answer those questions, you’re going to be using specific disciplinary ways of thinking and disciplinary tools, so you can really start to align things there. Here’s kind of the raw material you’re working with, here are the skills and other kinds of thinking tools you’re going to be using, and here’s what you need to be able to do in the end. If you can really line those things up, I think students walk away with so much more than if they are simply approaching it as, “Okay, we’re covering this, I’m going to take a test on this, and then probably I can forget most of it.”

John: Ken Bain has written about this extensively in terms of picking those big goals and developing it in What the Best College Teachers Do.

David: Yeah.

John: And we just recently had a podcast with Christine Harrington, where she talked about that in terms of building a syllabus, starting with those big questions and building your course around that. It’s a really important topic to help build student motivation and interests and tie everything together.

David: I really admire the work of both Ken Bain and Christine Harrington. Back when I got started with all of this, in addition to Understanding by Design and history, Lendol Calder was doing a lot of work in this area—and he still is today—and I think he pulled this word out of Understanding by Design, but “uncoverage.” So what can we uncover about discipline and disciplinary thinking? Because if we’re using the coverage model, yeah, we’re covering lot of things, but we’re also covering up many of the fundamental aspects of the discipline. If what you have in front of you is a history textbook, it’s written in a single authoritative voice. This is what happened, this is how it was, this is how it is. There’s something really misleading about that, because that’s not how the discipline of history really works. Where did all that knowledge come from? What are the debates? And so forth. So I really liked that idea of uncoverage and I wanted to acknowledge both Understanding by Design and Lendol Calder for sharing that concept with me. I think it’s a really helpful way of thinking about what we should be doing even in our introductory courses, or maybe especially in our introductory courses. What can we do to uncover the key concepts and ways of thinking in our disciplines?

Rebecca: When you decided to shift from a coverage model to more of an uncoverage model, did you decide to just jump in with two feet or did you take more of an iterative approach? I’m sure other faculty who are interested in this idea would like some guidance on that.

David: Well, yeah, so do as I say, not as I did. I really jumped in with two feet and maybe even got in over my head for a while. And you know, that’s something you can do from time to time. I don’t regret doing that. At the time I was a participant in the Wisconsin Teaching Fellows and Scholars Program—which I’ve now been co-director of for six years. It’s actually my final year—but back then I was just getting my feet wet in the scholarship of teaching and learning. And I believe I had a one-course reassignment that year and so I had just a little bit more time to try to implement a pretty radical makeover for my intro history class and there have been many iterations since then, including some pretty significant changes because, of course, you always learn a lot when you make big changes in a classroom. There are always things you would do differently. That said, I think you really can just choose some part of the class and really start working on getting to some of that deeper learning and assessing in new ways. You can try out new ways to assess with new ways to get students working more directly toward your learning outcomes.

Rebecca: You just mentioned the Teaching Fellows and Scholars Program at your institution. Can you tell us a little bit about what that is?

David: This is actually a UW System program so it’s statewide and it’s been around for probably 30 years, although it has evolved quite a bit ―I think by the late 90s, it was taking a turn toward the scholarship of teaching and learning—so it’s a year-long professional development opportunity for instructors in the UW system and we have many campuses here across the state and each of those campuses sends a couple of representatives to participate in this program. We have something we call Faculty College that meets every year and that’s several days, usually in late May, so kind of right after the academic year has ended. You have these really dedicated teachers going right back at it, like, “Hey, we’re going to spend almost a week working on improving teaching and learning.” So that’s actually a broader program that the Wisconsin Teaching Fellows and Scholars participate in that and that’s sort of where we launch the cohort each year. Then a little later in the summer we have a week-long summer institute and the fellows and scholars are actually designing scholarship of teaching and learning projects at that point. So they’re figuring out what kind of research questions would they like to look at in their classes, what kind of evidence of student learning are they going to gather, and so forth. Then we come back together again in January and do some troubleshooting on the project which they’ve been working on through the fall semester and they typically are gathering more data in the spring semester or more evidence to analyze. And then they actually do poster presentations in April at a statewide teaching and learning conference. So it’s a pretty substantive program. A lot of folks do end up continuing to work on their project and ultimately publishing something. It’s not necessarily a requirement, but many of the participants do go on and remain active in the scholarship of teaching and learning. For me, I think it’s really important because it provides a community in the UW system around teaching and learning, and so while you’re actually working on your projects for that year, you have some support. You have a support network to help you figure out how to handle your data analysis or how to get through the IRB process or whatever it might be. But then that community continues on afterwards and so we’ve had quite a few…thinking of three or four different books…published on the scholarship of teaching and learning where many of the participants that were in the book project came out of the Wisconsin Teaching Fellows and Scholars program so there’s the signature pedagogies book, it’s called Exploring Signature Pedagogies and then Exploring More Signature Pedagogies, two wonderful books and Regan Gurung and I recently published collection of essays on Big Picture Pedagogy, and that’s taking a kind of interdisciplinary approach to the scholarship of teaching and learning. And most of the folks in there came out of the UW system, and many of them were former teaching fellows and scholars. So it’s a really interesting program that I think really enriches what we’re doing here in Wisconsin.

John: That sounds like a wonderful program. I wish we had more of that here.

Rebecca: Yeah. Do you have a little bit of advice for people who are interested in starting in the scholarship of teaching and learning? That’s one of the things that we’ve talked about quite a bit on this podcast and anytime we can get someone to provide a little insight into getting that started…

John: In fact, Regan was on a few months back.

David: It can be intimidating to get started in the scholarship of teaching and learning, especially if you’re coming from a discipline that doesn’t normally do research on or with living people. Like, you know, for me as a historian, my research subjects are typically dead and gone. [LAUGHTER] So it’s quite different to think about doing research on my student learning or with my students and their learning. So I think it’s helpful to start small and just to try to think of the question about student learning that you can answer using methodologies that you’re already pretty familiar with. So you don’t want to tackle a project that’s going to require you to learn how to do advanced statistical analysis if you don’t have experience with that already. So I think it’s important to realize there are a lot of different ways to collect and analyze evidence of student learning and if you continue to develop as a SOTL scholar you will explore more and more of those ways of gathering and analyzing evidence. But it’s completely fine to start out with something that you’re already comfortable with and familiar with and you’re really asking a question—or you can start with a simple question about—what are my students learning in this particular area? I know there’s a struggle here, I know there’s something difficult here, and I’m going to pay some extra attention. I’m going to look at this in a more systematic way than I normally would just in the regular course of grading some papers or exams.

John: In your paper on the rise and fall of the coverage model, I think you noted that there’s a growing emphasis on the scholarship of teaching and learning within history and that seems to be resulting in a growing network of scholars who are working in this. Could you tell us just a little bit about that?

David: Sure. Let’s see, back in 2006 a group came together called the International Society for the Scholarship of Teaching and Learning in History and I don’t know exactly what the membership or the participation in that group is but it’s a substantial group of history scholars who are really interested in promoting the scholarship of teaching and learning in history. Along with that, the American Historical Association now maybe for a decade or so has been working on something called the Tuning Project and that’s focusing on learning outcomes for history majors and that project has involved many, many people across the country in all different subfields of history thinking about what is it that we really expect and want history majors to learn and that transcends any particular content area. In other words, if you’re asking that for about history, and we’re thinking about historians coming in who are, not only United States historians, but looking at many different periods of time and in places around the world, you can’t just come up with, “Well here’s a list of twenty thousand facts [LAUGHTER] that we expect all history majors to know.” It was apparent that wasn’t going to work. And so what the Tuning Project has done, it’s allowed our discipline here in the United States to really think carefully about historical thinking and the basic skills and practices of being a historian and that at this point, it’s all very well spelled out. And history departments can then use that to improve the learning outcomes at their program level and you can take that down to the level of individual courses as well. So I think between the scholarship of teaching and learning and this assessment oriented Tuning Project, that we’ve had a real increase in interest and commitment here when it comes to thinking about what does it mean to learn history and to learn how to do history?

John: The last question we always ask is, what are you doing next?

David: I’m happy to say I have a sabbatical a year from now, just a one-semester sabbatical in the spring of 2020, and I’ve really become interested in environmental history and more broadly environmental humanities (I mentioned that earlier in terms of the first-year seminar I’ve been teaching). And my project is going to be to try to articulate a pedagogy for environmental history and environmental humanities and what I’m thinking of there is that there’s more to this field of study than the usual kind of content and skill mastery. And I think there are a lot of areas where this is true, but there seems to me to be a kind of affective component here, an emotional component, as we’re facing up to the environmental degradation and numerous environmental crises in our time. I say facing up to—sometimes we’re not facing up to those problems—and it can just be overwhelming from the standpoint of a learner—especially a young person—to really come to terms with the content that we would have to look at to the areas of environmental studies more generally. So I’m looking at contemplative pedagogies to think about one way to help students come to terms with and to deeply process what it is we’re studying when we look at problems with sustainability. For example, climate change. I’m really excited to dig into that next year.

John: Those sound like fascinating topics, and they should be fun to look at. And it’ll be an interesting challenge working with those issues, especially now.

Rebecca: Yeah it sounds really intense, but really needed.

David: Yeah, I think it is going to be necessary to grapple with those issues going forward.

John: Thank you for joining us. This has been really interesting.

Rebecca: Yeah, thank you so much.

David: Oh, thank you. I really enjoyed the conversation and my tea cup is empty, I don’t know about yours but I think I’m going to have to go get another cup.

John: Yeah, mine’s empty too.

Rebecca: Yeah mine’s getting really close.

John: I don’t know if you can see it, but we have a whole table covered with tea back there in the back of the room.

David: Oh I see that now.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

73. The Injustice League

Posted in active learning, comics, diversity, equity, evidence-based practices, first-year classes, higher education, learning, pedagogy, teaching
March 20, 2019

Difficult conversations like those around injustice and inequity can be challenging to facilitate no matter the student body, but first-year students have additional barriers to overcome like establishing a sense of belonging on campus. In this episode, Dr. Margaret Schmuhl joins us to discuss how comic books and programming outside of the classroom can help first-year students develop the confidence to engage with complex social issues. Maggie is an assistant professor in the Department of Criminal Justice at the State University of New York at Oswego.

Show Notes

  • Mya Brown – Assistant Professor in the Theatre department at SUNY Oswego
  • Amy Bidwell – Associate professor in the Department of Health Promotion and Wellness at SUNY Oswego
  • ACUE – Association of College and University Educators – certificate of effective college instruction

Transcript

Rebecca: Difficult conversations like those around injustice and inequity can be challenging to facilitate no matter the student body, but first-year students have additional barriers to overcome, like establishing a sense of belonging on campus. In this episode, we examine how comic books and programming outside of the classroom can help first-year students develop the confidence to engage with complex social issues.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Margaret Schmuhl an assistant professor in the Department of Criminal Justice at the State University of New York at Oswego. Welcome back Maggie.

Maggie: Hi everyone.

John: Good to have you back.

Maggie: It’s good to be here.

John: Our teas today are….

Maggie: I am having a black ginger and peach tea.

Rebecca: Oh, one of John’s favorites.

John: It is [LAUGHTER]

Rebecca: I’m having English afternoon tea.

John: And I have Tea Forte black currant tea.

Rebecca: Oh, another one of your favorites.

John: I have many favorites.

Maggie: That’s a favorite of mine too.

John: We invited you here to discuss your first-year Signature Course here at as Oswego called the Injustice League: Crime, Justice, and Inequality in Comic Books. Sounds like a really fun course. Can you tell us a little bit about the course and how it differs from your other introductory criminal justice courses?

Maggie: This class was a lot of fun to teach. In the class we read various comic books, we watched different superhero movies and we talked about, within those comic books, what it means to have justice, to recognize injustice, and how society responds to crime and maintains or perpetuates various inequalities in those stories and movies.

Rebecca: How does that class differ from the other classes that you teach in your subject area? Because it’s a First-Year Signature Course, so that has particular meaning at our institution.

Maggie: Yeah. So the signature courses here at SUNY Oswego are about bringing a student engagement aspect to our academic course content. And so in this class there’s a balance of introducing our subject matter (in my case, Criminal Justice Studies) to the students but through a really fun way, but also working with students to help them with their academic success and getting engaged with each other and with our campus community.

Rebecca: Are these usually majors that are in this class or non majors?

Maggie: So I actually had a mix of majors and non majors. I had probably about 19 students. I had about 10 or so majors and so about half of the class were non majors. And they came from a variety of disciplines, including computer science, communication studies, and we even had some undeclared majors in the course.

John: And this is part of a broader initiative that we talked about in a prior podcast, and we will include a link to that in the show notes for anyone who wants to learn more about the first-year signature courses here. So one of the purposes of this, as you said, is to build more of engagement with the college community and also amongst themselves so that students will feel more connected. And one of the first things you did, I believe, was take them on a field trip. Could you tell us a little bit about that field trip?

Maggie: In Oswego, there’s a local comic shop that we ordered some of the students comic books from. In the very first class I asked them which comic books they were interested in purchasing and how many students we’re going to accompany me to the comic book shop here in town. And so to get to the comic book shop, you have to take the public bus, or at least if you don’t have a car, and many first year students don’t, they have to learn how to take the public bus. And so part of this field trip was not only obtaining some of the course materials for the class, but also getting the students familiar with public transit in the town and how to navigate a new place with them. So we arrived at the comic book shop and the owner was very gracious to us. She made us cookies and we had coffee and the students picked up their books and some of them even got some additional materials. We had a lot of fun. It was an amusing trip bringing a bunch of college students on a bus, and some of them their first time using public transit, and the bus drivers were even entertained by the group of us, so we had a really good time.

Rebecca: I think you also discovered the infrequency of the buses….

Maggie: Oh yes.

Rebecca: …in our town, right?

Maggie: Yeah…

Maggie: Oh yes.

John: Particularly on weekends.

Maggie: Yeah, particularly on weekends. We did wait about an hour for the bus on Sunday. So that was a little bit of a lag, but we made it through.

John: I should note that the comic book shop is actually owned by the wife of a former member of my department. It’s Arlene Spizman who runs that store.

Maggie: Yeah, Arlene was wonderful.

John: She’s a very nice person.

Maggie: I didn’t realize she had that connection.

John: In fact, I just finished a paper with her husband.

Rebecca: I’m sure it can be difficult to have an authentic conversation about justice in general, especially with a diverse population of students and maybe students that don’t know each other very well. How did talking about comic books as a way to get into the topic help facilitate those discussions?

Maggie: Comic books offer a different world for students to experience some of the concepts and some of the issues that we struggle with as a society. And so to be able to visually see these issues play out across the panels, it’s a place where students don’t feel nervous or threatened, it feels safe. They’re taking these comic books and they’re finding ways to relate with them and work out some of their preconceived notions or feel like it’s okay to start working on some of these biases and issues in society.

Rebecca: It seems like it has a lot in common with some of the other topics that we’ve talked about on the podcast before, like simulations and role playing, where it’s a place to escape the real world and talk about something really challenging in a so-called fake environment, but really they’re working out real-life issues and biases and all kinds of things that can be really difficult to talk about, but it’s a lot easier to talk about character that’s not real.

Maggie: Absolutely. Yeah. One of the comic books we read is an X-Men comic book called God Loves, Man Kills and we talked a lot about the concept of othering and what it means to target out and marginalize a group of people and in a lot of ways X-Men plays out what has happened in race relations in society and in other groups who have been historically marginalized. And so for students to consume this information through a comic book, they can better reflect on their own experiences and start to understand the position of others in society.

Rebecca: It probably also makes it a lot easier to make mistakes when you’re talking about that. I think sometimes students don’t want to talk about touchy topics because they’re afraid of offending someone or saying something in the wrong way but if it’s not about anybody real….

Maggie:Yeah

Rebecca: …then it’s not going to hurt someone’s feelings.

Maggie: Yeah, absolutely…. And another thing that we did in this course right from the get go was to set ground rules for discussion and conversation. And so I had the students come up with various guidelines for discussions and we would write them down so that we could refer back to them as we continued throughout the semester, so that they all understood that they had a responsibility to each other to make sure that everyone was comfortable and safe in this classroom. It really helped to facilitate a lot of these very difficult discussions in a very similar way that comic books themselves kind of help us talk about very critical and very upsetting social issues.

John: They also come in probably very familiar with many of these comics because they’ve seen them in movies, and some of them may have read some of these as well. Could you give us some specific examples of some issues in criminal justice that you were able to address using comic books?

Maggie: In terms of the classroom breakup, we have many students who were avid comic book readers. And we had many students who were somewhat interested in comic books but were more in tuned with the recent TV shows and movies that have come out of Marvel and DC Cinematic Universes. And so there’s, you know, quite a range of knowledge around this. But for the most part, most all of the students were interested in some kind of criminal justice aspect of their studies. And so, one of the concepts that we tried to discuss in this class was the issue of mass incarceration in society. And so what we did was we read a book called Bitch Planet, and in this planet, women are incarcerated on another planet. It has a lot of strong connections to some of the ways that society has restricted women’s rights throughout history and how the punishment of women has differed across time and across place. So to be able to see these concepts in a comic book and to talk about the parallels that exist in our society was a really, really cool process for the students.

John: Did you mostly focus on comic books they were familiar with, or you mentioned Bitch Planet, which was one that perhaps many of them hadn’t seen before. Did you bring in many that were things that they hadn’t expected or that they were less familiar with?

Maggie: There was really a mix. I even had some criminology textbooks that had various criminological theories played out in comic book form and we read a few of those to give us a baseline of various theoretical perspectives on criminal behavior. But most of the comic books I’d say we’re falling in the mainstream. I think that’s what students were typically looking forward to, but they really did enjoy the new reboot of Miss Marvel, with Kamala Khan and Bitch Planet and those were perhaps a little more on the periphery than Black Panther and X-Men.

John: How did students react to this? Did they generally find it interesting? Were some students troubled by using comic books? What about the imbalance between those students who were very avid comic book fans with those who were less familiar? How did that play out?

Maggie: Some of the very avid comic book fans in the class had a lot more context to really draw from when discussing histories of the Joker or Black Panther and the development of the character over time. But because comic books have become so popular in mainstream media, with TV shows on Netflix and pretty much a new Marvel movie coming out each year, that students really had a lot to draw from. Students didn’t need a great depth of knowledge of comic books prior to coming to this class.

John: For those students who were avid comic book fans, was it a little more challenging, perhaps, than they expected to look at some of these things through perhaps a more critical lens?

Maggie: I think that comic books, even if you don’t have a great background of reading comic books, or knowing the development of various characters, I think comic books allow for anyone to just pick them up and start thinking about them in a different way. They’re relatively quick reads, which really helps. Students can read them a couple of times and start to reflect back on some of the course concepts and theories that we discussed and how they apply and pull out those very specific examples. So I think the medium of comic books really provides a great range of abilities for students.

Rebecca: What are some of the things that you learned from teaching this class that you started employing in other classes?

Maggie: Oh, that’s a good question. One of the things that teaching the Injustice League has helped me with my other courses is to really think about being explicit with what I expect and what I hope students learn from various assignments and activities. In the Injustice League, students are entering college for the very first time and so they may not always understand why we’re reading this particular article or how it relates to the comic book that we’re reading today. And so for me to slow down as an educator and say that “Here’s why we’re doing this. Here’s what the research is showing us about why low-stakes testing in this class is a good thing.” That’s helped me in my other courses be more explicit with why I’m making decisions in various teaching practices.

Rebecca: Have you done anything else that’s related to bringing more comic books to other classes or field trips or some of the other things that brought the fun piece to the class that I think really energized the group as a whole?

Maggie: In my research methods class, I’m hoping that students will be able to assist in it by going out into the community and surveying people about dating formerly incarcerated persons. And so I think to get them out into the community and to start locating various areas of the community will bring some of that campus engagement aspect to it. In my crime-mapping class we actually started geocoding some of the locations around campus and so these are more upper-division courses but I’m trying to, even though the winter months make it a little more difficult to get outside, but trying to get outside of the classroom and really talk about how important it is to be connected to our community and to understand our relationship with the community.

John: I believe there was also some type of a video or a movie that you showed and I think other classes participated in that. Could you tell us a little bit about that and how it fits into other classes?

Maggie: There was a collaborative effort among some of the first-year signature course instructors to bring our students together in a common place. We ended up watching a episode of Luke Cage and this particular episode really spoke to a lot of the different courses that were being offered in the Signature Course program. Obviously my course, as one that deals with comic books and crime and justice and inequalities, Luke Cage is a very good example of many of the concepts that we talked about in class. But Mya Brown in the theater department also taught a class called Blackish Mirror and it followed the development of black characters on television. And so this was also a really good place for her class to talk about how various stereotypes that they had learned existed and/or were resisted against in Luke Cage. We also had a professor from political science and from communication studies, talk about political organizing or activism in Luke Cage, as well as narrative and the use of narrative in TV shows. We even had a signature course instructor in the health and wellness department… their class made snacks for the students to enjoy at the event.

John: Healthy snacks.

Maggie: Yeah, healthy snacks and it was brilliant. The students loved it. They created a snack mix that could be created and replicated by using ingredients found on campus. So that was a really cool way to bring in even a discipline that’s not necessarily focused on examining social inequalities in media to this event, and so it really spoke to a lot of students across various disciplines.

John: And we should note that, that person was Amy Bidwell, who was on an earlier episode. Were there any surprises in teaching the class that you didn’t expect?

Maggie: The class was a lot of fun. I don’t think I’ve had as much fun teaching any other classes I had teaching this class. It was really fun to pick up a hobby of mine, something like reading comic books, to bring this to the classroom and to start and challenge students to think about the media they consume in a new way, and how it reflects what we do in society and various values that society has. One of the most surprising things in the classroom was really how much of a community the students had at the end of the class. They had been speaking about other courses and working together on other projects and planning their course schedules for the next semester so that others would be in their courses together and so that was a really cool outcome of the class.

John: …and I believe you also opened an Instagram account for the class.

Maggie: I did and so you can follow it @the_injustice_league_oz… each word is underscored. I won’t say that I have many followers on the Instagram account but a lot of the students who did follow it seemed to really enjoy it.

John: And are you going to be teaching this again?

Maggie: I will be teaching this class next fall. So I’m very much looking forward to the next cohort of Injustice League members.

Rebecca: Did you carry on the superhero fantasy world theme throughout the class? You talked about rules for discussion or rules for engagement at the beginning. It’s almost like world building. Did you think about theming that more? Could you talk about how you might have done some of that?

Maggie: All of the designing my syllabus was all thinking of the class as being a part of a group of superheroes as opposed to just a group of students in the class. I even designed the midterm exam to look like a top-secret mission directive from their Professor S, which is me. The secret mission was about identifying various concepts that we talked about in class and applying them to a new comic book that we hadn’t read in the class. And so, in this midterm exam, they got to explore some of their favorites that we may not have gotten to touch on the class. It was a good opportunity for them to get creative and think about how these theories and concerns about justice translate across various stories.

John: And that way, you’re giving them some autonomy, but you’re also helping them develop transfer skills so they can take the things you learned and apply them in new circumstances, which is a really good practice.

Maggie: Even one of my students, when we were discussing moral panics, stopped into my office hours one day and was ecstatic because he had just realized that his journalism course was talking about moral panics, and so to be able to identify these concepts across disciplines was also a really cool outcome of the class.

Rebecca: You talked a bit about the class being really fun to teach. And part of that’s because you brought your hobby and your discipline together. But were there other things that made the class fun? I can imagine that you’ve all thought about yourselves as a part of a league. So maybe that you felt more connected to your students, or am I kind of projecting?

Maggie: Oh, absolutely. So, I called myself Professor S as a play on Professor X in X-men and so the students really loved that and they had a really good time with the way we even addressed each other in the class. The Instagram account even helped create more of a community by bringing in various pictures of each other doing or identifying various comic things across our everyday lives and interactions.

John: How did you first get interested in comic books?

Maggie: Actually, my first interest in comic books came from graphic novels and reading Persepolis as a kid. But, of course, I fell in love with the Marvel Cinematic Universe and I’ve watched pretty much all of the movies in chronological order.

Rebecca: Of course [LAUGHTER]

Maggie: I called it “research,” the summer I rewatched them all, it was a wonderful time. After reading a novel like Persepolis it was also something that really got me interested in criminal justice and society and inequalities in social life.

John: Did the students seem more open to discussing some of these issues having been exposed to them through comic books?

Maggie: Comic books reflect a lot of what is going on currently in society and they provide us a way to talk about really difficult topics of racism and sexism and things that occur and that people and students are experiencing in their everyday lives. So using comic books to facilitate these conversations is really important for students just beginning to question some of these processes.

John: One aspect of this course, as you said before, was to help introduce students to college life and help them create bonds and connections. But that also frees you up quite a bit because you don’t have a standard curriculum. Is this the first time you’ve ever taught a class where you didn’t have a fixed amount of material you had to cover in the course?

Maggie: Yeah, so this class was really flexible in that way. As I look back on the class, I’d say that it’s equally as important for us to be talking about some of the content about comic books and the sociological and criminological aspects of them as it was to help students become more connected to their community and to their campus, but also to ensure that they will be successful students at moving forward. And so this class really allowed me to work on some of their questions that would just come up, like calculating a GPA or registering for classes. And so the flexibility that exists in this class lets me respond to the students and their concerns in the moment and to occasionally tie-in some of those issues in current events to what we’re discussing in these comic books.

Rebecca: I could also imagine that it allows for the tangents that might occur as you start talking about something related to the comic book but you think it’s a valuable discussion. But if you have a finite amount of material in a finite amount of time, you might not be able to b go down those rabbit holes, but they can be such valuable conversations.

Maggie: Yeah, absolutely. You know, the flexibility of this course isn’t just to my benefit, but to the students benefit, where they can ask questions and we don’t have to worry about how much material we get through. We don’t have that curriculum that requires various elements to be covered and so the students can explore some of their questions in a very meaningful way.

Rebecca: I think it might be useful to just clarify that these classes are not part of any specific major, and they’re not a prerequisite for anything. So that’s what we’ve been talking about in terms of them being kind of freeing. I don’t know if we explicitly stated that.

John: Did you get to know the students better than you would in a typical introductory class?

Maggie: O ne of the good things about this particular classes is that there were only 19 students, and so it really allowed for me to get to know each of them individually and be able to see their personalities through our discussions and to have that comfort level with the classroom to talk about what made their day not so great today and what they really enjoyed about the weekend. And so to have that sort of informal relationship in a very formal setting was a really cool experience.

Rebecca: I know that one of the things that I’ve been thinking about after hearing many of the faculty who taught the signature classes talk about their classes is just finding ways to have some more of those informal opportunities in class, but also thinking very carefully about the content that I think that needs to be covered versus what maybe actually needs to be covered. There tends to be a disconnect, We think we need to cram in so much stuff. What are some of the key principles and things? And can we go into more depth for some of those if students are interested? And I’ve allowed that to happen a little bit this semester, and it’s been really delightful, I think, for everybody involved.

Maggie: Yeah, that was one of the things that I struggled with in the class. At the very beginning I was treating the class like a topics course and cramming, or at least planning to cram, a ton of information in. A few weeks in, I realized that it just wasn’t going to work for this type of class, that this class really did need time to facilitate these relationships and to help students learn and navigate their first semester here on campus. And so to have that flexibility for them to be able to explore their questions and concerns on campus and off campus was a important part of this class.

Rebecca: Sometimes I think that these functional aspects of being a student can get in the way of learning. So spending the time and just addressing those concerns that are preoccupying a student can free them up to actually think about the content and spend time investigating it. So, if they’re really concerned about figuring out their GPA or really concerned about making sure they’re registering for the right classes, addressing that concern up front can actually free them to focus on learning.

Maggie: Yeah, absolutely. And a lot of students, they come into the classroom and they think that college is going to be just like their high school experiences and so that studying in the same ways is going to be effective for them or that GPA’s and what credit hours are. There’s a lot of new information that makes transitioning to college more difficult one then, say, transitioning from their middle school to their high school . And so I think this does give them the time in class to talk with a faculty member to try to work out some of these questions in a way that they may not get to in their other courses and so it does certainly alleviate some of their anxiety around these issues.

Rebecca: When there’s not a context like that I think the option is going to office hours or something and that can be really intimidating, I think, for first-year students, or they just have no idea what office hours are for, which is another thing.

Maggie: Right, or how to book an appointment…

Rebecca: Yeah.

Maggie: …and all of that. And so I know many of the First-Year Signature Course instructors, including myself, tried to have individual meetings with students to break the barrier of setting up an appointment for office hours and how to draft emails to your professors and such and so I think it really helps them not be as nervous about getting the help they need and the resources that they may need in the future moving forward.

John: Because in the past if they were called into go to someone’s office after class…

Rebecca: Right, it was a bad thing, yeah.

John: Exactly. And so, you know, that’s something they do need to get past and it takes a while often and by then sometimes a little too late. So that’s really helpful.

Rebecca: Speaking of criminal justice, right? [LAUGHTER]

Maggie: Colleges, it’s a different culture than they’re used to and so to get assimilated to that culture is really important in many different ways.

Rebecca: Right, it’s like mentoring instead of a penal system.

Maggie: Right? [LAUGHTER]

Rebecca: Kind of a weird word flip there.

Maggie: Yeah, absolutely.

John: And one of the nice things about this whole program is it was set up as a randomized controlled experiment where students were asked if they were interested in courses and then they were randomly assigned or not assigned. And there’s some work that’s being done right now analyzing how their outcomes compared to outcomes of the students who were not in one of these groups, and they’ll be followed a bit to see how this works overall. So, I’m looking forward to seeing more, but the preliminary results they have, as were reported in the meeting this morning, were fairly positive.

Maggie: Yeah, retention was really good and so hopefully that’ll continue.

John: Semester-to-semester retention….

Maggie: for underrepresented populations, yeah. There was…

John: …was 100% retention semester to semester.

Maggie: Yeah.

John: It’ll be interesting to see if that persists, because that has not always been the experience of Freshmen.

Maggie: Right, and hopefully it does and I think one of the things this Signature Course program is trying to promote are those students and faculty relationships and that if students have a strong bond with a staff member or faculty that they’ll be more successful in all aspects of their academic life.

Rebecca: We always wrap up by asking, what next?

Maggie: Well next, I’m currently meeting with various new faculty members for the Signature Course program so we’re going to work our way through more course prep, and I’m very excited to meet the newest members of the Injustice League next Fall.

John: And you’re also joining the cohort of people in ACUE…

Maggie: Yes.

John: …which is starting up here on campus very shortly.

Rebecca: …another league.

Maggie: Yeah, another league of sorts. [LAUGHTER] I’m very excited… very excited for that as well.

John: T hank you. It’s been a lot of fun talking to you about this course and I wish I could take most of these courses.

Rebecca: I know, they’re always so much fun to hear about, but I think they give us lots of prompts and interesting things that we can start to consider in other contexts too.

Maggie: Absolutely. Thank you for having me.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

71. Small Teaching Online

Posted in active learning, evidence-based practices, higher education, learning, online learning, pedagogy, peer instruction, professional development, teaching, teaching centers
March 6, 2019

Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, Flower Darby joins us to explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

Show Notes

Transcript

John: Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, we explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Flower Darby an Instructional Designer an Adjunct Instructor in several disciplines and the author (with James Lang) of Small Teaching Online, which is scheduled for release in June 2019. Welcome Flower.

Flower: Hi John. Hi Rebecca. Thank you for having me. I appreciate that. It’s good to be here.

Rebecca: We’re really glad that you’re joining us as well. Today our teas are:

Flower: I am drinking Builders tea. Good, strong cuppa here.

Rebecca: Sounds yummy.

John: We have some of that next door. I am drinking ginger peach gree n tea.

Rebecca: I have my Golden Monkey again today.

John: We ran a faculty reading group here in the Fall semester of 2017 based on Small Teaching. Many faculty found that to be highly inspirational and we had over 100 people participate in that. One of the things that came up quite a bit is how this might be applied online. So there’s a lot of people interested in your forthcoming book. Could you tell us a little bit about how this book came about?

Flower: Sure. So Jim Lang came to my campus—Northern Arizona University—in January 2018 and delivered a talk on Small Teaching and as we know, the book has been very impactful for faculty around the country and around the world. And while he was at our campus, when it came time for the question and answers, somebody raised their hand and said, “Sure, but how do you do this online?” And Jim’s immediate response was, “That’s the first question I always get at every talk that I give,” and he said, “I don’t know. I would need a co-author because I don’t know how to do this online, but that would be a great book.” So I thought about that for a few days and then I approached him and I said, “Pick me. I would love write that book with you. I can see the value of it, I can see the need for it.” So that’s how the conversation began.

Rebecca: How does this extend the approach that was used in Small Teaching?

Flower: Well, it follows the same principles for certain that there is learning science that we can draw on to help us make the everyday decisions in our teaching and learning that have really an outsized impact on student learning and outcomes. So there are little things that we do on a day-to-day basis and we can draw from the research to discover what will have the most impact. Again, understanding that in order for faculty to really be able to implement something new, it’s got to be feasible. It must be doable. The daunting overhaul of a major course redesign is so off-putting that most faculty won’t get around to it, myself included. When I have gone to multiple workshops and conferences and sessions or read about an approach. And I, “That is a great idea,” and I spend about five minutes thinking about how I might incorporate that into my class and then I say, “Too much work. Too much time. I don’t have that time available. I don’t want to implement something that’s only half baked,” and the idea gets left out. So in our online classes, there are so many things that we can do that are on that small scale but will have that outsized impact on our students’ engagement and their learning. And so that’s what this book sets out to do, is to explain a lot of those principles and draw on the research that we have to show faculty how they can make these changes in their online classes.

Rebecca: We’ve talked many times on our podcast about the lack of preparation for faculty teaching in general and that’s certainly true for online teaching. You might have taught a face-to-face class, and then all of a sudden, now you’re teaching an online class and boom, you have to figure it out. Can you help us think through what are some things that faculty can do as they’re new or getting used to being an online teacher?

Flower: Sure, and I think that’s really the point here. Centuries, millennia, compared to the way that we teach and we coach and we mentor face to face, or even as we’re doing here using video conferencing software, but it’s a real-time interaction. Well, online teaching is very, very recent, say 20 years or so. And faculty don’t have the experience that they bring into the physical classroom. You may have heard of the phrase of the apprenticeship of observation coined by Dan Lortie. And this is the idea that by the time a teacher steps into a classroom to teach, he or she has had years and years of experience in a physical classroom being a student and observing what happens and how things go and thinking, if somebody chooses to be a teacher, then they’ve clearly put a lot of thought into how they want to teach. Well, we simply don’t have that for online. I do expect this to change in coming years. But the fact is right now that most of our faculty have either never taken an online class or if they have, it may be a very limited experience, not the years and years that they came out of K-12 with. And the same, quite frankly, is true for our students. They’re also pretty inexperienced at an online classroom. And the way this pans out is that literally faculty and students both don’t know what’s supposed to happen in an online class. They don’t have the social norms, they don’t know what the classroom looks like. If you think about it, when you walk into a physical classroom to start teaching, you know what’s in the room and you know what’s supposed to happen. You see the desks or the tables, you see a lectern at the front, you see a whiteboard or a projection screen, and students and faculty understand what is supposed to happen. Students go and sit in the desks, they face front, they wait for the faculty to come to the podium. It’s rare that a student would walk into a classroom and at the beginning of the hour, just step into the lectern. Students know that’s not what they do. But my argument is we don’t have that kind of social norming convention for online classes…yet. I think it’s coming, but right now many of the people who find themselves in our online learning environments go into that space, and they don’t know what things should look like, they don’t know where the light switch is, they don’t know where the desks are, where the whiteboard is. So just that whole lack of experience is rather disconcerting. And it’s hard to know what to do. Faculty don’t have experience—they have haven’t seen models, students are equally unprepared—so there’s a lot of work to be done here just to understand what should happen in an online class, what the furniture is, where it should be to facilitate learning. That’s where those gaps happen for faculty. You ask, “How can faculty prepare themselves?” I could talk for days about that question. It’s a growing need and some institutions are beginning to recognize the importance of doing a much more thorough job of preparing faculty to teach online. But I will argue that those institutions are still pretty few and far between. I would say, based on my research and my experience, the vast majority of faculty who are teaching online have not had specific development in that area. They have not observed peers’ classes. In fact, what can happen is a negative effect. Very commonly, when faculty begin teaching online, they are handed somebody else’s content. We’ve seen that happen and that’s a mercy in a way because that way faculty who are new to teaching online don’t also have to develop the course. But what can easily happen is that the content that might be given to a new faculty member might not actually be exemplary in the design and the delivery of that material. So then what happens is the only experience that faculty get is observing the content and the structure of a less than ideal example and then that’s the model that they have and they think, “Oh, I guess this is how it is”. So work can be done on developing better exemplars, better development programs. I believe, as faculty are coming out of online graduate programs down the line a little way, I believe, will have better experienced faculty and students. A lot of research going on in this area, but that work is all to be done.

Rebecca: As you were talking, I thought, you provide a nice model, it’s a nice way of thinking about it, you don’t know where the furniture is.

Flower: Mmm-hmm

Rebecca: It sticks with me. I was thinking about that the experience that a lot of faculty and students have is more in the realm of social media and so they’re looking for cues that are similar to those kinds of environments. The activity that’s happening in those environments is really different than the kinds of activities we would expect to happen in an online platform for learning.

Flower: Right, that’s a great point. We interact with other people so much online and on our devices using social media and what’s interesting to me is that we can really engage with people in those online spaces. Somebody tweets something that’s a little bit incendiary or provocative and you get all kinds of people jumping in and commenting and you know, sometimes things get heated, or a really heartwarming moment is tweeted or shared on Instagram, and people are all over that post. But the opposite is kind of true in our online classes. Indeed, I feel like we could bring in some of the techniques from social media into online classes. I’m not saying that faculty should all have a component of Twitter or Instagram in their online classes, but what I’m saying is that it’s possible to deeply engage people in online interactions. And that’s not a feature that, I would say, generally characterizes online classes—we usually hear the opposite, that it’s not engaging, it’s difficult to drum up those discussion posts—and I feel like if we could draw some of those principles from how we interact with people online, in social media, of using our devices, if we could bring those into the online classes right away, we’ll see more engagement and engagement precedes learning. Students have to want to be there in order to learn when we’re engaging them and if you could imagine posting a discussion post and then you can’t wait to see what people are responding. We do that all the time on Facebook or Twitter sending something out and then, “Oh let me see! Did people like that? Did people say anything?” And we just naturally are drawn into those spaces to check and see what are people’s reactions? Well, if we could design that kind of a discussion board for online classes, where it’s so interesting and engaging that people want to rush back and see who’s talking to them, who’s replying to them, that would go away way to improving the online learning experience for both faculty and students.

John: That’s not an experience though that many people teaching online find in their discussion forums. Are there any hints or tips that you can give people to make their discussion forums a bit more engaging so that students don’t wait until the last minute to do the standard three posts or whatever is required in that course?

Flower: Great question, John, and a big one. And again, thinking about Small Teaching ways of making small changes, I heard of an example recently where faculty asked students to reply to their peers posts using a GIF that just represented—one of those funny moving little images that sort of expressed—their reaction. And that’s an example of bringing in new ways of engaging and it’s not rocket science. It’s also perhaps a little more fun, which is important to bring into an online class. A great way of sort of getting students to think differently. But if that idea doesn’t resonate with you, maybe you might want to try offering options in your discussion board questions. I’ve supported over 100 faculty, I might even say, hundreds of faculty in the design and development of their online courses and what I see sometimes is one question for students to answer and oftentimes it’s kind of black and white. It’s hard to discuss a question like that. So first of all, craft questions that are discussable, that there’s some debate around that you can make different arguments or points of view. Tie those questions to students’ experiences. How is the content impacting them personally? Where do they see these concepts in their own life and experience? And, even better, provide three or four different questions that students could choose to respond to and then ideally, everyone isn’t all talking about the same question, so that’s more of a natural way of fostering some conversation in an online discussion.

John: One of the nice things about tying it to personal things, I would think that that would also help build more of a sense of community within the group because the students get to know each other a little bit better, which may affect their engagement in other activities,

Flower: Right. Anything that we can do to increase the value and the relevance of what we’re asking students to do online is hugely impactful, and it doesn’t have to take much. I have a colleague who teaches an online First-Year Seminar course, which in a way is a bit of an oxymoron because First-Year Seminar courses are often designed to really hook in our first-year students who are transitioning to university life, but she was tasked with developing and teaching a really highly engaging and supportive Freshman First-Year Seminar class. And one of the things that she does is she brings in a discussion board and one of the prompts is, “If you could be a superhero, what would your superpowers be?” And again, maybe on the surface some people might think that’s a bit trivial, but what she’s doing is she’s getting students to talk about character traits and hero qualities and concepts that rely and relate to the material that they’re engaging with… yet in a fun and a more personal way. And it certainly does a lot to foster those relationships that are so important for online classes to build that community. Absolutely.

Rebecca: I think one of the methods that we hear a lot about in terms of online learning is the ability to do quizzing and retrieval practice and interleaving through quizzing. But are there some other ways that we can integrate some of these evidence-based practices that aren’t maybe the typical solutions that we tend to think of online?

Flower: I think one of the most underutilized functions of the Learning Management System is what we call adaptive release or conditional release. And I actually want to pause here and say that these Learning Management Systems have come a long way in recent years, and they still have a long way to go. [LAUGHTER]

Rebecca and John: Yes. [LAUGHTER]

Flower: For many faculty and students, the functionality is lacking, the furniture is stark, they’re not attractive places to be and as I said earlier, engagement precedes learning. If you were to ask yourself for online faculty who are listening, “Do you want to be in your online class?” I suspect many faculty would struggle to answer with a resounding “Yes.” And so here’s a shoutout to our LMS developers to think about space design and the experience of students and faculty in these spaces. Having said that, there is some very interesting functionality that is oftentimes underutilized and I would argue that’s because, again, faculty may not have the preparation and the exemplars to begin teaching online. With adaptive or conditional release—it’s called different things in different systems—you can set a task that then opens up the rest of the content in that module. And I love to use this. You can use it equally effectively at the end of an online module or at the very beginning to open the next module. Now what you can do with this is you can embed retrieval practice exercise. Or, you know, drawing from Jim Lang’s book, Small Teaching, a predicting exercise works equally well. A curiosity provoking exercise, and all it really has to be is an assignment where students submit whatever it might be. A two sentence summary of what their big takeaways were from the previous module, or predicting what might be in the coming module or posing some questions… this can be written, it could be a recorded submission for students who might find it easier to talk through their ideas. Once students submit that element, then the rest of the module opens. Before they do that they can’t access any of the content. These things don’t even have to be graded, you can just set them to be worth zero points, but they are mandatory because the students can’t proceed with the content until they submit them. So when you think about that feature, there’s a lot of creative things that you can do that don’t impact faculty grading time. That’s a big tenet of the new book is we can’t overburden faculty with grading and yet tie into those practices that we know from the research are effective.

John: One of the chapters of your forthcoming book is on fostering student persistence and success. Could you give us perhaps a strategy or two that might be useful in encouraging student persistence? Because I know one of the problems in online classes is they often have higher drop, fail, and withdrawal rates than face-to-face classes. What are some techniques that faculty can do to help improve student persistence in the class and the program?

Flower: That’s right, great question. As you point out, the attrition rates in online classes are remarkably higher. And we also find that for students who are less prepared for higher education, if they don’t succeed in that class, then the odds increase dramatically that they won’t actually persist and attain a college degree, and that’s a problem. But as I was saying earlier, a global concern is that online classes are not nice places to be. And if your listeners have any pushback on that, please feel free to reach out and engage with me on that assertion. But what can we do to just make the place a little more pleasant? How can we be warm and friendly and supportive and encouraging? How can we allow our humanity—even our personality—to show through? I was speaking with a good friend and a colleague of mine just a few months ago when I was delivering a little talk about this book and he was telling me though he’s been teaching online for 10 years, that he’d never thought of just being himself in his online class. And he explained to me that he loves teaching in person—he’s quite a character, super dynamic, very engaging, funny, loves to interact with his students in the classroom—and yet, he told me when he goes into his online class, it’s like a robot. There is no trace of his personality. And other people are saying this too, just be yourself in those online classes and make a deliberate effort to infuse warmth. But a specific strategy that people might want to try is to assign a goals contract as one of the items that are due in the opening module—or the orientation module—and a goals contract, you’ll see different kinds of variations around, but here’s the two pieces that I really like. A lot of people are talking about assigning sort of a memo of understanding or a contract where students agree that in this online class, they should schedule set times, they should plan on X number of hours per week, they should reach out immediately if they have questions. People are doing that. I like to embed a different element as well, which is to require students to set a couple of goals and it can be literally two. What are two goals that you have for your learning, or your success, your ability to earn an A in this class? And then an interesting twist is to ask students to identify one potential challenge. It’s still the case, I have my students all the time saying, “Well, my computer is in the shop, [LAUGHTER] it’s sort of all of a sudden, it busted and now it’s at the technician and I can’t do my online tasks.” So helping students to think in advance about a scenario such as that and of course, in that particular case, many campuses have computer labs or libraries where students can go and access another way to get into the course but maybe they haven’t thought about it in advance. So in the goals contract, ask students to set two goals, identify one potential challenge that might come up, and identify a strategy for how they can address that particular challenge. And certainly, identifying one challenge is not going to cover the range of things that happen in life during the course of an online class, but I think it sets the tone to get students thinking that one little hiccup doesn’t mean that we’re all done with this online class and we just have to sort of fade away and stop participating. And then what you can also do, periodically throughout the class, is you can ask students to revisit those goals that they set for themselves. How are they doing with that? What kind of progress are they making? Are there some strategies that aren’t working for them? Do they need to recommit to the intentional and deliberate scheduling of their class time? Just helping students be very explicit about what their plan is to succeed and finish the course.

Rebecca: What I like about what you’re saying is, it switches from really having the faculty member impose everything, and have the students be co-authors of the class to some extent, and they have some ownership over the space, which generally means that they’ll probably commit more.

Flower: What we know about online learning is that students must have a higher degree of self regulation, self direction, they must be more motivated, and be able to manage their time well. And if students don’t have those things, it’s much less likely that they’ll persist and finish an online class. And yet, when you think about it, online classes work directly against a student’s ability to do those things and here’s what I mean. When you are teaching in person, when you’re a student in an in-person class, you know that every Tuesday and Thursday at 9:30 you’re supposed to be in the classroom and it’s a natural way to help students hold themselves accountable for doing the work. Now I know sometimes students come to classes and they’re not fully prepared, but there’s still that built in mechanism where they’re going to be in the same room with their faculty member with other students. There’s a social element of accountability that’s like, “Well I know I’m supposed to show up and I should have my stuff done,” or “There’s a test next week and I need to be ready.” Well those real time interactions and those interactions with physical people don’t tend to happen in the typical asynchronous online course. Very often—I would say 99% of the time, probably—an online student is sitting at home by himself or at the coffee shop by herself. If she has a quick question about something, she can’t do what she does in the classroom and say, “Hey, did you understand what we’re supposed to do on that particular assignment?” or “Hey, faculty member, can you just re-explain that? I’m not quite there yet.” There’s no way to get that immediate response, that immediate quick guidance that might take two minutes in a physical classroom. So, students don’t have the accountability, they don’t have the physical presence of the instructor or the student, and so we have to go above and beyond in our efforts to build in structures that help students develop the kinds of self-regulated skills, the kinds of self-directed learning skills. Many of our students are not coming in with those skills already but we know students who do have those skills will be much more successful, so let’s build that into our curriculum. Let’s help them develop some of those, let’s talk to them about the importance of monitoring their own learning, and let’s structure exercises that will help them to do this. I’m pulling a lot of this material from Linda Nilson’s book—It’s called Creating Self-Regulated Learners—Although that book is not necessarily focused for an online environment, I think it can be hugely helpful to our online students to be very transparent with them about the importance of developing these habits, these behaviors for success. And as I said, structuring exercises and graded assignments that help them to do that, to hold them accountable.

Rebecca: Following up with what you just said, there’s a chapter in your forthcoming book called “Creating Autonomy.” Can you talk a little bit about small ways that we can give students autonomy in the classroom and in an online space?

Flower: Sure. And again, let’s be sure to keep that focus on small, doable, feasible changes, things that you could do in maybe a 15-minute work session and have it rolled out for your online class. One thing that we could do is to develop a self-enroll group structure. Many online faculty like to bring in collaborative learning tasks to, again, foster that community and the peer to peer instruction and learning that is so important as we know, but I think oftentimes we sort of assume that what we should do is purposefully group students, and there’s certainly value to be found in designing purposeful groups. But what can also be very interesting is to allow students to enroll themselves in groups that might cover a range of different topics. For example, sometimes I teach Educational Technology online classes. And if I were teaching that class today, I might offer five different groups that students can sign up for on a first-come first-serve basis. And one might be virtual reality, and one might be mobile learning, and one might be writing in digital spaces. So students could naturally choose a topic that they’re more interested in pursuing and when students have that level of autonomy, to make that choice of what their going to focus on, that’s one way of embedding just an opportunity for students to exercise that autonomy. Another even easier way I’ve already mentioned here is to offer students a choice between whether they want to submit a written task or whether they prefer to record on video or audio. Students carry these amazing devices in their pockets all the time with high- tech recording equipment embedded right in them. And students love the freedom of just being able to talk through their ideas, their responses, you can get a much more authentic response from students. Teach them how to use the recording software, or how to upload the video or the audio clip into the LMS and now you’ve got an easy choice that you can give students. If you prefer to write this, go ahead. If you prefer to record it, do it that way.

John: One thing that struck me is I used VoiceThread last year in an online class and I expected they’d actually use the video option with it very often. I gave them the choice of whether they use just voice or voice and video or use a video recording and yet none of them ever presented on video, which surprised me, given how common that is in social media. Why might that be?

Flower: Well, sure. I also require video discussions in some of my classes. And what I have learned is that people are nervous, especially in an academic setting about how they come across on camera. I feel like audio is a little bit less threatening, but sometimes people don’t like the way they look. And, you know, faculty too. [LAUGHTER] A lot of faculty are uncomfortable with recording on a video, and yet, it’s the way of the future. So right now, currently, I’m teaching my graduate level class on technological fluency and leadership. I require those video discussions and I say to them, “Are you nervous about doing this? Well, I want you to do it anyway,” because video interviews right, over Skype, or Zoom, as we’re doing here today, or video resumes. These are a thing that are happening and helping people to get more comfortable with showing their face on camera. I also talked to them a lot about the importance of seeing their peers faces and how much we can learn just from that. In the program that I teach now, students tend to take classes with the same people. But in my class, they always say, “I’ve never put a face to a name, how nice it is to see you,” and it makes a huge impact in terms of that community element. But I talk to my students very explicitly. Now, it’s also really important to think about situations where a student may not want to represent their face and there can be very good reasons. I had a tragic situation just last year where a student did not post the video, she posted a static picture of herself. I came to find out at the end of the semester. When she did that I was like, “Oh well, dock a few points, whatever. That was weird. Why’d she do that?” I later found out that she had been in a domestic abuse situation and she was ashamed of the way that her face looked because it was still very visible—the damage—and it just struck me to the core. An arbitrary decision that I made that it’s so important to talk to each other and look each other in the eye and she had a really, really strong reason for not wanting to do that. So back to that topic of offering a choice, what I do now is I tell my students, “If there’s a really good reason that you don’t want to show your face on the video, please send me a quick note. You don’t even have to tell me details, but just explain that you’re going to choose to do this other thing instead,” and posting a static picture is still pretty effective. So I think it’s very important to remember that our online students are people. They have lives and we need to be thinking about the decisions that we’re making in our teaching and how that might come across to a student… how it might induce anxiety in ways that we never anticipated.

Rebecca: One of the things that I wanted to follow up on and see discussed, the self-enrolling groups and collaborative work online. I think we have clear ideas about how that might work in a physical classroom, but not always a good clear way of how we can coach students through collaborative learning online. So even small, quick things that came up in Small Teaching, like Think-Pair-Share, you can envision how to do that in a classroom, but maybe have no idea how to do that in an online classroom.

Flower: It’s a great question, Rebecca. And I know that there’s actually a lot of pushback from online students and sometimes online faculty about the value of collaborative learning activities. It just so happens that my husband is in an online Master’s program right now and so I’m living with the student experience. And it’s frustrating to our online students—many of whom are not traditional 18 to 24 year olds, they might be returning adults—and one of the reasons (in fact, a primary reason) that our online students choose that modality is because they have busy lives. A big percentage of our students have jobs, families, obligations, and they need to do their work when they have time. That might be 8 pm, that might be 11 pm after all the kids are in bed. It might be 6 am, I like to do my online class at 6 am. When you require students to work in groups in an online setting, you’re removing that degree of scheduling flexibility that students value in an online class. So if you choose to require online activities, I have certainly moved towards lower stakes and opportunities that don’t require real-time meetings between students online. And you mentioned a great one. Think-Pair-Share can be set up in an online class. So there’s lots of ways that you can do this but the first thing that came to my mind is you could set up groups of two, and you could auto-enroll students in a group of two, and then they have their own individual discussion boards. In most Learning Management Systems when you have groups you can have kind of a private discussion board where students can interact with each other there, or I’m a big fan of letting some of the learning come outside of the Learning Management System. So let students know who their buddy is, have them exchange phone numbers, and they can just talk on the phone. Sometimes we forget those simple solutions. But a Think-Pair-Share—and so many ways that you could set this up, you could change it from module to module so people are always working with somebody else—just share an idea, discuss something, take it offline, come back and just write or record a quick summary of how that interaction went. When it’s not such a high stakes assignment, students can better engage in those opportunities. It’s so much easier to find 15 minutes to talk with one person than it is to find an hour with four working adults who all have family obligations. So I love the idea of lowering the stakes and embedding lots of little opportunities for students to work in pairs or in groups of three where it’s easier to coordinate. There’s less pressure about the online group member who never does the work—sorry, but that’s a thing—and just help students see other ways of interacting. Now, with my instructional designer hat on I want to remind us of the importance of making sure that online collaborative work aligns with the outcomes of the course. Very important to think about why you’re asking students to work together. Does this actually relate to what you want them to learn and get out of the course? Very important to pause, ask yourself some of those questions before you randomly assign group work because we should have group work, which I’m guilty of doing. [LAUGHTER] It’s an easy thing to do, “I guess we should have group work,” but really pausing to think carefully about the purpose of that. And then again, maybe thinking creatively about those lower stake ways of connecting students and facilitating some more authentic interactions. Maybe they’re going to text each other. That’s fine, they’re talking. We do a lot of talking on text these days. Help students connect in ways that are not so stilted, which is often what we see in the use of the discussion board and the LMS.

Rebecca: I found that too, I use Slack a lot in my classes, because it’s a common platform for designers and people in that realm to communicate professionally, and they love it. It’s convenient, it’s on their phones, takes it away from a clunky interface…

Flower: Sure.

Rebecca: .. some of the LMS’s have and it’s really productive. And they’re able to do that midnight chat with each other.

Flower: Yes, absolutely. Again, let’s think creatively about tools that students already have. I honestly believe that a lot of Learning Management Systems actually raise barriers to student learning because most of them—although this is getting better—most of them don’t have a super robust mobile app and so a student, really to engage with coursework, has to find a place where they can sit down and log into the computer and access the course and jump through a million hoops before they can even get to where the learning is. Whereas if we take some of that learning into apps that they’re already using or things that they’re doing on their phone anyway where it’s in their pocket, we can communicate in real time. Now I need to exercise caution here because many faculty think, “Oh great. I’ll do Slack, and I’ll do VoiceThread, and I’ll do Flipgrid, and I’ll do Twitter, and I’ll do Pinterest, and it’s just going to be so interesting and fun.” Well, if there’s a reason for using some of those tools, absolutely. If those tools are just shiny entertainment—bells and whistles—then you may want to think again. Another important consideration if you’re asking students to use tools that are not in the Learning Management System is whether those tools are fully accessible for students, whether there’s any fee that’s involved, whether students might have to set up a new account with a new password, that might just be a hassle. So really you want to think carefully about what you’re asking students to do. Are the tools fully accessible and usable and cost friendly? Do they support your learning outcomes? And yet, if a tool that you’re thinking about using passes all those tests, then by all means jump right in. This semester I’m using Remind which is the simplest tool on the planet and the most effective. [LAUGHTER] It’s more in use in K-12 currently than in higher ed. It’s simply a text app that anonymizes people’s phone numbers. So I invite my students to sign up for my Remind list. I don’t require it. But then I can easily send a quick little 140 character reminder, “Don’t forget this assessment is due on this particular day,” or “New content has just been released. Login when you get a chance.” The message goes right to where the students are and because I make it optional, nobody is required to have the annoying instructor on their phone all the time. But students who want some additional support with managing deadlines and the class experience really appreciate the use of the simple tool called Remind.

Rebecca: I think one of the things that we know from a lot of evidence-based practices and books that have come out—including Small Teaching—is that frequent feedback is useful. But we also know that frequent feedback can seem really daunting to a faculty member, and time consuming. So are there ways that you would suggest managing some feedback opportunities online, but keeping it easy, quick, and reasonable?

Flower: Sure. Another great question. Another underutilized approach—at least in my experience supporting the faculty that I work with—is the ability to embed feedback into auto-graded multiple choice or true- false types of quizzes within the Learning Management System. So in most of these systems you can design feedback that will show up for students as soon as they submit the quiz. You can set those quizzes to show students which questions they got right or wrong and in the wrong answers you can embed feedback that says, “Please review pages 32 to 35 of this chapter. That is where you’ll find this information.” Similarly, you could encourage or embed challenging feedback and by that I mean, “Great, you totally know this material. If you’re interested in learning more, you may want to check out this website or this resource,” to offer students a range of experiences and engage students at their different levels of experience with the content. To be fair, setting up that kind of embedded feedback takes a little bit of time in the first place, but many of us teach those online courses over and over again, and once you’ve done that work, you can benefit from it time and time again. If you’re not sure how to do that in your Learning Management System, just about every institution has a Learning Management System support team with instructional designers or system admins, help desk folks who can walk you through the creation of that kind of embedded feedback. And it’s timely, it’s right there when the students are thinking about that problem in the first place, it’s relevant, and it’s a great way to automate some useful feedback for student learning.

John: You have a chapter in this forthcoming book on developing as an online instructor. Are there some general suggestions that you can give to faculty who’d like to improve and develop new skills or improve skills as an online instructor? Besides buying the book, of course. [LAUGHTER]

Flower: Right. Great question. Again, what this comes down to for me is that it’s just new. It’s just new for a lot of people. And to be honest, I suspect that many online faculty didn’t really set out to be great online faculty and many faculty are not finding the experience quite as rewarding as they might find the classroom experience. In fact, I have some data to back me up on that. The 2017 survey of faculty and information technology from EDUCAUSE Center of Analysis and Research found that of over 13,000 faculty respondents, 91% said that they don’t prefer to teach online. 9% said, “Great, I love to teach online.” That’s 91% of us who would rather teach anywhere else. [LAUGHTER] So how can we cultivate that joy, that buzz that we get in the classroom? We love teaching. If we didn’t, we wouldn’t be doing it because we don’t get paid enough. How can we cultivate that for ourselves? Now a barrier or a common challenge is time. Who has time to go and learn how to do a whole new skill? It’s different than teaching in person. But there are, again, small things that we can do to increase our awareness. One of the most effective things that you can do as an instructor is to seek an experienced and a thriving online instructor and ask to shadow that class. Ask to be added into that class shell and just observe. How does that person interact with students? What are the structures? What is the teaching? What happens while the class is in session? That can be hugely impactful, it’s usually free, [LAUGHTER] and faculty can invest the amount of time that they have. In fact, this is how I first got started with online teaching over 10 years ago, is before I was going to teach a class. Luckily, this offer was made to me the semester prior to my first online class, just to observe another class and see what happens in there. Simple structure could easily be set up for faculty who are scheduled to teach a new class in the Fall, have them observe or shadow a class in the Spring or the Summer, and yet an often overlooked solution. Certainly there’s lots of online resources. There’s podcasts like this, there are blogs that people are writing about innovative things that they’re doing, but sometimes just finding a thriving online faculty to interact with, shadow, observe, be mentored by, can be the most effective way to learn how to do this better.

John: I even sometimes encourage faculty to join a MOOC, because often you can find some interesting practices there that scale without necessarily requiring much effort on the part of the instructor.

Flower: That’s right. That’s one of my other recommendations and I hope I haven’t given all of the book away here. [LAUGHTER] But one of the other recommendations in that chapter is just to take an online course in whatever form that you can. Whether it’s a MOOC, a lot of organizations like the Online Learning Consortium, Quality Matters, offer online professional development opportunities for faculty. Even if it’s not about teaching online, just go take a class that is online. Or maybe personal interest. Sign up and take Spanish online. And having the experience as an online student is hugely impactful to help you understand what your students are going through. Even as a faculty if you’re taking a course and you’re reading the instructions going: “Now, what am I supposed to do with that?” Immediately, you have much more clarity about what your students might be experiencing and then you can take steps to address those kinds of gaps or areas of concern that might be in your own class… that you may not have previously seen before.

Rebecca: I think the recommendation of taking a course outside of your normal domain or area of expertise is key because you’ve got students who are in an environment they’re not familiar with, with a topic they’re not familiar with. And so to kind of simulate that, I think is key.

Flower: Right?

Rebecca: I know I’ve done that in the past and it’s like, “Oh yes, I forgot what it was like to be a beginner.”

Flower: Absolutely. In fact, I had a really interesting process or experience this past fall semester where I was supporting a redesign in a large cap biology class of liberal studies—or general education biology class—large enrollment. My background is in English literature… the humanities. I don’t think I ever took a hard science class in college because I did an honors program where we could do more sort of ethical concerns related to science. But I went to that class frequently throughout that semester and I clearly remember the first day. 240 students and me and I was sitting in the lecture hall with the students and it was just very, very impactful. Putting me in a situation that was foreign to me—I don’t teach large cap classes, I don’t know a thing about biology—I do now, I know a little more [LAUGHTER]—But I was a novice learner in a very foreign environment and that’s what our students are in our online classes, which is really quite anxiety producing if you think about it. Going into an unknown space, not knowing what’s expected, you don’t know how to get ahold of your faculty member a lot of the time. So just being intentional about helping students be more comfortable and more at ease in our online classes—be more available to them—can make a big difference. And again, you get that insight differently when you choose to place yourself in a situation where you’re a novice, and you’re not really sure what to expect. That’s a great point.

John: Are there any other topics that we should address that we haven’t raised yet? Anything else you’d like to emphasize?

Flower: You know, really only one thing comes to mind and that is an insight that I had literally this past week, which is that I feel like sometimes online faculty—myself included—have somehow developed the notion that we don’t really need to talk with our students. And let me explain what I mean by that. Again, I’m teaching an eight week—it’s an accelerated graduate level course right now—I’m busy. My students are busy. And on a whim a couple of weeks ago, I said, “Well, I know you have this assignment coming up by Sunday night, I’ll be available on Saturday between the hours of 1 to 5pm.” I don’t like to work on Sundays. I tell my students that if you want to just pick up the phone and call me on Saturday, go ahead. So that weekend, I did. I had a student who called me and she was a chatty Cathy, and we stayed on the phone for quite some time, but she got a better understanding of the assignment and how to be successful. Well two weeks later, which was this past weekend, it was my daughter’s 11th birthday and I was right in the middle of finalizing all the food preparation and everything else. And lo and behold, there’s my phone ringing and I can tell that it’s not a connection of mine. And I went, “Uh-oh, it’s one of my students,” [LAUGHTER] because I had said Saturdays 1 to five and that same student who had called me a couple of weeks prior called and we had a great conversation. 15 minutes, I was able to keep chopping the carrots while I was talking with her. And it just occurred to me, that wasn’t really a convenient time for me personally because I was doing that final party prep, but so what? The student needed help in that moment and just taking the time to answer the phone and talking through a couple of quick questions, it was helpful for her, and it just got me thinking about how, you know what, I don’t think a lot of us really talk to our online students, like, literally talk on the phone. I know some faculty have the online office hours, I know people are using video conferencing systems, I’m available, but one of the things I’ve started doing is just saying, “Hey, if you have a quick question, just call me. We’ll talk it through.” And sometimes a five-minute conversation can ease that student’s anxiety and answer a few questions. This happened to me again yesterday where a student was like, “Before I submit tonight, can I please just check in with you?” I talked with her while I was commuting to campus and it’s just a way of talking person-to-person, humanizing the online learning experience. But like I said, I think somewhere along the line personally I had formed this opinion that we don’t actually talk to our online students. And I don’t know why that’s a perception because if you’re teaching in person you talk with your students. If there’s somebody who has a question after class, you stay a few minutes after and answer those questions. But I think for online faculty somehow we’ve missed that connection and it can be a powerful and so simple solution to helping our students thrive and succeed. I think faculty and students both overlook some of those simple solutions. It doesn’t have to be a long, tedious, written interaction in a discussion forum. It could be a phone call, and so much can be conveyed through the tone of voice and emphasis, just as I’m doing here today. And as we all do, when we’re teaching live. Just picking up the phone and calling the students or inviting them to call you. Simple, powerful.

Rebecca: I think you’re pointing to something that I know I’ve experienced even though I don’t teach online regularly. It’s just online communication is always written and it feels daunting and it feels really time consuming. And it feels like, “Oh I got to sit down and dedicate time to do this.” So it’s nice to be reminded that there’s other ways to respond.

Flower: Just in my own work somewhere along the line, I forgot about the phone in my day-to-day job. My full-time job is as an instructional designer and it seems like we never just pick up the phone anymore. It’s always email. And as you said, it just takes longer, especially if you have a little bit of confusion and you’re going back and forth on email. I literally in the past few months, I’ve just remembered how to pick up the phone and call somebody. Have a five-minute conversation, you get your questions answered. And just reminding ourselves of the importance of real- time interactions sometimes, and moving away from the requirement that everything needs to be written all the time. I’m a big fan of video announcements, I do that all the time in my online classes and again, the reason I do it is because tone of voice, inflection, emphasis, and funny faces sometimes, or just emphasis where I might just kind of widen my eyes a little bit to explain that, you know, “This is really important. Pay attention and focus.” Just finding these other forms of communication apart from writing can make a big difference in the online learning experience as well.

John: We always end with the question, what are you doing next?

Flower: Well, I’m not quite done with this project. [LAUGHTER] So I’m wrapping up this book development. But what’s really making me passionate now is to really focus on being a crusader for online education. It’s undervalued. It’s under-supported. I know that faculty don’t see the joy of teaching online and I know that students approach it the same way like, “Well, I have to get this degree and I guess this is a convenient way to do it.” I just want to advocate for how online learning and teaching can be impactful, can be rewarding, and joy giving, and you don’t see that reflected even in the coverage of teaching in higher education. Most of the time, the focus is on what we’re doing in the classroom and that’s so important, but there’s a big gap. What are we doing in our online classrooms? I just want to move into that space and encourage people to think about how they teach in person, and how to do those things in their online classes in ways that are not so daunting that they never get around to it.

Rebecca: This has been really great. I’m looking forward to picking up your book and maybe thinking about teaching online. [LAUGHTER]

Flower: That’s right. And if you don’t mind, Rebecca, I’ll just pick up on that, which is that again, I think a lot of faculty don’t say, “Hey, wow, what a cool opportunity. I totally want to teach online.” For many faculty it’s a daunting prospect, “I don’t know how to do this.” But it can be a really great way to reinvigorate your teaching—to find new ways of finding and addressing those challenges. Keep in mind institutions have the support professionals, instructional designers and such, who can help if you’re thinking about moving into online teaching. Talk with some of those faculty support folks, talk with your colleagues, and jump right in. It’s more fun than a lot of people think.

Rebecca: Well thank you so much for joining us today.

John: Thank you.

Flower: Thank you. What an absolute privilege and honor to be here. Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, and Jacob Alverson.

70. Dynamic lecturing

Posted in active learning, attention, higher education, learning, lecturing, pedagogy, peer instruction, teaching
February 27, 2019

The lecture has dominated instructional practice for several centuries. In the last few decades, though, the lecture mode of instruction has often been criticized by advocates of active learning approaches. In this episode, Dr. Christine Harrington joins us to discuss evidence on the effectiveness of lectures and how we can create lectures that better support student learning. Christine is an associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing and several other books related to teaching, learning, and student success. Christine has been the Executive Director of the Student Success Center at the New Jersey Council of County Colleges.

Show Notes

  • Dr. Christine Harrington Associate Professor of Educational Leadership at New Jersey City University (NJCU) Previously served as Executive Director of the Center for Student Success at the New Jersey Council of County Colleges (NJCCC)
  • Todd Zakrajsek – Co-author of Dynamic Learning
  • Dr. Neil Bradbury – Professor of Physiology and Biophysics at Rosalind Franklin University of Science and Medicine
  • Bradbury, N. A. (2016). Attention span during lectures: 8 seconds, 10 minutes, or more? Advances in Physiology Education,40(4), 509-513.
  • Richard Mayer- Professor of Psychology and Multimedia Learning at University of California at Santa Barbara
  • Mayer, R. (2019). How Multimedia Can Improve Learning and Instruction. In J. Dunlosky & K. Rawson (Eds.), The Cambridge Handbook of Cognition and Education (Cambridge Handbooks in Psychology, pp. 460-479). Cambridge: Cambridge University Press. doi:10.1017/9781108235631.019
  • New Jersey City University Ed.D. in Community College Leadership program
  • Dr. Harrington’s book, Dynamic Lecturing can be purchased from Stylus Publishing and listeners can use promo code: “ETS20” (excellent teaching series) to receive a 20% discount on the Dynamic Lecturing book or Dr. Harrington’s other book, Designing a Motivational Syllabus

Tea For Teaching episodes referenced

Student Feedback tools

Transcript

John: The lecture has dominated instructional practice for several centuries. In the last few decades, though, the lecture mode of instruction has often been criticized by advocates of active learning approaches. In this episode, we examine evidence on the effectiveness of lectures and how we can create lectures that better support student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Christine Harrington, an associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing and several other books related to teaching, learning, and student success. Christine has been the Executive Director of the Student Success Center at the New Jersey Council of County Colleges. Welcome Christine.

Christine: Thanks.

John: Welcome.

Rebecca: Actually it should be “welcome back.”

Christine: Thank you for having me again. I’m looking forward to a new conversation.

John: Our teas today are:

Christine: I am not today.

Rebecca: I am drinking Lady Grey.

John: And I’m drinking ginger peach black tea today. We’ve invited you here this time to talk about one of your other books on dynamic lecturing. It’s not uncommon for people to argue that lecturing is ineffective, but it’s still one of the most common forms of instructional delivery. Why is lecturing so often discouraged?

Christine: This is an interesting question. I’m not really sure where this stems from but I think that the push for active learning started to pit the lecture against the active learning approaches, the collaboration. And I really do believe that there is significant value in both approaches and I’m not sure why it became an “either or” kind of situation. But unfortunately, it really has, and one of the reasons that I decided to write this book was because the lecturing is the most common method, you know, it’s still the tried and true method of faculty rely on all of the time. And yet, there are very few resources or support to help faculty be effective at lecturing. If you go to a professional development conference, a teaching learning conference, you’re hard pressed to find a session—unless I’m there, I guess – on lecturing. I actually haven’t seen another one yet—so there really are not any resources for faculty on this and it’s not really fair that it got such a bad reputation, because there’s not validity in that thinking.

Rebecca: So that leads to a good question. What is the research on effective lecturing?

Christine: So in terms of what is effective, I think one of the first questions that we need to ask ourselves is, “Well what are we talking about?” because much of the research that exists out there, if you’re going to research a boring, monotonous lecturer… who’s got the old yellow papers and just is reading and not even looking at the students and engaging them, or you’re talking about a lecture that is dynamic and the presenter is passionate and excited about the topic. We’re not always measuring the same concept. And that’s true in group work as well. So really, when we talk about teaching and learning practices, and we try to look at the literature about what works and what doesn’t, it’s very complicated because of the complexities associated with the teaching and learning processes. However, there is research out there that does support the lecture. What much of the research really points to is that the lecture is most effective for novice learners. For students who have very little background knowledge in the subject matter, they need to have someone who’s an expert present that information in a way that they can take it in so that they are developing that expertise and hopefully learning that content. If you ask them to just engage in what’s been called inquiry-based learning or case-based learning, the research really shows that that approach is not as effective if you don’t have the background knowledge. So what in essence happens is that well-intentioned faculty and teachers use that approach and end up wasting a lot of precious learning time, because the students in the groups aren’t equipped yet to be able to tackle those high-level questions and to figure it out without the guidance. There’s some interesting research out there on the importance of it being done well, but also making sure that the lecture is done before the group work is done, so that the foundational knowledge kind of sets the stage for some of those more what we call traditional active collaborative learning experiences. So if you want to look at novice learners, you’re going to see a strong correlation between the lecture working and student success outcomes. And then there’s something called the expertise reversal effect. What happens there is the more that you know about the subject matter, the less helpful the lecture is and the more helpful those more active collaborative learning group exercises are. I still today learn from TED Talk or a great lecture. But I’m also going to really get a lot of value—especially in my area of expertise—out of dialoguing with other experts and engaging those conversations because I have a strong foundational background. So it’s kind of interesting when we think about “Does the lecture work or does it not work?” it depends on who you’re talking about, in what subject, for what purpose, under what conditions. So it’s not as simple as a yes or no, but I will tell you that there is a significant body of research that says the lecture is effective. It’s not that we need to throw the baby out with the bathwater, we just need to make sure we’re using that method effectively.

Rebecca: One thing that you hinted at, Christine, is that with lecture from an expert, there seems to be an opportunity to ask questions, which is why just reading about stuff maybe isn’t always sufficient. If you have that opportunity to ask questions of the lecturer or am I reading into something that I shouldn’t be reading into?

Christine: No, I think you’re right on track there. And actually, I think that’s part of the reason why live lectures are more effective than online lectures… because in a live lecture, you as the expert get to see the puzzled looks on the faces. So even if they’re not offering up the questions, you can say, “I think I need to throw in another example.” And students can ask for clarification along the way, so that you’re not just going on without them following and getting the concepts that you’re discussing. Lectures are not one-way kinds of teaching methods, but they really are two-way processes. Even though it seems heavier on the expert delivering to the student, the student can be very engaged and also asked to be engaged through questioning and other activities during the lecture.

John: And it’s not necessarily an “either-or” condition… that you can embed active learning activities in the lecture, right?

Christine: And that’s exactly how I would define a dynamic lecture: that you would incorporate what I’ll call brief active learning breaks into the lecture because we don’t want to just talk at students for a really long time… and let’s face it, as faculty, we could do that. And quite honestly, it’s the easiest thing to do, especially when you know your subject really well. And you can just walk in and start talking about your subject matter. But that’s not necessarily going to lead to the highest levels of learning. So when we look at what our learning outcomes are for the course, we need to structure our course in a way that’s going to help students learn and achieve those learning outcomes. So in order to do that, it’s really powerful to build in some brief reflective opportunities for students to engage with the content. And Mayer has really done a lot of work in this space, saying that it’s a cognitive focus that needs to be emphasized more so than an interactive focus. So the breaks don’t always need to be a social or group break. It doesn’t have to be a partner activity or a small group activity. Some of them can be independent activities. The key is that once you get a certain amount of information, you need to process that information and digest it and there are a variety of techniques that you can use to help students really learn that content. One of my favorite studies—which is a little bit disturbing, and in fact, I always say it hurts our feelings when we find out what it actually says—there’s a research study that compares students who had no pause in their lecture. So that the professor just kept talking the entire time sharing all the expertise in an effective strategy and an effective way. And then another condition where the instructor paused three times for two minutes each. So we’re talking about a total of six minutes during the class period that the professor stopped talking and during that time the research study was set up such that the students had to do what’s called a “compare and share” of their notes. They had two minutes to take a look at what their partner wrote down, fill in any gaps that they had, and engage in that. And at the end of the day, what they found out was that the students in the pause condition really outperform significantly students in the no-pause condition and it sounds exciting at first until you realize that “You mean, If I stopped talking for six minutes my students learn more?” like that kind of hurts their feelings. [LAUGHTER] But it’s true. Sometimes especially at the end of a class, we start talking faster and faster and trying to give more information as if that’s going to lead to high levels of learning. We need to keep in mind that students need time to digest and process. And it’s really important that we strategically and intentionally build in those opportunities for our students during our lecture.

John: And as they’re building their own mental models, just giving them a little time to process it and to compare notes takes advantage of peer instruction. You’ve got some reflection going on there, and you’ve got a little bit of retrieval going on there. So there’s a lot of evidence-based strategies that are embedded in that basic activity.

Rebecca: I think it also helps students who might start feeling panicked because they didn’t get everything in their notes… give them a second to maybe fill it in and then they don’t feel so panicked and they can focus again. If you get anxious because you feel like you’re behind, it’s really hard to focus.

Christine: Absolutely, and it is human nature for us. Our attention wanders, no matter how wonderful a lecture is, life is happening to you. Sometimes it’s easy to have mind wandering happen and you don’t want students to be penalized for that happening for a brief moment. So giving them opportunities to get back on track and refocus I think is really important. And John, as you mentioned before, there are several of these great learning breaks that you can use that are very, very much grounded in the research. You mentioned retrieval practice, for instance and we all know about the testing effect and how powerful that is. And I think we focus primarily on taking tests and encouraging quizzing—and that’s definitely an important component of what we should be doing in the way that we structure our classes so that students get to benefit from the testing effect—but quite honestly, we don’t have to grade everything and not everything needs to be called a quiz or a test. But if you ask students to do the classic, one-minute paper, for instance, that really is retrieving the content that they just learned and giving them practice at doing that will make it more likely that they transfer those actions into their world outside of the classroom, so that when they’re studying, they also engage in those same kinds of evidence-based practices.

Rebecca: I think a lot of times when you hear good lecturing, people think about TED Talks and maybe some of the storytelling and things that happen or the visual strategies that are used in those talks. Are there elements of those that come into strong dynamic lecturing in the classroom? Are there things that are missing from those that we should be thinking about in our own classroom?

Christine: I think you talked about something that’s really important: Storytelling. For ages and ages storytelling has been a way that we have learned and I think we have all been on the edge of our seats in a lecture that was based on storytelling. We want to know what’s going to happen next. And the lecture really can become the story of our discipline. And we can weave in personal stories and examples and things to make the content come alive for our students. It really puts it into context for them and helps them identify and see the relevance of the material that’s being discussed in their real world application. So to me, I think storytelling is probably one of the reasons why lecturing is so effective if it’s done well and you are weaving that in and mesmerizing your students with the chapter content. The the key element is doing that effectively. So I would say, “Absolutely, that is great.” Although TED Talks, when you think about them, obviously they’re online and they’re one directional still. So stories can be more just told by the storyteller and not have audience participation or they’re stories where you think back to your days in elementary school, where the teacher would pause and ask for you to get engaged in the story and maybe predict what would happen next, and to think about examples from your own world. And I think that’s what we can do in the live lecture, sitting there with students face to face we can give them those opportunities to do a prediction. “What do you think this research study is going to find? What is the key finding going to be? You heard how the study was set up, what are the implications of that?” You’re getting them to think about it and to be really engaged in the story and participants in the story, I think is one of the areas where we can as faculty enhance the effectiveness of the lecture.

Rebecca: How does a faculty member learn to be a better storyteller?

Christine: I think that some of that’s natural, I think some of us are more naturally better storytellers than others. But one of the strategies that I suggest to faculty is for you as the expert to take a step back and to think about what are the key elements of your story? Or what are your big ideas of your lecture? Because it’s all natural, and it all flows to you as an expert, because you know, this material so well. But for your students who are getting exposed to it for the first time—or maybe on the second or third time hearing this content—they don’t know what the big ideas are. So I think that one of the strategies that faculty can use to become better storytellers is to almost map out what are the many chapters in this book that I’m telling, right? Who are the characters and the main players? What are the big theorists that we’re going to talk about? Or, what are the big researchers that we’re going to discuss? And what are the key variables—or the factors really—that are going to comprise this story? I think one of the most helpful things you can do as a faculty member to strengthen your lecture, is to step back and identify what are the three big ideas or major elements of this lecture for today. And if you’re able to do that, and then clearly communicate those to the students in the beginning and throughout every time that those big ideas are getting introduced, that will really help students hone in on the most important elements versus getting lost in some of the details of the story.

John: That’s one of the ways it helps reduce the cognitive load of the students so they can focus on those key points, without getting lost in the details that they’re not quite ready to incorporate into their models. What do we know about student attention and how we can keep student attention during a lecture? I know sometimes when I have a large class of 3 to 400 some odd students, sometimes their attention will wander. What can be done to try to keep that, a more constant level of attention and focus?

Christine: Well I don’t know what you’re doing, John, I don’t have that problem. [LAUGHTER] No, only kidding. [LAUGHTER] Attention during a lecture as an interesting topic. You know, I’ve been going around the country doing a lot of presentations on dynamic lecturing. And as you know, my colleague and co author, Todd Zakrajsek speaks on this topic quite a bit. He said, “Christine, try this out when you go present. I want you to ask the audience how long a student can pay attention during the lecture.” He said, “Just throw the question out there and see what they say.” And I have done this and he has done this, you know, we’ve compared and shared notes of ourselves and immediately people are throwing out numbers. It doesn’t take very long at all, the numbers usually start out like and hover in that 15 to 20 minute range —although I get, you know, some wise guys in the audience a 90 seconds and some others who are more optimistic, saying larger amounts — but immediately they’re throwing numbers out. So I said to Todd when I was writing the book, “I don’t know what’s wrong with me, but I’m usually pretty good at finding the research. I can’t find the research on this topic.” I said, “I heard this over and over. I’ve been to professional conferences where the presenter has said 15 to 20 minutes is all that folks can maintain their attention for, so you’re going to have to change things up if you want them to stay attentive and I couldn’t find it.” And we had this really deep conversation about it. And he says, “Now go and ask those audiences, how long would a student be able to pay attention if they were reading” and immediately you hear silence for a minute, because they’re processing and they’re trying to decide how long it would be for reading. So you don’t get answers right away. What almost always happens is somebody says, “It depends” pretty quickly. Or if they do say any numbers — and they do sometimes — they’re usually saying it in a much more tentative voice, lower, not as loud and enthusiastically, and not with confidence. Todd and I will say to our audiences is, “Why is that? Well doesn’t it depend for both the lecture and the reading?” It depends a lot on whether I liked the book, if it’s written in a way that I can understand it, am I able to take it in — if I’m reading a really dense, heavy textbook in a subject I don’t know and I don’t understand what I’m reading — I’m going to be done with that in a couple of seconds. But if I’m reading something that I really enjoy, and I have some background knowledge on and I’m able to take in and I care about, then I’m going to read for hours at end. I mean, you could be on the beach reading all day long, right? The same is true in a lecture. There really is no magical number about how many minutes a student can pay attention to. It depends on how much they care and their personal variables as much as the professor variables. Obviously, the attention span would be longer in an interactive, engaging presentation versus a monotonous boring presentation. The folks who talk about the 15 to 20 minute mark, it’s really not based on the research. However, there is some mind-wandering research that does say that students report higher levels of mind wandering in the second half a class as compared to the first half of class. It seems that there is a small drop around that 20 minute mark, that may have been where it came from. But, I have to be honest with you, some areas I can tell you those robust data or we have like hundreds of studies — this is a handful of studies, we don’t have an enormous body of research in this space — but the good news is, is that it started to get people thinking about: “I guess, if we need to keep students attentive, then we need to switch things up.” And I don’t think that was bad practice. Despite it not being founded on good research, I think that the result was probably positive if faculty were in fact incorporating some active learning break that could be advantageous for student learning.

John: Actually, this is a topic we talked about in an earlier podcast. In Episode 16, one of the people we interviewed, Neil Bradbury, had written a paper on attention span during lecture because he was faced with the same thing. He kept being told that you should keep your lectures or videos to 10 to 15 minutes or so at the most. And so he went to try to find the research and he published this in the Advances in Physiological Education a couple years ago. He found that it was based on a study that was really just looking at note taking and it was done in one class and it was based on an analysis of students’ notes and how much they wrote during different periods of the lecture, which had very little relationship necessarily to the importance of what was being discussed and so forth. And that became cited over and over again in other studies, and then people just started repeating it without ever seeming to go back and analyze that. And I think there was another podcast where someone had looked at attention on videos, he was looking at how effective videos of different links were on student learning — it was in chemistry, I believe. There really isn’t much research on a student attention span. And that obviously will vary, as you said, with the quality of the presentation. The students are willing to spend hours watching movies, we don’t see them walking out or starting to chat with their friends 10 or 15 minutes in, normally.

Christine: Right. And I think that some of the online video research that people rely on is sometimes is for non-educational purposes, so they’re looking at the attention span of someone watching a video from a marketing perspective. But when there’s no grade attached, we’re in a different situation. Hopefully we’re with an audience that has some motivation, they’re in the class, they’re in college, so their motivational level, I think, is very different than a consumer. I think it’s problematic that we’re trying to bring all of this really heavy, deep content into these like very brief news clips. But don’t think that this is the way that students are going to learn best. I remember I was working with someone and they were convinced they had to be two minutes or less. I’m like, “What can I accomplish in two minutes or less?” I mean, I can give you a quick news flash, but if you want to have deep learning, we’re going to have to have a deeper conversation. And I suspect you’re going to want additional examples from me, and you’re going to want me to share the relevance and that will actually help you. So, I think online videos are not as engaging as in-person videos. We probably do need to have them maybe in shorter chunks, but the key is is trying to bring their attention back and your initial question was, “How do you maintain their attention throughout?” And one of the strategies I think that we’re all very aware of is that we need a hook or something at the beginning of a presentation or beginning of a class. But we don’t really think about the hook throughout the class. So I advocate for faculty to go back to those three big ideas that I asked them to identify and identify a hook or an attention grabber before you introduce each big idea. It can vary, I get kind of silly in my classes sometimes, and we’ll use hand gestures and things of that nature. But for the faculty member who’s not comfortable doing that, you can do something as simple as saying, “Here’s big idea number two coming your way,” right? Because it doesn’t have to be that complicated. So I think the idea is that our lectures just like textbooks are filled with more important content and less important content and when we’re talking about the more important content, the chapters bring attention to that with bold headings and things of that nature. What are we doing in our lecture to help them see? Where is the bold heading of our lecture? Do they get to see those subheadings? Can they figure that out or are we in a little rabbit hole of detail somewhere… that they don’t necessarily even need all of that information?

John: In one of our earlier podcasts, Alex Butler was using an example where there were certain key big ideas in his class and he used images on that. And he put those images on whenever there was an application of those big ideas. And that sounded like a really nice application continued over the whole semester, not just within a single lecture, even.

Rebecca: I couldn’t help but hear as you were talking, “accessibility, accessibility, accessibility.” because structured content is one of the biggest themes of accessibility — or one of the biggest principles to make things accessible digitally — but you’re talking about the same exact concept in a lecture. What is the skeleton or the outline of what we’re talking about so that people can kind of fill in the blanks? And sometimes you have to make that obvious to someone who doesn’t have the expertise that you have because it’s obvious to you what those are.

John: To develop the scaffolding that they need to make sense of it all to fit it all together.

Rebecca: And an outline is like a scaffolding.

Christine: That’s excellent. Yeah, I love it. That’s an excellent point. That’s terrific.

Rebecca: I was wondering, Christine, if you could talk us through one of your lectures, one of your classes. What does it look like? What does it feel like?

Christine: Sure. So I begin class with an activity called “dusting off the cobweb.” So the first thing that I do is, they know as soon as we walk into the class together, they got to put their books away, their notebooks away, and they need to just rely on their brains to engage in practice retrieval. And the question on the table is, What did we talk about last class? So they have a minute and a half to begin that exercise. And they’re talking with a partner and they’re trying to remember what they recall from last class. After about a minute and a half or so then I have them open up their books and their notebooks and fill in any gaps. So they’re going to continue to talk about What did we talk about? And at this point you hear, “Oh, my God, I can’t believe I forgot about that” and I’m sitting out there thinking, “Yeah, I can’t believe you forgot about it, but I’m glad you’re remembering now.” So about three minutes or so passes, and then what I do is I’ll randomly call on students. I’m not a fan of randomly calling on students if you don’t give them processing time… talking about accessibility issues and students with disabilities and processing information in a different way. But after I gave you three minutes to do what I call a fairly low-stakes engagement, I think you’re all fair game. So I’m going to call on you and you’re going to remind us of what we talked about from last class. So now we spent about another five minutes or so recapping some of the highlights. If they don’t mention something that I think is particularly important for us to refresh on, especially given the new content that we’re going to do, then we do that. So I always begin with this because what that does is it activates their prior knowledge so that they’re ready to take in the new information and the new information will be easier to learn because it can stick onto that previous knowledge that they just learned from the last class. So that takes about, you know, five or seven minutes or so. The next thing I usually do in my lectures, I shift to a reading assignment so they usually have some kind of reading assignment — might be a journal article, might be the chapter — and they have questions that they need to answer. This is another activate their prior knowledge kind of activity and holding them accountable for the reading and learning outside of class so that I’m not having to spoon feed everything to them. So now we spend another 10 or 15 minutes where they go over those questions, and this will be with a different partner, and I actually go around the classroom and do spot checking of their assignment while they’re engaged in those activities and I grade them — they’re, you know, low stakes kind of grades, but I grade them nonetheless. So it keeps them on track and it really keeps them focused and interesting that you mentioned that the visual image before because sometimes I’m asking them to do their reading assignments in picture format, like I want them to use either SmartArt or graphics or images because otherwise I find that they’re trying to like copy content that they don’t get. So I’m trying to get them to digest the content as best they can when we’re working together. So then when we’re done with that, now as I spot check, I can see which topics seem to be the ones that they got pretty well. And I still will go over them, but in a much briefer way versus the topics that I could see that they might have been struggling with and they had questions like, sometimes I’ll be walking around, it will be like, “Dr. Harrington, I really didn’t understand this question” and that might be a theme. So then that’s the one that I’m spending a little bit more time on. It’s kind of a very modified version of the intro teaching. It’s not quite exactly following that model but I’m using that reading activity to kind of guide the lecturing and then during the lecture, I will identify what my three big ideas are and I usually do some kind of gesture. So I’ll introduce it with like a gesture about whatever the topic is, get them really excited. I get a little loud and excited, “It’s big idea number one time!” and then I go into the content and then I always have a practice opportunity after it. So after every big idea — and that varies, it might be as simple as a turn and talk or one-minute summary. Sometimes I put them into smaller groups to do like a case study or to develop and answer Socratic questions related to the content. So that piece will vary depending on the nature of what it is that I’m talking about — and I’ll repeat that through the next two big ideas. And then at the end of class I usually will do a very quick — it might be only five minutes — like a preview of the next chapter to make it a little bit easier for them to read. So I’ll highlight stuff, I know that a content’s coming up, let’s say it’s the learning chapter in psychology. I know classical conditioning is often challenging for them to wrap their hands around, I might give them an example and expose them at least to some of the vocabulary and the language that’s going to be in the chapter so that they’re all set and ready to roll with the reading assignment for the next time. That’s kind of what a typical lecture would look like for me.

Rebecca: Thanks. I think a lot of times we often hear some best practices but don’t really take time to think about how that actually plays out throughout an entire class period.

Christine: Mmm-hmm. In the back of the book, you’ll see that there are lots of forms that I have created for faculty and one of them is the sequencing. So, on one of the forms, I’m asking them to plan by identifying: what are the big ideas? How are they going to draw attention to those? What examples are you going to give? What kind of active learning break are you going to give? And then there’s another document that helps them sequence the activity: to always begin with some kind of introductory activity, and then going through those three big ideas and at the end, some kind of concluding activity to get them set for the next learning adventure.

John: Those worksheets and forms that you provide at the end of the book are superb, and that alone is a good reason to buy the book, in addition to all the other wonderful content included in it.

Christine: Thank you.

John: For those people who use multimedia in presentations… who use PowerPoint or visual imagery or perhaps videos, do you have any recommendations on how multimedia could be used or how presentations can be designed to more effectively maintain student attention?

Christine: Absolutely. I think that this is another area that’s gotten the baby thrown out with the bathwater. You see all these sessions, Death by PowerPoint… that PowerPoints are overdone, and sometimes people are rolling their eyes if you’re going to use a PowerPoint. Well again, the PowerPoint is an incredibly effective tool, if it’s done well. If it’s done poorly, it’s an incredibly ineffective tool. We need to make sure we’re using evidence based practices for creating the multimedia, whether it’s slides, videos, whatever tool it is that you’re using. And this is an area that has a robust amount of literature. Mayer has done, I think hundreds of experimental studies on what works best with multimedia presentations and has really found that adding visual images really enhances learning. In psychology, we have a concept called the picture superiority effect, where our memory for pictures is stronger than our memory for word. Something I didn’t mention before in terms of my presentation, my lecture, I always have the PowerPoint as my visual backdrop. It’s not our textbook on slides, it’s really a visual story. It’s kind of like the picture to the storybook. That’s what I view the multimedia presentations to be. So if you look at his research, basically you should have one big giant image and maybe a couple of words associated with that image, and that would be the best PowerPoint slide. And I joke with a lot of my faculty colleagues and I’m like, “Look, I know that I’ve had those slides with so many bullets.” In fact, I’ve heard professional say you’re only to have so many bullets and so many words. Well again, I can’t find any research on any of that. No, stop the bullets, stop the words, go with one big image and just a couple of words. But I said to my faculty colleagues, “I know I too have had slides that had too many words on them. And I’ll tell you exactly why that was the case. It was because I was just starting to teach that class and I didn’t want to forget something. So it was a tool for me, it was not a tool for them. What I learned to do is to have two separate tools, I could still have my additional notes if I didn’t want to forget something. But it is not a visual aid to put it there because it’s not helping them, it’s hurting them.” So to create a powerful presentation really means you need to think clearly about what images are best going to communicate your content, and then to put that up on the board as a visual backdrop, and then take any notes that you need to put it aside and it can be helpful to share those notes. Again, going back to the accessibility issue, if you have notes, why not share them with students? It’s a good idea to share them, but they’re not visual aid for your lecture. Because students can’t do what Mayer called, the redundancy principle. But I actually like to call it the be quiet or shut up principle. We can’t listen and read at the same time. So if you have a slide that has a lot of words, you have students saying, “Hmm, should I be reading this? or should I be listening to what that person is saying? I can’t do both at them same time.” And what usually happens is nothing, so you don’t get anything out of it. If you do have to use a lot of words on the slide, which I think would be a very rare occasion, then you should shut up and let them read it or read it together — I don’t think it matters one way or the other. I’m not familiar with any research that points you in one direction — and then describe it, but don’t talk over your slides. That really is problematic for learning. It’s not even that it’s not helpful, it’s actually harmful to learning.

John: In presentations, instructors will often use some technology to get feedback from the students. What are some effective ways of getting feedback from all of your students or for many of your students, during a presentation?

Christine: Well, I think that the technology tools available today really allow us to engage our students in a new way. So whether you use something like a Poll Everywhere, or a Kahoot! tool or clickers, or asking them to engage via Twitter, there’s so many tools out there that can get students engaged. And I think especially with large classrooms, if you’re trying to lecture, sometimes you don’t know whether they understood the concept that you described. And even asking a quiz question about it, and having them answer it via technology can show you and them whether or not everybody’s kind of on the right track. Now, of course, whenever you introduce technology, you also introduce the possibility of increased distraction. So you have to be mindful. I think that you want to be careful about it. And I think students often respond well to faculty when they believe that you really care about them and are interested in them being successful. So by sharing the rationale and structuring it before you begin to use those tools — and you might even make a joke about it and say, “Look, I’m going to be having you pull out your cell phones but you’re going to also have to put them back, you know, so we’re going to be like on cue here, it’s out, in, you know, like bring them into the class, put them out of the class.” — So I think it is important for us to recognize that the temptation to be distracted is going to be high once we start using their cell phones, because there could be a message on there that they all of a sudden pulls them into a different direction. But the value of getting everyone engaged is powerful and some faculty will even count those to increase their accountability throughout the class and to keep them more focused on the questions if they do count in some way. But I think you have to be careful about that because some students may need more time on task to learn that content. So if I didn’t get it in five minutes, I don’t think I should be penalized. So, it is a tricky process and I think you just need to know your students and what works best in your classroom. But there’s so many great tools out there that really allow faculty to engage students throughout, and engage them in the practice retrieval actions as well.

Rebecca: So one of the things that faculty always want to know is whether or not they’re doing a good job. So how would you recommend faculty get useful feedback about the quality of their lectures and maybe tips for improvement?

Christine: Yeah, I think this is a really important point because I have heard about some faculty actually getting poor evaluations just because they use the lecture. So I think that we have to do a variety of things. First of all, we need to, A: educate our peers, and obviously our chairs and deans and whatnot, about the value of the lecture. And then we need to then figure out well, how can we best evaluate whether or not a faculty member is doing an effective lecture or not? In the back of my book, I also do have a chapter on evaluating lectures and I think that the listeners would probably find it valuable to go through that to see. Because what I basically have done is, taken all of the research-based practices and turn them into kind of self-evaluation or peer-evaluation questions. You can also engage in self assessment as well as the peer assessments, I think both are critical to have you really think about. I think it’s essential that we talk to the person who’s observing us ahead of time so they have context for what’s happening. So to have kind of a pre-meeting that really describes what’s happened and transpired in the class before this isolated lecture that you’re coming into, so that they know any of the story about why it is you might be doing what you’re doing. And also to ask them specifically: “You know, I’m trying out this new brief active learning break, we haven’t done this one before. Maybe we’re going to try asking one another Socratic questions, could you help me be another pair of eyes to see, were people engaged in this? Were they struggling? Were the instructions clear? Were they taking the ball and running with it right away or where they fumbling and not really knowing what to do immediately and needed more guidance?” I think that there are so many strategies, it’s not like there’s a wrong or a right way to do it exactly. I think that the key is, are you integrating the research based practices in a way that supports the learning goals of your class? So really keeping hyper focused on the learning outcome that you’re trying to accomplish that day.

John: So we always end with a question: What’s next?

Christine: Well, I mentioned to you before that I just took on a new position, so I’m now at New Jersey City University. I am a faculty member and also co-coordinator in our brand new (Ed. D.) Community College Leadership program. So as part of this, I mean my primary focus is going to be obviously the launching of this new program, some of this includes curriculum development, and marketing, and recruitment. And I’m really excited to get this program off the ground because we really need to build the leadership capacity in the community college sector, at all levels. So very excited about that. But I do have a new book that I’m going to be writing, I actually just got the contract last week, it’s going to be about the guided pathways movement, which is all focused on increasing student success completion rates. And it’s primarily a community college initiative, although, many of the four year colleges also have been getting into the space, and the book is going to be focused on engaging both full- and part-time faculty in Guided Pathways Leadership. So to get them engaged in this movement, and to really see themselves as leaders in the student success reform efforts… so really excited about that and I have some other potential things in the mix to that hopefully will pan out as well. But lots of great stuff happening for me, so I really appreciate this opportunity and look forward to staying connected with both of you.

John: We really appreciate you joining us.

Rebecca: Yeah, we always have a lot of fun and walk away thinking about a lot of things we can start doing to improve our own classrooms.

John: And you can find Christine’s books at Stylus Publishing, and I believe there’s a discount code available for our listeners.

Christine: Yes, if they put in ETS — which stands for excellent teaching series — 20, they will get a 20% discount on the Dynamic Lecturing book but also the Designing a Motivational Syllabus, and I believe it’s going to work for all the books in the series that will eventually come out. So it will be a 10 book series once all of the books are out and published, but ETS20 would be that discount code.

Rebecca: Well, thanks for joining us. We always have a great time.

Christine: Well, thank you. Always my pleasure, I appreciate it.

John: Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, and Jacob Alverson.

65. Retrieval Practice

Posted in active learning, assessment, evidence-based practices, first-year classes, higher education, learning, metacognition, pedagogy, retrieval practice, teaching, Uncategorized
January 23, 2019

Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, Dr. Michelle Miller joins us to discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Her academic background is in cognitive psychology and her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications.

Show Notes

  • Miller, M. (2014). Minds Online: Teaching Effectively With Technology. Cambridge, MA: Harvard University Press.
  • Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968.
  • Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 1199327.
  • Kahoot
  • Bray, Niki (2018). 43 to 0: How One University Instructor Eliminated Failure Using Gamified Learning. Blog post
  • Retrievalpractice.org
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Pennebaker, J. W., Gosling, S. D., & Ferrell, J. D. (2013). Daily online testing in large classes: Boosting college performance while reducing achievement gaps. PloS one, 8(11), e79774.
  • Pauk, W. (1984). The new SQ4R.
  • Thomas, E. L., & Robinson, H. A. (1972). Improving reading in every class. (a discussion of PQ4R)

Transcript

John: Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, we discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today we’re welcoming back Dr. Michelle Miller. Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications. Welcome back. Michelle.

Michelle: Thank you so much. It’s so great to be here today.

Rebecca: We’re so happy to have you again. Today’s teas are:

Michelle: Well, I’m drinking Coco Loco, which is a blend from a local tea shop here in Flagstaff, Arizona. Steep Leaf Tea. And Coco Loco is a lot like what it sounds like. It’s chocolate and banana. So tea snobs may scoff at my choice, but it’s wonderful.

John: IIt sounds good.

Rebecca: And I think I saw a nice silver teapot that was poured into a green and blue tea mug.

Michelle: Yup.

Rebecca: A nice tall one.

Michelle: That’s what I need.

John: And I’m drinking ginger, peach green tea.

Rebecca: I went with a Christmas tea today. So Michelle, we invited you here today to talk a little bit about retrieval practice. Can you first start with defining that for us and letting us know what it is?

Michelle: Right. So retrieval practice is essentially the act of pulling something out of memory. So that is, in memory research, what we would term retrieval. So something is stored in memory and we want to pull it out so we can actively use that information, have it in our conscious minds and so forth. And so we go through this usually very fast process called retrieval. So retrieval practice is specifically the act of doing it, and we contextualize that with learning. So, when I’m trying to learn something or I’m the process of learning something, to say, “Oh, what was that fact I remembered, or what can I say about this?” When we do that, it produces something that we can call the testing effect. So this is kind of the clearest example… not the only… but the clearest example of retrieval practice in action during learning is when we sit down to take a quiz, take a test or something like that. So all the excitement that’s happened around retrieval practice in higher education, and really in the rest of education today, is around this finding, which has been replicated many, many times: that tests are, as one person put it, not neutral events in learning. When we take a test on something, that has a very powerful effect on our ability to remember it in the future. So, really simplified down to its core, tests help us remember in the future; when we take a test it strengthens our memory. So that’s what retrieval practice is and it can, as we’ll maybe talk about today, take many, many forms in learning settings. And I did want to clarify too, this is something that I definitely don’t want to take any kind of credit for discovering this. This has been around and has been known about for a long, long time. Some of the big names who are associated with this: Jeffrey Karpicke, Robert Bjork. Roddy Roediger, there are quite a few really heavy hitting cognitive scientists and cognitive psychologists who have established this. But there are many, many of us who are out there trying to disseminate this to other teachers around the world so that we can all tap into the power of this. And I have done a little bit of work in this area with my colleague here at Northern Arizona University, Laurie Dixon, who’s another psychologist… and we teamed up some time ago to look at a very practical implementation of retrieval practice in an Introduction to Psychology course that we conducted some years ago and this is a course, you can imagine, where just trying to get students to perform even a little bit better is a big project. So we examined how even something kind of basic… it was very high tech at the time… but just basic web quizzes that came packaged with the textbook. We said, “Well, if we actually assigned students to do these as part of the course, and if they went through and treated these as opportunities to learn, not just assessments in and of themselves, would that have any systematic impact on course performance?” And we found that in fact, there was a significant improvement associated with that. So that’s kind of the landscape of what retrieval practice is, and why we’ve been so interested in discussing this in the psychology of teaching and learning.

John: In fact, I saw you present on that about 11 or 12 years ago in Orlando at one of the NCAT conferences and it convinced me to completely revise how I was giving my classes and it’s made a big difference and resulted in some significant improvements in student learning. Given that we know so much about retrieval practice. Why are faculty resistant to doing this?

Michelle: Wow, that’s a great question, and that is one that I have been really facing a lot these days in my own practice, talking to other faculty members, and disseminating this through some different activities I do in this area. It is easy for those of us who work in this area, cognitive psychologists in particular, but a lot of us who like you heard about this a long time ago. We’ve seen the power of it. We forget that to other faculty, this can be a very off-putting concept. And so it’s really great for us to think about why that is. And I always think that’s really good for me to kind of go back to that and say “Yeah, not everybody is sold on this idea. And there are good reasons for that.” So like with a lot of things that we talk about in teaching and learning, I think that these really break down into two neat categories: there’s the philosophical issues that people have with it, and then there’s more practical and logistical issues with it. So kind of tackling those one at a time. Philosophically, when people say, “Yeah, I understand about the research but this just goes against something that I believe as a teacher or how I want my classes to be. Here are some ways that that can play out. First off is this idea, that I’ve heard in one form or another quite a few times, and that is this notion of superficial learning. So “Okay, sure, there’s a study that showed that maybe people retain something a little bit better. But surely that’s not this deep learning, whatever that is.” That’s a concept that we all want. So do tests and exams… just testing… create just a superficial form of learning? And while, of course, I understand that, and I absolutely applaud faculty for really thinking deeply about that issue, and caring about it. I Here’s the thing… we got to define that. We social scientists, that’s what we do… we have to kind of break things down and say, “Okay, what does deep learning mean?” and I don’t know that anybody has kind of definitively done that. But when I look at that, I say, “Well, this is not just one research study that showed a little improvement in a lab test. There’s quite a few studies that do use realistic types of materials. It’s not all just contrived laboratory studies. Furthermore, there’s also studies that show that when students engage in more quizzing and testing on material that they actually are able to transfer that learning better. And that is a very, very big deal in teaching and learning as a lot of us know is not just getting students to be able to solve a problem in one context or work a concept in one context, but can they do it in the next circumstance? And that very difficult process is aided by quizzing… and to me, what could be deeper learning than learning that transfers? So that’s part of it. Some of it is perceptions around multiple choice quizzes and tests. There’s an assumption too that if we’re talking about quizzing we must be talking about multiple choice questions… and first off, sometimes in larger classes, those are the final assessments… in that Introduction to Psychology course that we studied years back, that’s what the assessments were… so, having students practice in that format I don’t think that we should necessarily dismiss that. And as we can talk about in a little bit, there’s lots of ways to induce retrieval practice that actually don’t involve multiple choice questions. So, there’s a bit of that as well. And something that I’ve talked about with some faculty recently, too, is this baggage around K through 12. And maybe that’s something that’s resonant with you all.

John: Yeah, that’s given the testing effect a somewhat bad name, because high-stakes testing is being used in a lot of what’s going on with K to 12. But that I don’t think is what retrieval practice as you’re suggesting is all about.

Michelle: Right, and I have to be cautious here. I really like how you laid that issue out in K through 12, that there is a reputation problem… and that has happened because of the high-stakes standardized testing policy in the United States. And I got to be careful because I don’t want to represent myself as an expert in K through 12 or in K through 12 education policy. But I don’t think you have to be an expert in that to know that there’s been a lot of same pretty well justified public pushback against over-testing in K through 12. And yeah, I think that we absolutely do have to be aware of that. Students come to us in higher education… that’s a system that many of them have been through and our faculty are very aware and very cognizant of that too. So, nobody’s a blank slate here, not our students, not our fellow faculty. We have assumptions and ideas and experiences about testing that happen. I think those can be addressed. But, yeah, that is definitely another very big barrier. We got to differentiate between high-stakes standardized testing for the reasons it’s done in K through 12, and low-stakes testing and quizzing for learning as proponents of retrieval practice would have it.

Rebecca: Some of the pushback I’ve heard from faculty fall into two categories related to this as well. One is that they assume that retrieval practice is best implemented in 100, 200 level introductory classes instead of upper level 300, 400, graduate-level classes. And then the other area is that paper and pencil tests don’t make sense and all disciplines. And so they assume that a test has to be in a paper/pencil format, which could be online testing, or it could be multiple choice or it could be essay questions. But I think that, from being someone in the arts, like there’s other ways to test beyond that, but we don’t think of those as tests.

Michelle: Right. That’s another great lens through which to look at this issue; that we do need to broaden the definition to draw more attention to this and to make it a more appealing concept. But yes, how can we make it broadly appeal across lots of disparate disciplines? Not only does it not have to be a multiple choice type of exam, maybe it’s not a pencil and paper exam at all. And we as faculty have to think about what makes sense there. You make a good point about the levels concept. I think, these days most of us have heard of, “Well, there’s one particular Bloom’s taxonomy…” which is a wonderful framework for getting us thinking about being systematic about what we’re asking students to do with the information that we’re bringing to a course and trying to do things like align the teaching we do with the assessments that we have. That’s wonderful. However, I think it does ingrain in us that idea that “Well, just knowing things is sort of at the bottom… You sort of get that out of the way, and then we go on to the good stuff.” And from a cognitive perspective, these relationships are much more fluid and much more interdependent, so that yes, absolutely, the higher thinking that is what we want. That is what we should want. Or if we’re in highly applied disciplines (if we’re in the arts, for example), we need students to be able to do things with that information. But they have to have that. So I think it challenges us to think of new ways with that concept as well.

John: One of the barriers I think some people have is they don’t like to grade tests and so forth. But one of the things you mentioned in your book is that the testing effects been known for a long time, but it was really difficult to implement in terms of low-stakes testing, particularly when you’re teaching at a larger scale. But, as you’ve done yourself, and as you suggest in your book, computer technology makes it easy to automate some of this… certainly more easily for multiple choice and free response and similar things. But it makes it a whole lot easier for both students to have multiple attempts at learning something using some type of mastery quizzing and it makes it a whole lot easier for faculty who don’t have to spend all their time grading.

Michelle: Right and that’s absolutely where the practical stuff comes in. So, we’ve worked through some the philosophical objections, that: “No, this does not turn your classroom into some terrible assembly-line concept of learning. It’s not going to create a bad relationship with your students. It’s not going to simply carry on a legacy of bad policy as people perceive it. It’s not going to do those things.” And then, we do get to: “Alright, but what is this going to do to my life as a faculty member?” …and this is important stuff. Those who know me know I’m a big fan of James Lang’s work in his books. One of his more recent books is called Small Teaching and there’s a lot of different takes on it in that book, but it really hits home with respect to “We do have to think about not everybody’s in a position to, nor should we even try sometimes, to just take everything down to the foundation and rebuild.” That we do really need to think about “Do I want to do this, if it’s going to create 800 more questions for me to grade?” Is this a sort of a situation where, “Well, I don’t want multiple choice, so I’m going to have to give these open-ended questions. I’m gonna have to give feedback and I have 200 students, what will happen? If I am going to go with multiple choice questions… well, how am I going to do this? to have to write all of this?” And yes, it’s one of the most powerful outcomes of the educational technology revolution that makes it workable, and scalable in a sense, even with large classes to do these types of things and to bring them in. So, that is definitely a message that I hope faculty think about if they’re on the borderline of wanting to bring in more retrieval practice into their classes.

Rebecca: I’m in a discipline where the multiple-choice questions are using things digitally doesn’t always work for testing and practicing some of these basic things, and there’s not a good way to automatically grade it. But one of the strategies that I’ve used is actually some self grading, which has actually worked pretty well. I just check and I have them write notes about anything that they got wrong. So, it demonstrates that they’ve tried to understand when we go over it, and I give credit based on how thorough those notes are, rather than whether or not they got the question right or wrong. And that’s made a difference in my classes. And had I not come up with that solution, I think I would have abandoned it because it would have been too much work. But it it actually is working pretty well.

John: When we had a reading group on Small Teaching last year, one of the things that was widely adopted by faculty was a very simple form of retrieval practice where they had students at the start of each class reflect back on what they had done in the previous class. And most of them have continued to do that in subsequent classes as well.

Rebecca: One of the other barriers that faculty might raise is the idea that students don’t take low-stakes things seriously, or that they don’t put the same kind of time into it that they might for something that’s high stakes. Can you talk a little bit about how we might help students find value in retrieval practice and subsequently also with the faculty then?

Michelle: Right, so that, ”Well, we can put it out there, but will they do it?” I’ve kind of crossed this philosophical and practical barrier for myself of giving some credit for pretty much anything that I am hoping that students will do. I put the work in to set it up and I do believe as a teacher that there’s reason to believe that will help their performance, that I need to work it into the syllabus somewhere. I don’t think it detracts from learning, necessarily, to say, “Yeah, there’s some points associated with this.” And especially with our students who, for many of us, are going in a million directions at once. They’re juggling jobs, multiple classes, sometimes their own families. So having an incentive in the form of points—having some kind of a payoff—I think, helps them make that decision that this is at least worth the time to do. I think the other thing that we probably can all do more of, and that I’ve done more over the years, is framing and honestly marketing this to students… communicating with them about why. And when students disengage from an activity like this, when they say, “Ah, why do I have to sit down and do this thing? This is just another test. Oh, no.” …really conveying the excitement and the goodwill that we have in setting those things up can go a very, very long way. Of course, a student, if they just look at it inside and they have no context for why this was put into place, they’re going to have them say, “Well, maybe I won’t do that.” But when we can market to students, we can say “There is a lot of research that shows that this is a very, very good use of your time. And hey, you’ve probably taken a lot of tests in your life that were really about measuring or sorting you and figuring out what you know and what you could do. This is a very different kind of test.” So that can go along way and get the C students nodding: “Alright, I get it. I get why she put this assessment right here.” I think a lot of us have hit on the practical strategy too, that the little Easter eggs or goodies that we plant in the form of questions that get re-used on the higher stakes assessment. So most of us will have tests for measurement at some point in our courses. And yes, students really do pick up on it when you use one, two or more of those items that were in, say, the gamified quiz that you ran in class or the reading quiz that they did beforehand. They can see those and say, “Oh, wow, I got feedback on that. I got an opportunity to practice…” and if it draws in a few more students who see it as almost a legitimate form of cheating, honestly…. like a fun and sanctioned form of getting an advance sneak peek at the exam, then great! Then that maybe is an opportunity for them to come in and see that. So there is that. Actually taking it seriously ourselves, not just in the form of saying, “Well, here’s the points I’m going to give you for this,” but spending class time on it. That’s a big bridge for a lot of us to cross, right? Because we as teachers tend to be very focused on “Oh my gosh, class is for covering material. We use all these sort of distance metaphors to talk about what we want to do with our class time. But if I say, “You know what, guys, I believe in this, and I believe in it enough to where we’re going to spend the entire class period before the final exam….” I did that twice last week myself, when running exams. Or we’re going to spend five or 10 minutes of the beginning of every class period doing this, as one project recently published about doing. If you show yourself doing that, and offer them that, I think that also goes a long way towards it. And I guess to just say, well, taking it seriously… here again, what does that look like? What does that mean to different people? And we can kind of a little tongue-in-cheek say, “Well, why do we have to take it so seriously?” Sometimes games and learning can happen when it is presented in a more fun context. So not everything has to be deadly serious or spending hours and hours and hours of stressful time on. There are occasions when a light-hearted approach can be perfectly good and can still get us involved in that really critical activity of retrieval

Rebecca: I can share an example of doing that in my classes. We’ve done design challenges and sometimes we challenge other classes that are happening at the same time. That reinforces some of the basic principles that we think that students should be doing and reminds them of it… and they might work in a team…and then we have a competition. And it’s fun and what have you. And students like those, it breaks up the day, it makes it more fun. And then I’ve also done things where I give class time to do little design challenges in class that might be individual and then they can level up to working with a partner to finish solving a problem or something… and students value that. They recognize that next time they’re trying to do a project on their own,that it’s easier because they’ve had that practice or that opportunity for the retrieval practice. And my students have actually ended up asking for more of those opportunities.

Michelle: Great.

John: Could you go back just a little bit and tell us what you did in those couple of classes last week before your final. We’re recording this, by the way, in early December during finals periods in both of our campuses, but we’ll be releasing it a few weeks later… to put that in context.

Michelle: Oh, okay. Well, I’d love to. …and I’ve been talking about retrieval practice as you pointed out for years, and I still discover new ways to infuse this into courses. And the context for this is my cognitive psychology undergraduate course. It’s a 200 level. So it’s a lower-division course. And it’s about 60 or 70 students. And as you can imagine, it is a bit of a tough sell. For many of the students it’s their first encounter with this side of psychology. It’s not as intuitive as some other areas of psychology. So there’s a lot to learn and a lot of motivation to be done. One of the ways we bring this and in this course is using a technology called Kahoot, that’s spelled K-A-H-O-O-T,, and it’s really very intuitive and functions very smoothly… relatively free of bugs. That’s good stuff. It’s a program for doing gamified quizzes of various kinds. What I did, and at different points in the semester, and then really amped up in the last week of the semester is running these gamified quizzes. And this is something, by the way, that I hit on and got the idea to try based on a colleague named Niki Bray, who’s from Tennessee, and has actually done some really systematic work in reformulating some of her courses around in class quizzing in just really ingenious way. So I saw some of her presentation and I said I’ve got to try this for myself. I went with multiple choice questions. Kahoot does have some parameters… questions do have to be short…very, very short. And to some people that may be off-putting, but you can put together quite a few of these. And so we would put this up on the projector and students have the option of dialing with either their laptop or their smartphone and weighing in on each question. The neat thing about it is it has an algorithm for giving points based on your speed as well as your accuracy and it’s got a little leaderboard so you can actually have a little in-class competition. Now some people who use this do require all students to do it and they actually issue points for performance. Now, I presented it very much as a practice activity… and made it very, very clear because of my philosophy, I’m not going to assume that all students have devices or have smartphones or laptops or that they want to do that. But I said, “Look, remember we’re talking about retrieval practice, guys. So the real meat and potatoes of this is not buzzing it on your phone. That’s fun. But the real benefit of this activity is what you’re doing sitting there in your seat. And you could be doing this with a piece of paper if you want.” And that is what a few students opt to do. They try to answer the questions, they know what they need to go back and review and so on. And it’s nice because it spits out at the end, a whole report that tells me right away… Okay, which questions do we need to revisit? Which ones did students have the hardest time with and so on. That’s one of the things that I just brought in. And yeah, it was a big deal. I sacrifice a chapter of “coverage” so that we would have more time at the end of the semester. But to me, I would rather have students going into the final knowing that they’ve had this retrieval practice and they have a better chance of performing really well. Earning a good grade on this material I care about than honestly cramming in a little bit more mileage in terms of the quantity.

Rebecca: Sounds like fun.

Michelle: You know what, it does really bring a fun factor. There’s been a lot of different variations on in-class polling, and I will admit to this, I actually purchased and am the proud owner of a physical buzz-in quiz device, complete with a whole spaghetti nest of wires and an incredibly abrasive, buzzer sound, and everything. So previous to this, my educational technology did include… I think I could have up to eight intrepid volunteers who would play a quiz game and then I would have to appoint a points keeper and all this… and props are always a lot of fun. So when I say I believe in bringing in retrieval practice, I really do walk that walk. But I will say doing it by a smartphone does allow for more participation, and I don’t have to worry as much about the minutiae of scorekeeping and stuff like that.

John: I played with Kahoot a little bit at my classes at Duke and students have loved it.

Rebecca: We’ve talked a little bit about framing things so that students take the practice seriously. What do we do about the students who just push back, it’s like “This is too much work. This is a lot of extra time…” or that sort of argument.

Michelle: I think that that’s another piece of that barrier to more faculty adopting this, not just the work involved, but realistically, student opinions. Student evaluations matter a lot to faculty life. And of course, we all want to have that wonderfully rewarding semester, not the semester where we feel like we’re at odds with our students. So I do think a little piece of this is we do anticipate sometimes worse and more pushback than what actually happens in the end. So I think we have a fair amount of sort of a dread factor when we go into something new like this. But that said, when students have an issue with more quizzing or more testing, here’s how those come out. I think, first of all, we do have to sometimes separate out the technology aspects of it from the quizzing or testing, per se. So, as John mentioned, this is one of the amazing things that educational technology does. But the flip side of that is, if you’ve ever used technology for education, in any shape or form, you know that it breaks down. And when it breaks down at a big class, that’s a headache for everybody, it’s a misery. So if you’re trying it, and things are not working out, you just got to figure out “Okay, how much of this is the assignment, the activity per se, and how much of it is that wonky quizzing thing that I got from the publisher and it fell apart, or students hanging up when they’re trying to dial in with the poll and fine tuning those. I mean, when I first used kahoot, I decided it would be a lovely idea to put four chapters worth of material into a 40-question quiz. And when you get used to as you know, that they took a long time, I mean, 12 questions can keep you going for a very long time, especially if you’re discussing… and some students got kicked out part way through and then they weren’t on the leaderboard and I could have set the whole thing aside. But really, that was more I needed to get the technology working. So that’s a big piece of it. I think that sometimes it is a perception issue with the timing. Now, those of us who are just all over this as a teaching technique, we like to do reading quizzes before we talk about that material in class. So chapter three is up on the syllabus and your chapter three reading quizzes due on Sunday night before we do that…, and that can provoke a fair amount of confusion and honestly griping with students. They say, “Why did I get tested on this when we didn’t do it… we didn’t cover it in class yet?” And what do you know, that’s another framing and communications issue. Once we know that that is why we’re doing that you get so much less on that side of things. But here too we do also ourselves have to follow along with what we say. If we say you’re doing this reading quiz so that we’re establishing a foundation. we don’t have to teach everything in the class itself. We could spend the time applying. Well, guess what, then you do have to do that. So if I assign you to do the reading quiz on chapter three, and then on that Monday morning I go, “Okay, we’re going to go over chapter three starting at the top. And I’m going to show you all the PowerPoints for all these things that you already read.” Well, yes, then student morale and student support will fall apart at that point.

Rebecca: Those are very good reminders.

John: Yeah, I’ve pretty much adopted this approach all through my class beginning when I first saw you present it. So I have students do the reading in advance, I have them take reading quizzes with repeated attempts allowed on those. And then in class, I have them working on clicker questions, and I and the TAs go around and help them when they get stuck on problems. But there is an adjustment and students, especially when I first started doing it, would generally say, “But you’re not teaching me” …and you do have to sell students on this a bit. One source of resistance is that when students take quizzes, they often get negative feedback. When they read something and they read it over again, it looks more and more familiar. They’ve got that whole fluency illusion thing going and they become comfortable with it, and they feel that they’re learning it… or similarly, if they hear someone give a really clear presentation on a topic, it feels comfortable. They feel like they’re learning it until they get some type of summative assessment where they get negative feedback, and then they feel the test was somehow tricky. But it’s a bit harder to convince them that actually working through retrieval practice, watching videos at their own time and pace, reading material as needed, and then spending class time working through those problems is as effective. I’m getting better at it. But it’s been a long time trying to convince students and I did take a bit of a hit in my evaluations, especially the first few times.

Rebecca: Do you mean learning’s not easy?

John: Students would like learning to be easy.

Michelle: You know, it’s funny, I think almost in a way… see what you think about this idea. But it’s almost a mirror image of our illusions as teachers, right? That I gave a wonderful clear lecture and I assigned wonderful readings and I saw students highlighting them. Therefore, students must have assimilated this knowledge and it must be in there. So I think it challenges our students but it also challenges us as well. It can be quite an eye-opening experience to running something like a Kahoot. And that brilliant point that I gave this great example for…. what do you know…. 7% of the students actually nailed it. And so we can all use a reality check… teachers and the students. And you mentioned re-reading, that’s another one where I’ve had some pretty intensive conversations with other faculty, I’ll kind of say, “Oh, well, and there’s this great research that shows that students tend to re-read when they’re studying and we know that from a memory standpoint, that is really, really ineffective.” And faculty will say, “Whoa, whoa, whoa.I want my students to be re-reading.” Of course, now when they say re-reading, they may be picturing deeply interrogating a text… annotating it… looking at it from a different perspective. And absolutely, that’s a wonderful part of scholarship. But that isn’t what we’re talking about. We’re talking about students re-reading as a study form and mistaking highlighting for deep interrogation. So, just like with the rest of knowledge bases versus higher-order skills. This is another words “both and” it cannot be “either or.” Yes, we want students re-reading in the right ways, but students or teachers cannot mistake that for learning sometimes.

Rebecca: So, let’s say you’ve just convinced all of our listeners that we need to be doing this, how do we bake it into our course designs?

Michelle: Well, I think that, really getting creative with this, and as I talked to faculty when I visit other schools or talk to faculty of my own institution, I just see all of these new ideas all the time about how to do this. Once you do get that critical epiphany of alright, a test doesn’t have to look like a test on the surface… it does not have to be the ritual of “Okay, you’ve got a number 2 pencil that’s breaking and I’m standing over you while you have a panic attack for an hour.” That’s not what it is—that it’s about the retrieval. Once you get that, all kinds of creativity opens up. So I’ve also started sometimes on the very first day with the syllabus quiz. So, especially if you have a smaller class, we all struggle with that “I spent all week writing the syllabus and it’s incredibly important, and then I’m getting questions about stuff that’s on it all the way through the semester.” So, really on the first day, what I do is I take a very light-hearted approach and I divide the students up into teams, just physical teams. Everybody’s got a copy of the syllabus, sometimes I fake them out and I say, “Okay, we’re going to go over the syllabus point by point…” and I say, “No, you can read the syllabus. We’re actually going to do this other thing which research shows will actually help you remember it.” So the task here is that each team formulates a set of questions… you can make just a few. like three… so a little bit of teamwork. So formulate three good questions off the syllabus. And you’d be amazed, they come up with questions that even I can’t answer sometimes without having to cheat. So they talk about it. And then of course, you go around the room in some arrangement and each team gets to ask the other team their questions, and if you really want you could keep score and everybody loves bragging rights for being the team that stumped the other teams and won the most points. So, it can literally start right then. With that idea that I don’t read stuff to you in this class, this is about you. And I’m also not piling a huge amount of work on myself either apart from being the moderator and the MC and having written the thing in the first place. I don’t have to write questions. They’re writing the questions and they’re answering them too. So those are some of the ways that we can do that. I think especially in our larger classes we do want to think about things like peer grading or peer review of open-ended question responses. We do want to take advantage of things like publishers’ test banks to set those up as reading quizzes. And you’d mentioned earlier about “Well, what about this not being suitable for upper-division classes?” I had an upper-division class that just wrap this semester where we had reading quizzes as well. It may have been an upper-division course but it also serve that purpose of “Hey, you take a basic quiz over the chapter on Sunday and that really sets the stage for us to have a more substantive discussion.” All of those things. are ways that we can do this. I think open-ended reflection as well… so tests that look nothing like a test but are still retrieval practice. You’d mentioned about reflecting on what you learned last class period. So this sometimes goes by the name “brain dump.” And I did a bit of this last semester as well. So, this by the way, I do want to credit the great website retrievalpractice.org. So retrieval practice is actually that high profile it has its own website. It’s an absolute treasure trove of ideas. So, with the brain dump the way that I did it, or similar to what it sounds like you did, every now and again we start off class with you writing down everything you remember from last time on a piece of paper. I had students then turn to their neighbor and compare notes and see what they came up with. And I didn’t grade those. This was not a heap of grading for me. But in the end what they did turn into me for accountability and a few points was a very short reflection, just on an index card. So, “What surprised you?” and I review those and they’re very eye-opening but again I’m not there to police or micromanage what they put on their cards. So, that’s retrieval practice too. There’s lots of different flavors that we can bring in.

John: I do something similar with asking students to reflect on the reading in each module we work through. But because I teach a fairly large class, I didn’t want to have to deal with all the index cards. So I just have them fill up a simple Google form. And then I can skim through it and assign grades much more easily than shuffling paper. But it’s the same basic idea and it’s worked quite well.

Michelle: Absolutely.

Rebecca: Do you have any other examples of the kinds of little tests that faculty can run that don’t look like tests?

Michelle: Right? So tests that don’t look like tests. Well, here’s another that was probably a little bit more practical for a small class, and this is how I ran this. It, like the brain-dump exercise the way I ran it, was also very cyclical and very student generated. So we started out each class at the beginning of each week with students generating a set of quiz questions based out of the assigned reading. So we didn’t actually, in this particular example, have pre-quizzes or something like that. But students came in knowing that they can bring their book if they wanted. But they would need to sit down and write for me three questions, whether short-answer or multiple choice. Then I can flip through those and I would really quickly put those together for a quiz that went out the subsequent day. So, we’re alternating between generating questions and then returning to those questions. And then I would have pass out. I told them treat this like a realistic test. Actually try to retrieve everything, but, you know, when times up, you’re going to get to go back to it. And then they would grade it themselves. So I didn’t actually do the grading either. And that is really great for spurring discussion. And “Oh, my gosh, I thought I knew the difference between reliability and validity, but now that I tried to answer it, I realized that I didn’t.” And then you can throw it back out to the student who now has bragging rights for having had their question selected and say, “Hey, what did you mean? What was the right answer? And why did you put that down there?” And at the end of the day, they kept that quiz too. So it was really very much in their own hands to do. So there’s that. Other creative ideas that I’ve run across over the last couple of semesters… There’s a great project out there, run by Bruce Kirchhoff at University of North Carolina at Greensboro. So I got the wonderful pleasure of talking to that group last year. And he and some colleagues working in the area of botany actually put together a freestanding, custom built mobile app that students could take with them that presented different kinds of quizzes over the sorts of things that botany students really need to know like the back of their hands like how to identify different plants and how to discriminate among different examples and they found some empirical evidence that this actually raised performance up quite a bit. I’ve heard too of another Professor put together some surveys in Qualtrics, the surveying software took advantage of its ability to actually text message people and send them the questions. So this was an opt in sort of activity and it’s one that they didn’t just have it run 24-7 because it was a little intrusive, but what it did is it sent students questions that they could answer at different intervals throughout the day, which also takes advantage of another principle from applied memory research which is spacing. So students are getting these unpredictable questions, they have the option to answer them, and they could be happening even when they’re not in class. So those are some other ways. Some of them look like tests, some of them don’t. But those are creative ways to get students engaged in that practice.

Rebecca: As a faculty member, we often advise students and mentor students who might be struggling in other classes, ones that we don’t have control over. Are there ways that we can help those students use this methodology to do well in those classes where it might not be embedded?

Michelle: Right. And that’s so great that you bring that in as well because ultimately that is what we need as teachers and that maybe circles back to yet one other piece of objection that I’ve seen… actually this time in a published article from a few years back that said, well are we doing too much for students? Are we scaffolding them too much so they’re going to grow really dependent on these kinds of aids like reading quizzes, and reading questions. But if we also have in our mind very intentionally, that what we want students to have at the end of the day is also something they can walk away with, I think that we do have to be very mindful of like, “Okay, let’s not create the impression that just because retrieval practice is so important that you have to sit there and wait for me to put together this specific kind of reading quiz for you.” So I think here, the really powerful message is once again that one students take this to heart… once they’ve not just been told this, that “Oh well, you should quiz yourself as a study strategy,” but they’ve seen that I believe in it to the point where I’m going to put time, energy, and work into it as part of my class. And maybe they’ve even seen the results… they’ve now had their own little before and after experience of what happens when I do this… that they can be more likely to take this forward. So having more faculty across the curriculum endorse this is a powerful idea that “Hey, this is how your mind works. This is how your brain works. Your brain doesn’t just soak up stuff that’s in front of it. You soak up stuff that you have to answer questions about.” That I think is going to be a powerful message. And there’s actually another article out there that I just absolutely love it was done by a psychologist named Pennebaker and some colleagues at University of Texas at Austin some time ago. They replaced high-stakes assessments in their Introduction to Psychology course with these mobile in-class quizzes that were done every day. And one of the things that they report in this article is not just that students did better in that class, but that certain subgroups of students actually showed improvement, specifically closing of achievement gaps, really, in classes that they took after the psychology course. And these were classes that we’re not even in psychology. So you think about that for a minute. How does that happen? Now, nobody knows exactly for sure, because this was kind of an unexpected finding in the study was my impression, but a real possibility is that when students have sat in this class and every single day they have seen the power of taking a quiz… of spacing out their learning… and attacking it in a very active learning approach… Once they’ve seen that happen, they’re more likely to go home and say, “You know what? I can do this in my biology course too. I don’t have to sit there with the teacher’s quiz. But, if I attack it in the same way I might get the same results.” So I think with those things, we can have students walking away with that enduring practice that we want them to have. And it is funny too, because it brings up one of the, I’ll just say it was a really heartwarming teacher moment that I had this last semester when I did bring in these Kahoot quizzes. In the run up to the first exam, I had done my fancy little in-class quiz and was kind of patting myself on the back of what a great leader I was. So I came in to, I think it was the class right before this exam, and I come plowing into the classroom and it’s dead quiet. And there is a student at the front of the classroom. He has commandeered the podium and the computer and what has he done? He’s accessed the Kahoots, which I gave them the links. You know, they’re out there for all the students to see. And he is running them and the rest of the students in the class are taking them just as seriously as when I was administering them. So they’re running through the questions again, giving it another shot. I didn’t tell them to do this at all. It was one of these moments where I just backed out and closed the door after me. And I came back in when they were finished. And so I think those are the kind of moments that we can set ourselves up for when we really do bring this into our classes and we get behind it and a really authentic way.

Rebecca: I think one thing Michelle, in the way that I asked the question, I had also asked it from the perspective of a student hadn’t had the experience of doing a retrieval practice in a class. So, if you’re working with a student, maybe outside of class or as an advisor, are there things that we could do to help those students adopt those practices, even if they haven’t seen it modeled for them?

Michelle: So how do you help students when they haven’t had some of these experiences on their own? And I think this is a part of a bigger package of goals that I think a lot of us should have in supporting our students to really put students in the driver’s seat. To say, “Yeah, you don’t have to wait for a certain style of teaching or certain subject material in order to succeed with it.” I see that isn’t really part of a larger package of growth mindset honestly. So what can students do to make themselves the masters of this? Now, I think that there are some old standard, very traditional, approaches that are worthwhile. And when you look at those approaches, sometimes retrieval practice is at the core. So it may not be a matter of trying to get a very unfamiliar set of terminology or anything for students before, but really getting them to look at some of these approaches in a new way. So things like you’ve ever heard the term SSQ4R or PQ4R, there’s a couple of those that have acronyms and the things they have in common are that they tell students when you sit down to study or read a text or prepare for a test, here’s what you do. You don’t just start reading from the top with no goal in mind other than “Oh, I want to get a good grade later.” What you do is first of all, you set yourself up with questions. Which is, after all, a lot like retrieval practice. Start with a question, say, “What do I want to answer? Can I answer that now?” And if not, why not? To read your text or go through your material very intentionally around this questions, and then there’s always this piece of recite or review. That’s what one of those Rs stand for in some of those traditional systems. And that means closing the book. That means closing the book and saying, “Okay, I’ve sat here with this material for this amount of time. What can I actually say about it at this point?” So, sometimes directing students back into some of those and saying, you need to adopt these strategies, which are completely teacher-independent, they’re fairly discipline-independent as well. That can be good. If you’re doing them for the right reasons and with the right approach in mind, that’s very good. Encouraging students to take advantage of the publishers’ companion sites… Now this is a little bit more of an uphill climb. That is where we run into “Well, if there’s no points for it, Why should I do it?” But encouraging students to say “Look, you already paid for this textbook. To be crass about it, you paid all this money, did you know there’s a website over here and if you interact with those materials in this particular way that’s really, really likely to pay off for you?” So, besides just ensuring that our students have what I consider to be these basic foundational pieces of knowledge about the mind and brain: that we remember through testing, we remember through retrieval and an active engagement. All students should have that, but those are some specific things that we can counsel students to use across all their studies that really should pay off.

John: Do you have any other suggestions for those faculty that are thinking about expanding their use of retrieval practice?

Michelle: You know, just to really encourage and support faculty who are starting this journey, as it sounds like that you all have, to really re-examine how we can bring in this incredibly powerful principle, and to really reassure each other that “Yeah, this is not about really just piling so much more work on yourself.” We can sometimes even just re-examine the assessments and assignments that are already in the course and so that’s kind of one last piece of practice that I think that a lot of us can really stand to bring in. And that was a big thing for me as well to say, “Well, if I’m going to administer these tests… we administer tests, we administer midterms anyway… what else can we do to increase their value as learning experiences… as learning events. So here’s another where I’ve brought in various forms of test discussion activities. Instead of standing in front of the class with that deadly the day-after-the-test class period where I say, “Let’s go over it” to realize, you know what? That probably will not work that well. But one of the amazing things that we have learned from the research on retrieval practice is that it’s not just in the moment in the taking of the test, where this advantage to memory happens. It actually creates a sort of a receptive window for learning and for review when we’ve just taken a test on something. So if you’ve got a midterm in your class already, well, hey, why not carve out the time after that test is up to say give it back to the students… like I photocopy the exams before I’ve graded them. They have no feedback on them whatsoever. And I hand them back to students as a group discussion exercise. I say “Alright, here’s a blank copy of the exam, your group can fill out this blank copy together, just knowing what you know, revisiting all those questions, having those good discussions about what you understood and what you didn’t. And I offer a little bit of extra credit for really good performance on that. And there’s other ways to work that out so it actually takes off as a small group exercise in class. But regardless of the specifics of how you make something like that work, the spirit of it is the same. That when you sat down to do this, to try to drag all this information out of memory, that is not an end unto itself. It should be part of a bigger picture of learning in the class. And it’s sort of an untapped vein of potential we have as teachers and that our students have as well and that we can access it regardless of what the discipline is or how the class is setup.

John: …and students see it’s very relevant. I started doing something very similar in my econometrics class last year following a suggestion from Doug McKee who had been doing that in his class. And it worked remarkably well… and turned what was normally a pretty unproductive class period where we’d go over the test… and the people who did well with just be really happy and pretty much ignore any discussion, and the people who did badly were just sitting there unhappy and not really being very receptive. But when they sit there, and they’re explaining it to each other, it seemed like a really ripe time for them to learn the material much more deeply.

Rebecca: I remember the first time that you implemented that. You sent me a text message with a photograph of his students taking a test and it looked very active. [LAUGHTER}

John: …and they were having fun.

Rebecca: Yeah.

John: One of the nice things that came out as I was wandering around listening to their conversations, and I was hearing people say, “Oh, yeah, now I see where that came from.” Or someone would say, “Well, when did we learn this?” And then someone else would say, “Well, remember when we worked on these problems?” …and it just helped them make connections… and the power of peer instruction is so remarkable. I’m going to do it in as many of my classes I can.

Michelle: That’s perfect. That’s what we want as teachers, and that’s what our students benefit from.

Rebecca: So as you know, we always wrap up our podcast by asking what’s next?

Michelle: Oh, wow. So I am looking forward to kind of rebooting a course that I have not taught in some time. My senior capstone course in technology, mind, and brain. And that is a fun one. But it’s one that is going to take a lot of revision, since it is technology and things change so rapidly, as we all know. So that is going to be a big part of next semester. I also have a crop of research projects in various angles on teaching, learning and educational technology that I’m really excited to be moving forward in the next calendar year. One of those… really foremost among them is a project on virtual reality for learning. We have an incredibly creative and dynamic team looking at virtual reality here at Northern Arizona University. They put together an amazing series of interactive exercises that are part of the organic chemistry course here and teach some of the challenging concepts in that extraordinarily challenging gateway course. And so we now have a whole set of data from students who went through and did this at varying points during the semester. We got their feedback, we’ve got all different kinds of psychometric measures that we gathered from them at the time as well. So, I cannot wait to be tackling that and looking at all kinds of angles on how this part of technology is impacting student learning in this course.

Rebecca: Sounds like some exciting adventures.

John: That sounds like a wonderful research project. …looking forward to seeing what you find.

Rebecca: It was a real pleasure to talk to you again, Michelle. Thanks for spending some time with us.

John: It’s wonderful talking to you. Thank you.

Michelle: Oh, likewise, always a pleasure to talk to your listeners.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.