73. The Injustice League

Difficult conversations like those around injustice and inequity can be challenging to facilitate no matter the student body, but first-year students have additional barriers to overcome like establishing a sense of belonging on campus. In this episode, Dr. Margaret Schmuhl joins us to discuss how comic books and programming outside of the classroom can help first-year students develop the confidence to engage with complex social issues. Maggie is an assistant professor in the Department of Criminal Justice at the State University of New York at Oswego.

Show Notes

  • Mya Brown – Assistant Professor in the Theatre department at SUNY Oswego
  • Amy Bidwell – Associate professor in the Department of Health Promotion and Wellness at SUNY Oswego
  • ACUE – Association of College and University Educators – certificate of effective college instruction

Transcript

Rebecca: Difficult conversations like those around injustice and inequity can be challenging to facilitate no matter the student body, but first-year students have additional barriers to overcome, like establishing a sense of belonging on campus. In this episode, we examine how comic books and programming outside of the classroom can help first-year students develop the confidence to engage with complex social issues.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Margaret Schmuhl an assistant professor in the Department of Criminal Justice at the State University of New York at Oswego. Welcome back Maggie.

Maggie: Hi everyone.

John: Good to have you back.

Maggie: It’s good to be here.

John: Our teas today are….

Maggie: I am having a black ginger and peach tea.

Rebecca: Oh, one of John’s favorites.

John: It is [LAUGHTER]

Rebecca: I’m having English afternoon tea.

John: And I have Tea Forte black currant tea.

Rebecca: Oh, another one of your favorites.

John: I have many favorites.

Maggie: That’s a favorite of mine too.

John: We invited you here to discuss your first-year Signature Course here at as Oswego called the Injustice League: Crime, Justice, and Inequality in Comic Books. Sounds like a really fun course. Can you tell us a little bit about the course and how it differs from your other introductory criminal justice courses?

Maggie: This class was a lot of fun to teach. In the class we read various comic books, we watched different superhero movies and we talked about, within those comic books, what it means to have justice, to recognize injustice, and how society responds to crime and maintains or perpetuates various inequalities in those stories and movies.

Rebecca: How does that class differ from the other classes that you teach in your subject area? Because it’s a First-Year Signature Course, so that has particular meaning at our institution.

Maggie: Yeah. So the signature courses here at SUNY Oswego are about bringing a student engagement aspect to our academic course content. And so in this class there’s a balance of introducing our subject matter (in my case, Criminal Justice Studies) to the students but through a really fun way, but also working with students to help them with their academic success and getting engaged with each other and with our campus community.

Rebecca: Are these usually majors that are in this class or non majors?

Maggie: So I actually had a mix of majors and non majors. I had probably about 19 students. I had about 10 or so majors and so about half of the class were non majors. And they came from a variety of disciplines, including computer science, communication studies, and we even had some undeclared majors in the course.

John: And this is part of a broader initiative that we talked about in a prior podcast, and we will include a link to that in the show notes for anyone who wants to learn more about the first-year signature courses here. So one of the purposes of this, as you said, is to build more of engagement with the college community and also amongst themselves so that students will feel more connected. And one of the first things you did, I believe, was take them on a field trip. Could you tell us a little bit about that field trip?

Maggie: In Oswego, there’s a local comic shop that we ordered some of the students comic books from. In the very first class I asked them which comic books they were interested in purchasing and how many students we’re going to accompany me to the comic book shop here in town. And so to get to the comic book shop, you have to take the public bus, or at least if you don’t have a car, and many first year students don’t, they have to learn how to take the public bus. And so part of this field trip was not only obtaining some of the course materials for the class, but also getting the students familiar with public transit in the town and how to navigate a new place with them. So we arrived at the comic book shop and the owner was very gracious to us. She made us cookies and we had coffee and the students picked up their books and some of them even got some additional materials. We had a lot of fun. It was an amusing trip bringing a bunch of college students on a bus, and some of them their first time using public transit, and the bus drivers were even entertained by the group of us, so we had a really good time.

Rebecca: I think you also discovered the infrequency of the buses….

Maggie: Oh yes.

Rebecca: …in our town, right?

Maggie: Yeah…

Maggie: Oh yes.

John: Particularly on weekends.

Maggie: Yeah, particularly on weekends. We did wait about an hour for the bus on Sunday. So that was a little bit of a lag, but we made it through.

John: I should note that the comic book shop is actually owned by the wife of a former member of my department. It’s Arlene Spizman who runs that store.

Maggie: Yeah, Arlene was wonderful.

John: She’s a very nice person.

Maggie: I didn’t realize she had that connection.

John: In fact, I just finished a paper with her husband.

Rebecca: I’m sure it can be difficult to have an authentic conversation about justice in general, especially with a diverse population of students and maybe students that don’t know each other very well. How did talking about comic books as a way to get into the topic help facilitate those discussions?

Maggie: Comic books offer a different world for students to experience some of the concepts and some of the issues that we struggle with as a society. And so to be able to visually see these issues play out across the panels, it’s a place where students don’t feel nervous or threatened, it feels safe. They’re taking these comic books and they’re finding ways to relate with them and work out some of their preconceived notions or feel like it’s okay to start working on some of these biases and issues in society.

Rebecca: It seems like it has a lot in common with some of the other topics that we’ve talked about on the podcast before, like simulations and role playing, where it’s a place to escape the real world and talk about something really challenging in a so-called fake environment, but really they’re working out real-life issues and biases and all kinds of things that can be really difficult to talk about, but it’s a lot easier to talk about character that’s not real.

Maggie: Absolutely. Yeah. One of the comic books we read is an X-Men comic book called God Loves, Man Kills and we talked a lot about the concept of othering and what it means to target out and marginalize a group of people and in a lot of ways X-Men plays out what has happened in race relations in society and in other groups who have been historically marginalized. And so for students to consume this information through a comic book, they can better reflect on their own experiences and start to understand the position of others in society.

Rebecca: It probably also makes it a lot easier to make mistakes when you’re talking about that. I think sometimes students don’t want to talk about touchy topics because they’re afraid of offending someone or saying something in the wrong way but if it’s not about anybody real….

Maggie:Yeah

Rebecca: …then it’s not going to hurt someone’s feelings.

Maggie: Yeah, absolutely…. And another thing that we did in this course right from the get go was to set ground rules for discussion and conversation. And so I had the students come up with various guidelines for discussions and we would write them down so that we could refer back to them as we continued throughout the semester, so that they all understood that they had a responsibility to each other to make sure that everyone was comfortable and safe in this classroom. It really helped to facilitate a lot of these very difficult discussions in a very similar way that comic books themselves kind of help us talk about very critical and very upsetting social issues.

John: They also come in probably very familiar with many of these comics because they’ve seen them in movies, and some of them may have read some of these as well. Could you give us some specific examples of some issues in criminal justice that you were able to address using comic books?

Maggie: In terms of the classroom breakup, we have many students who were avid comic book readers. And we had many students who were somewhat interested in comic books but were more in tuned with the recent TV shows and movies that have come out of Marvel and DC Cinematic Universes. And so there’s, you know, quite a range of knowledge around this. But for the most part, most all of the students were interested in some kind of criminal justice aspect of their studies. And so, one of the concepts that we tried to discuss in this class was the issue of mass incarceration in society. And so what we did was we read a book called Bitch Planet, and in this planet, women are incarcerated on another planet. It has a lot of strong connections to some of the ways that society has restricted women’s rights throughout history and how the punishment of women has differed across time and across place. So to be able to see these concepts in a comic book and to talk about the parallels that exist in our society was a really, really cool process for the students.

John: Did you mostly focus on comic books they were familiar with, or you mentioned Bitch Planet, which was one that perhaps many of them hadn’t seen before. Did you bring in many that were things that they hadn’t expected or that they were less familiar with?

Maggie: There was really a mix. I even had some criminology textbooks that had various criminological theories played out in comic book form and we read a few of those to give us a baseline of various theoretical perspectives on criminal behavior. But most of the comic books I’d say we’re falling in the mainstream. I think that’s what students were typically looking forward to, but they really did enjoy the new reboot of Miss Marvel, with Kamala Khan and Bitch Planet and those were perhaps a little more on the periphery than Black Panther and X-Men.

John: How did students react to this? Did they generally find it interesting? Were some students troubled by using comic books? What about the imbalance between those students who were very avid comic book fans with those who were less familiar? How did that play out?

Maggie: Some of the very avid comic book fans in the class had a lot more context to really draw from when discussing histories of the Joker or Black Panther and the development of the character over time. But because comic books have become so popular in mainstream media, with TV shows on Netflix and pretty much a new Marvel movie coming out each year, that students really had a lot to draw from. Students didn’t need a great depth of knowledge of comic books prior to coming to this class.

John: For those students who were avid comic book fans, was it a little more challenging, perhaps, than they expected to look at some of these things through perhaps a more critical lens?

Maggie: I think that comic books, even if you don’t have a great background of reading comic books, or knowing the development of various characters, I think comic books allow for anyone to just pick them up and start thinking about them in a different way. They’re relatively quick reads, which really helps. Students can read them a couple of times and start to reflect back on some of the course concepts and theories that we discussed and how they apply and pull out those very specific examples. So I think the medium of comic books really provides a great range of abilities for students.

Rebecca: What are some of the things that you learned from teaching this class that you started employing in other classes?

Maggie: Oh, that’s a good question. One of the things that teaching the Injustice League has helped me with my other courses is to really think about being explicit with what I expect and what I hope students learn from various assignments and activities. In the Injustice League, students are entering college for the very first time and so they may not always understand why we’re reading this particular article or how it relates to the comic book that we’re reading today. And so for me to slow down as an educator and say that “Here’s why we’re doing this. Here’s what the research is showing us about why low-stakes testing in this class is a good thing.” That’s helped me in my other courses be more explicit with why I’m making decisions in various teaching practices.

Rebecca: Have you done anything else that’s related to bringing more comic books to other classes or field trips or some of the other things that brought the fun piece to the class that I think really energized the group as a whole?

Maggie: In my research methods class, I’m hoping that students will be able to assist in it by going out into the community and surveying people about dating formerly incarcerated persons. And so I think to get them out into the community and to start locating various areas of the community will bring some of that campus engagement aspect to it. In my crime-mapping class we actually started geocoding some of the locations around campus and so these are more upper-division courses but I’m trying to, even though the winter months make it a little more difficult to get outside, but trying to get outside of the classroom and really talk about how important it is to be connected to our community and to understand our relationship with the community.

John: I believe there was also some type of a video or a movie that you showed and I think other classes participated in that. Could you tell us a little bit about that and how it fits into other classes?

Maggie: There was a collaborative effort among some of the first-year signature course instructors to bring our students together in a common place. We ended up watching a episode of Luke Cage and this particular episode really spoke to a lot of the different courses that were being offered in the Signature Course program. Obviously my course, as one that deals with comic books and crime and justice and inequalities, Luke Cage is a very good example of many of the concepts that we talked about in class. But Mya Brown in the theater department also taught a class called Blackish Mirror and it followed the development of black characters on television. And so this was also a really good place for her class to talk about how various stereotypes that they had learned existed and/or were resisted against in Luke Cage. We also had a professor from political science and from communication studies, talk about political organizing or activism in Luke Cage, as well as narrative and the use of narrative in TV shows. We even had a signature course instructor in the health and wellness department… their class made snacks for the students to enjoy at the event.

John: Healthy snacks.

Maggie: Yeah, healthy snacks and it was brilliant. The students loved it. They created a snack mix that could be created and replicated by using ingredients found on campus. So that was a really cool way to bring in even a discipline that’s not necessarily focused on examining social inequalities in media to this event, and so it really spoke to a lot of students across various disciplines.

John: And we should note that, that person was Amy Bidwell, who was on an earlier episode. Were there any surprises in teaching the class that you didn’t expect?

Maggie: The class was a lot of fun. I don’t think I’ve had as much fun teaching any other classes I had teaching this class. It was really fun to pick up a hobby of mine, something like reading comic books, to bring this to the classroom and to start and challenge students to think about the media they consume in a new way, and how it reflects what we do in society and various values that society has. One of the most surprising things in the classroom was really how much of a community the students had at the end of the class. They had been speaking about other courses and working together on other projects and planning their course schedules for the next semester so that others would be in their courses together and so that was a really cool outcome of the class.

John: …and I believe you also opened an Instagram account for the class.

Maggie: I did and so you can follow it @the_injustice_league_oz… each word is underscored. I won’t say that I have many followers on the Instagram account but a lot of the students who did follow it seemed to really enjoy it.

John: And are you going to be teaching this again?

Maggie: I will be teaching this class next fall. So I’m very much looking forward to the next cohort of Injustice League members.

Rebecca: Did you carry on the superhero fantasy world theme throughout the class? You talked about rules for discussion or rules for engagement at the beginning. It’s almost like world building. Did you think about theming that more? Could you talk about how you might have done some of that?

Maggie: All of the designing my syllabus was all thinking of the class as being a part of a group of superheroes as opposed to just a group of students in the class. I even designed the midterm exam to look like a top-secret mission directive from their Professor S, which is me. The secret mission was about identifying various concepts that we talked about in class and applying them to a new comic book that we hadn’t read in the class. And so, in this midterm exam, they got to explore some of their favorites that we may not have gotten to touch on the class. It was a good opportunity for them to get creative and think about how these theories and concerns about justice translate across various stories.

John: And that way, you’re giving them some autonomy, but you’re also helping them develop transfer skills so they can take the things you learned and apply them in new circumstances, which is a really good practice.

Maggie: Even one of my students, when we were discussing moral panics, stopped into my office hours one day and was ecstatic because he had just realized that his journalism course was talking about moral panics, and so to be able to identify these concepts across disciplines was also a really cool outcome of the class.

Rebecca: You talked a bit about the class being really fun to teach. And part of that’s because you brought your hobby and your discipline together. But were there other things that made the class fun? I can imagine that you’ve all thought about yourselves as a part of a league. So maybe that you felt more connected to your students, or am I kind of projecting?

Maggie: Oh, absolutely. So, I called myself Professor S as a play on Professor X in X-men and so the students really loved that and they had a really good time with the way we even addressed each other in the class. The Instagram account even helped create more of a community by bringing in various pictures of each other doing or identifying various comic things across our everyday lives and interactions.

John: How did you first get interested in comic books?

Maggie: Actually, my first interest in comic books came from graphic novels and reading Persepolis as a kid. But, of course, I fell in love with the Marvel Cinematic Universe and I’ve watched pretty much all of the movies in chronological order.

Rebecca: Of course [LAUGHTER]

Maggie: I called it “research,” the summer I rewatched them all, it was a wonderful time. After reading a novel like Persepolis it was also something that really got me interested in criminal justice and society and inequalities in social life.

John: Did the students seem more open to discussing some of these issues having been exposed to them through comic books?

Maggie: Comic books reflect a lot of what is going on currently in society and they provide us a way to talk about really difficult topics of racism and sexism and things that occur and that people and students are experiencing in their everyday lives. So using comic books to facilitate these conversations is really important for students just beginning to question some of these processes.

John: One aspect of this course, as you said before, was to help introduce students to college life and help them create bonds and connections. But that also frees you up quite a bit because you don’t have a standard curriculum. Is this the first time you’ve ever taught a class where you didn’t have a fixed amount of material you had to cover in the course?

Maggie: Yeah, so this class was really flexible in that way. As I look back on the class, I’d say that it’s equally as important for us to be talking about some of the content about comic books and the sociological and criminological aspects of them as it was to help students become more connected to their community and to their campus, but also to ensure that they will be successful students at moving forward. And so this class really allowed me to work on some of their questions that would just come up, like calculating a GPA or registering for classes. And so the flexibility that exists in this class lets me respond to the students and their concerns in the moment and to occasionally tie-in some of those issues in current events to what we’re discussing in these comic books.

Rebecca: I could also imagine that it allows for the tangents that might occur as you start talking about something related to the comic book but you think it’s a valuable discussion. But if you have a finite amount of material in a finite amount of time, you might not be able to b go down those rabbit holes, but they can be such valuable conversations.

Maggie: Yeah, absolutely. You know, the flexibility of this course isn’t just to my benefit, but to the students benefit, where they can ask questions and we don’t have to worry about how much material we get through. We don’t have that curriculum that requires various elements to be covered and so the students can explore some of their questions in a very meaningful way.

Rebecca: I think it might be useful to just clarify that these classes are not part of any specific major, and they’re not a prerequisite for anything. So that’s what we’ve been talking about in terms of them being kind of freeing. I don’t know if we explicitly stated that.

John: Did you get to know the students better than you would in a typical introductory class?

Maggie: O ne of the good things about this particular classes is that there were only 19 students, and so it really allowed for me to get to know each of them individually and be able to see their personalities through our discussions and to have that comfort level with the classroom to talk about what made their day not so great today and what they really enjoyed about the weekend. And so to have that sort of informal relationship in a very formal setting was a really cool experience.

Rebecca: I know that one of the things that I’ve been thinking about after hearing many of the faculty who taught the signature classes talk about their classes is just finding ways to have some more of those informal opportunities in class, but also thinking very carefully about the content that I think that needs to be covered versus what maybe actually needs to be covered. There tends to be a disconnect, We think we need to cram in so much stuff. What are some of the key principles and things? And can we go into more depth for some of those if students are interested? And I’ve allowed that to happen a little bit this semester, and it’s been really delightful, I think, for everybody involved.

Maggie: Yeah, that was one of the things that I struggled with in the class. At the very beginning I was treating the class like a topics course and cramming, or at least planning to cram, a ton of information in. A few weeks in, I realized that it just wasn’t going to work for this type of class, that this class really did need time to facilitate these relationships and to help students learn and navigate their first semester here on campus. And so to have that flexibility for them to be able to explore their questions and concerns on campus and off campus was a important part of this class.

Rebecca: Sometimes I think that these functional aspects of being a student can get in the way of learning. So spending the time and just addressing those concerns that are preoccupying a student can free them up to actually think about the content and spend time investigating it. So, if they’re really concerned about figuring out their GPA or really concerned about making sure they’re registering for the right classes, addressing that concern up front can actually free them to focus on learning.

Maggie: Yeah, absolutely. And a lot of students, they come into the classroom and they think that college is going to be just like their high school experiences and so that studying in the same ways is going to be effective for them or that GPA’s and what credit hours are. There’s a lot of new information that makes transitioning to college more difficult one then, say, transitioning from their middle school to their high school . And so I think this does give them the time in class to talk with a faculty member to try to work out some of these questions in a way that they may not get to in their other courses and so it does certainly alleviate some of their anxiety around these issues.

Rebecca: When there’s not a context like that I think the option is going to office hours or something and that can be really intimidating, I think, for first-year students, or they just have no idea what office hours are for, which is another thing.

Maggie: Right, or how to book an appointment…

Rebecca: Yeah.

Maggie: …and all of that. And so I know many of the First-Year Signature Course instructors, including myself, tried to have individual meetings with students to break the barrier of setting up an appointment for office hours and how to draft emails to your professors and such and so I think it really helps them not be as nervous about getting the help they need and the resources that they may need in the future moving forward.

John: Because in the past if they were called into go to someone’s office after class…

Rebecca: Right, it was a bad thing, yeah.

John: Exactly. And so, you know, that’s something they do need to get past and it takes a while often and by then sometimes a little too late. So that’s really helpful.

Rebecca: Speaking of criminal justice, right? [LAUGHTER]

Maggie: Colleges, it’s a different culture than they’re used to and so to get assimilated to that culture is really important in many different ways.

Rebecca: Right, it’s like mentoring instead of a penal system.

Maggie: Right? [LAUGHTER]

Rebecca: Kind of a weird word flip there.

Maggie: Yeah, absolutely.

John: And one of the nice things about this whole program is it was set up as a randomized controlled experiment where students were asked if they were interested in courses and then they were randomly assigned or not assigned. And there’s some work that’s being done right now analyzing how their outcomes compared to outcomes of the students who were not in one of these groups, and they’ll be followed a bit to see how this works overall. So, I’m looking forward to seeing more, but the preliminary results they have, as were reported in the meeting this morning, were fairly positive.

Maggie: Yeah, retention was really good and so hopefully that’ll continue.

John: Semester-to-semester retention….

Maggie: for underrepresented populations, yeah. There was…

John: …was 100% retention semester to semester.

Maggie: Yeah.

John: It’ll be interesting to see if that persists, because that has not always been the experience of Freshmen.

Maggie: Right, and hopefully it does and I think one of the things this Signature Course program is trying to promote are those students and faculty relationships and that if students have a strong bond with a staff member or faculty that they’ll be more successful in all aspects of their academic life.

Rebecca: We always wrap up by asking, what next?

Maggie: Well next, I’m currently meeting with various new faculty members for the Signature Course program so we’re going to work our way through more course prep, and I’m very excited to meet the newest members of the Injustice League next Fall.

John: And you’re also joining the cohort of people in ACUE…

Maggie: Yes.

John: …which is starting up here on campus very shortly.

Rebecca: …another league.

Maggie: Yeah, another league of sorts. [LAUGHTER] I’m very excited… very excited for that as well.

John: T hank you. It’s been a lot of fun talking to you about this course and I wish I could take most of these courses.

Rebecca: I know, they’re always so much fun to hear about, but I think they give us lots of prompts and interesting things that we can start to consider in other contexts too.

Maggie: Absolutely. Thank you for having me.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

71. Small Teaching Online

Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, Flower Darby joins us to explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

Show Notes

Transcript

John: Face-to-face classes have been offered for centuries. Online instruction, though, is relatively recent and many faculty that teach online have little prior experience or training in online instruction. In today’s episode, we explore some easy-to-implement teaching techniques that can be used to help improve the learning experiences of our online students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Flower Darby an Instructional Designer an Adjunct Instructor in several disciplines and the author (with James Lang) of Small Teaching Online, which is scheduled for release in June 2019. Welcome Flower.

Flower: Hi John. Hi Rebecca. Thank you for having me. I appreciate that. It’s good to be here.

Rebecca: We’re really glad that you’re joining us as well. Today our teas are:

Flower: I am drinking Builders tea. Good, strong cuppa here.

Rebecca: Sounds yummy.

John: We have some of that next door. I am drinking ginger peach gree n tea.

Rebecca: I have my Golden Monkey again today.

John: We ran a faculty reading group here in the Fall semester of 2017 based on Small Teaching. Many faculty found that to be highly inspirational and we had over 100 people participate in that. One of the things that came up quite a bit is how this might be applied online. So there’s a lot of people interested in your forthcoming book. Could you tell us a little bit about how this book came about?

Flower: Sure. So Jim Lang came to my campus—Northern Arizona University—in January 2018 and delivered a talk on Small Teaching and as we know, the book has been very impactful for faculty around the country and around the world. And while he was at our campus, when it came time for the question and answers, somebody raised their hand and said, “Sure, but how do you do this online?” And Jim’s immediate response was, “That’s the first question I always get at every talk that I give,” and he said, “I don’t know. I would need a co-author because I don’t know how to do this online, but that would be a great book.” So I thought about that for a few days and then I approached him and I said, “Pick me. I would love write that book with you. I can see the value of it, I can see the need for it.” So that’s how the conversation began.

Rebecca: How does this extend the approach that was used in Small Teaching?

Flower: Well, it follows the same principles for certain that there is learning science that we can draw on to help us make the everyday decisions in our teaching and learning that have really an outsized impact on student learning and outcomes. So there are little things that we do on a day-to-day basis and we can draw from the research to discover what will have the most impact. Again, understanding that in order for faculty to really be able to implement something new, it’s got to be feasible. It must be doable. The daunting overhaul of a major course redesign is so off-putting that most faculty won’t get around to it, myself included. When I have gone to multiple workshops and conferences and sessions or read about an approach. And I, “That is a great idea,” and I spend about five minutes thinking about how I might incorporate that into my class and then I say, “Too much work. Too much time. I don’t have that time available. I don’t want to implement something that’s only half baked,” and the idea gets left out. So in our online classes, there are so many things that we can do that are on that small scale but will have that outsized impact on our students’ engagement and their learning. And so that’s what this book sets out to do, is to explain a lot of those principles and draw on the research that we have to show faculty how they can make these changes in their online classes.

Rebecca: We’ve talked many times on our podcast about the lack of preparation for faculty teaching in general and that’s certainly true for online teaching. You might have taught a face-to-face class, and then all of a sudden, now you’re teaching an online class and boom, you have to figure it out. Can you help us think through what are some things that faculty can do as they’re new or getting used to being an online teacher?

Flower: Sure, and I think that’s really the point here. Centuries, millennia, compared to the way that we teach and we coach and we mentor face to face, or even as we’re doing here using video conferencing software, but it’s a real-time interaction. Well, online teaching is very, very recent, say 20 years or so. And faculty don’t have the experience that they bring into the physical classroom. You may have heard of the phrase of the apprenticeship of observation coined by Dan Lortie. And this is the idea that by the time a teacher steps into a classroom to teach, he or she has had years and years of experience in a physical classroom being a student and observing what happens and how things go and thinking, if somebody chooses to be a teacher, then they’ve clearly put a lot of thought into how they want to teach. Well, we simply don’t have that for online. I do expect this to change in coming years. But the fact is right now that most of our faculty have either never taken an online class or if they have, it may be a very limited experience, not the years and years that they came out of K-12 with. And the same, quite frankly, is true for our students. They’re also pretty inexperienced at an online classroom. And the way this pans out is that literally faculty and students both don’t know what’s supposed to happen in an online class. They don’t have the social norms, they don’t know what the classroom looks like. If you think about it, when you walk into a physical classroom to start teaching, you know what’s in the room and you know what’s supposed to happen. You see the desks or the tables, you see a lectern at the front, you see a whiteboard or a projection screen, and students and faculty understand what is supposed to happen. Students go and sit in the desks, they face front, they wait for the faculty to come to the podium. It’s rare that a student would walk into a classroom and at the beginning of the hour, just step into the lectern. Students know that’s not what they do. But my argument is we don’t have that kind of social norming convention for online classes…yet. I think it’s coming, but right now many of the people who find themselves in our online learning environments go into that space, and they don’t know what things should look like, they don’t know where the light switch is, they don’t know where the desks are, where the whiteboard is. So just that whole lack of experience is rather disconcerting. And it’s hard to know what to do. Faculty don’t have experience—they have haven’t seen models, students are equally unprepared—so there’s a lot of work to be done here just to understand what should happen in an online class, what the furniture is, where it should be to facilitate learning. That’s where those gaps happen for faculty. You ask, “How can faculty prepare themselves?” I could talk for days about that question. It’s a growing need and some institutions are beginning to recognize the importance of doing a much more thorough job of preparing faculty to teach online. But I will argue that those institutions are still pretty few and far between. I would say, based on my research and my experience, the vast majority of faculty who are teaching online have not had specific development in that area. They have not observed peers’ classes. In fact, what can happen is a negative effect. Very commonly, when faculty begin teaching online, they are handed somebody else’s content. We’ve seen that happen and that’s a mercy in a way because that way faculty who are new to teaching online don’t also have to develop the course. But what can easily happen is that the content that might be given to a new faculty member might not actually be exemplary in the design and the delivery of that material. So then what happens is the only experience that faculty get is observing the content and the structure of a less than ideal example and then that’s the model that they have and they think, “Oh, I guess this is how it is”. So work can be done on developing better exemplars, better development programs. I believe, as faculty are coming out of online graduate programs down the line a little way, I believe, will have better experienced faculty and students. A lot of research going on in this area, but that work is all to be done.

Rebecca: As you were talking, I thought, you provide a nice model, it’s a nice way of thinking about it, you don’t know where the furniture is.

Flower: Mmm-hmm

Rebecca: It sticks with me. I was thinking about that the experience that a lot of faculty and students have is more in the realm of social media and so they’re looking for cues that are similar to those kinds of environments. The activity that’s happening in those environments is really different than the kinds of activities we would expect to happen in an online platform for learning.

Flower: Right, that’s a great point. We interact with other people so much online and on our devices using social media and what’s interesting to me is that we can really engage with people in those online spaces. Somebody tweets something that’s a little bit incendiary or provocative and you get all kinds of people jumping in and commenting and you know, sometimes things get heated, or a really heartwarming moment is tweeted or shared on Instagram, and people are all over that post. But the opposite is kind of true in our online classes. Indeed, I feel like we could bring in some of the techniques from social media into online classes. I’m not saying that faculty should all have a component of Twitter or Instagram in their online classes, but what I’m saying is that it’s possible to deeply engage people in online interactions. And that’s not a feature that, I would say, generally characterizes online classes—we usually hear the opposite, that it’s not engaging, it’s difficult to drum up those discussion posts—and I feel like if we could draw some of those principles from how we interact with people online, in social media, of using our devices, if we could bring those into the online classes right away, we’ll see more engagement and engagement precedes learning. Students have to want to be there in order to learn when we’re engaging them and if you could imagine posting a discussion post and then you can’t wait to see what people are responding. We do that all the time on Facebook or Twitter sending something out and then, “Oh let me see! Did people like that? Did people say anything?” And we just naturally are drawn into those spaces to check and see what are people’s reactions? Well, if we could design that kind of a discussion board for online classes, where it’s so interesting and engaging that people want to rush back and see who’s talking to them, who’s replying to them, that would go away way to improving the online learning experience for both faculty and students.

John: That’s not an experience though that many people teaching online find in their discussion forums. Are there any hints or tips that you can give people to make their discussion forums a bit more engaging so that students don’t wait until the last minute to do the standard three posts or whatever is required in that course?

Flower: Great question, John, and a big one. And again, thinking about Small Teaching ways of making small changes, I heard of an example recently where faculty asked students to reply to their peers posts using a GIF that just represented—one of those funny moving little images that sort of expressed—their reaction. And that’s an example of bringing in new ways of engaging and it’s not rocket science. It’s also perhaps a little more fun, which is important to bring into an online class. A great way of sort of getting students to think differently. But if that idea doesn’t resonate with you, maybe you might want to try offering options in your discussion board questions. I’ve supported over 100 faculty, I might even say, hundreds of faculty in the design and development of their online courses and what I see sometimes is one question for students to answer and oftentimes it’s kind of black and white. It’s hard to discuss a question like that. So first of all, craft questions that are discussable, that there’s some debate around that you can make different arguments or points of view. Tie those questions to students’ experiences. How is the content impacting them personally? Where do they see these concepts in their own life and experience? And, even better, provide three or four different questions that students could choose to respond to and then ideally, everyone isn’t all talking about the same question, so that’s more of a natural way of fostering some conversation in an online discussion.

John: One of the nice things about tying it to personal things, I would think that that would also help build more of a sense of community within the group because the students get to know each other a little bit better, which may affect their engagement in other activities,

Flower: Right. Anything that we can do to increase the value and the relevance of what we’re asking students to do online is hugely impactful, and it doesn’t have to take much. I have a colleague who teaches an online First-Year Seminar course, which in a way is a bit of an oxymoron because First-Year Seminar courses are often designed to really hook in our first-year students who are transitioning to university life, but she was tasked with developing and teaching a really highly engaging and supportive Freshman First-Year Seminar class. And one of the things that she does is she brings in a discussion board and one of the prompts is, “If you could be a superhero, what would your superpowers be?” And again, maybe on the surface some people might think that’s a bit trivial, but what she’s doing is she’s getting students to talk about character traits and hero qualities and concepts that rely and relate to the material that they’re engaging with… yet in a fun and a more personal way. And it certainly does a lot to foster those relationships that are so important for online classes to build that community. Absolutely.

Rebecca: I think one of the methods that we hear a lot about in terms of online learning is the ability to do quizzing and retrieval practice and interleaving through quizzing. But are there some other ways that we can integrate some of these evidence-based practices that aren’t maybe the typical solutions that we tend to think of online?

Flower: I think one of the most underutilized functions of the Learning Management System is what we call adaptive release or conditional release. And I actually want to pause here and say that these Learning Management Systems have come a long way in recent years, and they still have a long way to go. [LAUGHTER]

Rebecca and John: Yes. [LAUGHTER]

Flower: For many faculty and students, the functionality is lacking, the furniture is stark, they’re not attractive places to be and as I said earlier, engagement precedes learning. If you were to ask yourself for online faculty who are listening, “Do you want to be in your online class?” I suspect many faculty would struggle to answer with a resounding “Yes.” And so here’s a shoutout to our LMS developers to think about space design and the experience of students and faculty in these spaces. Having said that, there is some very interesting functionality that is oftentimes underutilized and I would argue that’s because, again, faculty may not have the preparation and the exemplars to begin teaching online. With adaptive or conditional release—it’s called different things in different systems—you can set a task that then opens up the rest of the content in that module. And I love to use this. You can use it equally effectively at the end of an online module or at the very beginning to open the next module. Now what you can do with this is you can embed retrieval practice exercise. Or, you know, drawing from Jim Lang’s book, Small Teaching, a predicting exercise works equally well. A curiosity provoking exercise, and all it really has to be is an assignment where students submit whatever it might be. A two sentence summary of what their big takeaways were from the previous module, or predicting what might be in the coming module or posing some questions… this can be written, it could be a recorded submission for students who might find it easier to talk through their ideas. Once students submit that element, then the rest of the module opens. Before they do that they can’t access any of the content. These things don’t even have to be graded, you can just set them to be worth zero points, but they are mandatory because the students can’t proceed with the content until they submit them. So when you think about that feature, there’s a lot of creative things that you can do that don’t impact faculty grading time. That’s a big tenet of the new book is we can’t overburden faculty with grading and yet tie into those practices that we know from the research are effective.

John: One of the chapters of your forthcoming book is on fostering student persistence and success. Could you give us perhaps a strategy or two that might be useful in encouraging student persistence? Because I know one of the problems in online classes is they often have higher drop, fail, and withdrawal rates than face-to-face classes. What are some techniques that faculty can do to help improve student persistence in the class and the program?

Flower: That’s right, great question. As you point out, the attrition rates in online classes are remarkably higher. And we also find that for students who are less prepared for higher education, if they don’t succeed in that class, then the odds increase dramatically that they won’t actually persist and attain a college degree, and that’s a problem. But as I was saying earlier, a global concern is that online classes are not nice places to be. And if your listeners have any pushback on that, please feel free to reach out and engage with me on that assertion. But what can we do to just make the place a little more pleasant? How can we be warm and friendly and supportive and encouraging? How can we allow our humanity—even our personality—to show through? I was speaking with a good friend and a colleague of mine just a few months ago when I was delivering a little talk about this book and he was telling me though he’s been teaching online for 10 years, that he’d never thought of just being himself in his online class. And he explained to me that he loves teaching in person—he’s quite a character, super dynamic, very engaging, funny, loves to interact with his students in the classroom—and yet, he told me when he goes into his online class, it’s like a robot. There is no trace of his personality. And other people are saying this too, just be yourself in those online classes and make a deliberate effort to infuse warmth. But a specific strategy that people might want to try is to assign a goals contract as one of the items that are due in the opening module—or the orientation module—and a goals contract, you’ll see different kinds of variations around, but here’s the two pieces that I really like. A lot of people are talking about assigning sort of a memo of understanding or a contract where students agree that in this online class, they should schedule set times, they should plan on X number of hours per week, they should reach out immediately if they have questions. People are doing that. I like to embed a different element as well, which is to require students to set a couple of goals and it can be literally two. What are two goals that you have for your learning, or your success, your ability to earn an A in this class? And then an interesting twist is to ask students to identify one potential challenge. It’s still the case, I have my students all the time saying, “Well, my computer is in the shop, [LAUGHTER] it’s sort of all of a sudden, it busted and now it’s at the technician and I can’t do my online tasks.” So helping students to think in advance about a scenario such as that and of course, in that particular case, many campuses have computer labs or libraries where students can go and access another way to get into the course but maybe they haven’t thought about it in advance. So in the goals contract, ask students to set two goals, identify one potential challenge that might come up, and identify a strategy for how they can address that particular challenge. And certainly, identifying one challenge is not going to cover the range of things that happen in life during the course of an online class, but I think it sets the tone to get students thinking that one little hiccup doesn’t mean that we’re all done with this online class and we just have to sort of fade away and stop participating. And then what you can also do, periodically throughout the class, is you can ask students to revisit those goals that they set for themselves. How are they doing with that? What kind of progress are they making? Are there some strategies that aren’t working for them? Do they need to recommit to the intentional and deliberate scheduling of their class time? Just helping students be very explicit about what their plan is to succeed and finish the course.

Rebecca: What I like about what you’re saying is, it switches from really having the faculty member impose everything, and have the students be co-authors of the class to some extent, and they have some ownership over the space, which generally means that they’ll probably commit more.

Flower: What we know about online learning is that students must have a higher degree of self regulation, self direction, they must be more motivated, and be able to manage their time well. And if students don’t have those things, it’s much less likely that they’ll persist and finish an online class. And yet, when you think about it, online classes work directly against a student’s ability to do those things and here’s what I mean. When you are teaching in person, when you’re a student in an in-person class, you know that every Tuesday and Thursday at 9:30 you’re supposed to be in the classroom and it’s a natural way to help students hold themselves accountable for doing the work. Now I know sometimes students come to classes and they’re not fully prepared, but there’s still that built in mechanism where they’re going to be in the same room with their faculty member with other students. There’s a social element of accountability that’s like, “Well I know I’m supposed to show up and I should have my stuff done,” or “There’s a test next week and I need to be ready.” Well those real time interactions and those interactions with physical people don’t tend to happen in the typical asynchronous online course. Very often—I would say 99% of the time, probably—an online student is sitting at home by himself or at the coffee shop by herself. If she has a quick question about something, she can’t do what she does in the classroom and say, “Hey, did you understand what we’re supposed to do on that particular assignment?” or “Hey, faculty member, can you just re-explain that? I’m not quite there yet.” There’s no way to get that immediate response, that immediate quick guidance that might take two minutes in a physical classroom. So, students don’t have the accountability, they don’t have the physical presence of the instructor or the student, and so we have to go above and beyond in our efforts to build in structures that help students develop the kinds of self-regulated skills, the kinds of self-directed learning skills. Many of our students are not coming in with those skills already but we know students who do have those skills will be much more successful, so let’s build that into our curriculum. Let’s help them develop some of those, let’s talk to them about the importance of monitoring their own learning, and let’s structure exercises that will help them to do this. I’m pulling a lot of this material from Linda Nilson’s book—It’s called Creating Self-Regulated Learners—Although that book is not necessarily focused for an online environment, I think it can be hugely helpful to our online students to be very transparent with them about the importance of developing these habits, these behaviors for success. And as I said, structuring exercises and graded assignments that help them to do that, to hold them accountable.

Rebecca: Following up with what you just said, there’s a chapter in your forthcoming book called “Creating Autonomy.” Can you talk a little bit about small ways that we can give students autonomy in the classroom and in an online space?

Flower: Sure. And again, let’s be sure to keep that focus on small, doable, feasible changes, things that you could do in maybe a 15-minute work session and have it rolled out for your online class. One thing that we could do is to develop a self-enroll group structure. Many online faculty like to bring in collaborative learning tasks to, again, foster that community and the peer to peer instruction and learning that is so important as we know, but I think oftentimes we sort of assume that what we should do is purposefully group students, and there’s certainly value to be found in designing purposeful groups. But what can also be very interesting is to allow students to enroll themselves in groups that might cover a range of different topics. For example, sometimes I teach Educational Technology online classes. And if I were teaching that class today, I might offer five different groups that students can sign up for on a first-come first-serve basis. And one might be virtual reality, and one might be mobile learning, and one might be writing in digital spaces. So students could naturally choose a topic that they’re more interested in pursuing and when students have that level of autonomy, to make that choice of what their going to focus on, that’s one way of embedding just an opportunity for students to exercise that autonomy. Another even easier way I’ve already mentioned here is to offer students a choice between whether they want to submit a written task or whether they prefer to record on video or audio. Students carry these amazing devices in their pockets all the time with high- tech recording equipment embedded right in them. And students love the freedom of just being able to talk through their ideas, their responses, you can get a much more authentic response from students. Teach them how to use the recording software, or how to upload the video or the audio clip into the LMS and now you’ve got an easy choice that you can give students. If you prefer to write this, go ahead. If you prefer to record it, do it that way.

John: One thing that struck me is I used VoiceThread last year in an online class and I expected they’d actually use the video option with it very often. I gave them the choice of whether they use just voice or voice and video or use a video recording and yet none of them ever presented on video, which surprised me, given how common that is in social media. Why might that be?

Flower: Well, sure. I also require video discussions in some of my classes. And what I have learned is that people are nervous, especially in an academic setting about how they come across on camera. I feel like audio is a little bit less threatening, but sometimes people don’t like the way they look. And, you know, faculty too. [LAUGHTER] A lot of faculty are uncomfortable with recording on a video, and yet, it’s the way of the future. So right now, currently, I’m teaching my graduate level class on technological fluency and leadership. I require those video discussions and I say to them, “Are you nervous about doing this? Well, I want you to do it anyway,” because video interviews right, over Skype, or Zoom, as we’re doing here today, or video resumes. These are a thing that are happening and helping people to get more comfortable with showing their face on camera. I also talked to them a lot about the importance of seeing their peers faces and how much we can learn just from that. In the program that I teach now, students tend to take classes with the same people. But in my class, they always say, “I’ve never put a face to a name, how nice it is to see you,” and it makes a huge impact in terms of that community element. But I talk to my students very explicitly. Now, it’s also really important to think about situations where a student may not want to represent their face and there can be very good reasons. I had a tragic situation just last year where a student did not post the video, she posted a static picture of herself. I came to find out at the end of the semester. When she did that I was like, “Oh well, dock a few points, whatever. That was weird. Why’d she do that?” I later found out that she had been in a domestic abuse situation and she was ashamed of the way that her face looked because it was still very visible—the damage—and it just struck me to the core. An arbitrary decision that I made that it’s so important to talk to each other and look each other in the eye and she had a really, really strong reason for not wanting to do that. So back to that topic of offering a choice, what I do now is I tell my students, “If there’s a really good reason that you don’t want to show your face on the video, please send me a quick note. You don’t even have to tell me details, but just explain that you’re going to choose to do this other thing instead,” and posting a static picture is still pretty effective. So I think it’s very important to remember that our online students are people. They have lives and we need to be thinking about the decisions that we’re making in our teaching and how that might come across to a student… how it might induce anxiety in ways that we never anticipated.

Rebecca: One of the things that I wanted to follow up on and see discussed, the self-enrolling groups and collaborative work online. I think we have clear ideas about how that might work in a physical classroom, but not always a good clear way of how we can coach students through collaborative learning online. So even small, quick things that came up in Small Teaching, like Think-Pair-Share, you can envision how to do that in a classroom, but maybe have no idea how to do that in an online classroom.

Flower: It’s a great question, Rebecca. And I know that there’s actually a lot of pushback from online students and sometimes online faculty about the value of collaborative learning activities. It just so happens that my husband is in an online Master’s program right now and so I’m living with the student experience. And it’s frustrating to our online students—many of whom are not traditional 18 to 24 year olds, they might be returning adults—and one of the reasons (in fact, a primary reason) that our online students choose that modality is because they have busy lives. A big percentage of our students have jobs, families, obligations, and they need to do their work when they have time. That might be 8 pm, that might be 11 pm after all the kids are in bed. It might be 6 am, I like to do my online class at 6 am. When you require students to work in groups in an online setting, you’re removing that degree of scheduling flexibility that students value in an online class. So if you choose to require online activities, I have certainly moved towards lower stakes and opportunities that don’t require real-time meetings between students online. And you mentioned a great one. Think-Pair-Share can be set up in an online class. So there’s lots of ways that you can do this but the first thing that came to my mind is you could set up groups of two, and you could auto-enroll students in a group of two, and then they have their own individual discussion boards. In most Learning Management Systems when you have groups you can have kind of a private discussion board where students can interact with each other there, or I’m a big fan of letting some of the learning come outside of the Learning Management System. So let students know who their buddy is, have them exchange phone numbers, and they can just talk on the phone. Sometimes we forget those simple solutions. But a Think-Pair-Share—and so many ways that you could set this up, you could change it from module to module so people are always working with somebody else—just share an idea, discuss something, take it offline, come back and just write or record a quick summary of how that interaction went. When it’s not such a high stakes assignment, students can better engage in those opportunities. It’s so much easier to find 15 minutes to talk with one person than it is to find an hour with four working adults who all have family obligations. So I love the idea of lowering the stakes and embedding lots of little opportunities for students to work in pairs or in groups of three where it’s easier to coordinate. There’s less pressure about the online group member who never does the work—sorry, but that’s a thing—and just help students see other ways of interacting. Now, with my instructional designer hat on I want to remind us of the importance of making sure that online collaborative work aligns with the outcomes of the course. Very important to think about why you’re asking students to work together. Does this actually relate to what you want them to learn and get out of the course? Very important to pause, ask yourself some of those questions before you randomly assign group work because we should have group work, which I’m guilty of doing. [LAUGHTER] It’s an easy thing to do, “I guess we should have group work,” but really pausing to think carefully about the purpose of that. And then again, maybe thinking creatively about those lower stake ways of connecting students and facilitating some more authentic interactions. Maybe they’re going to text each other. That’s fine, they’re talking. We do a lot of talking on text these days. Help students connect in ways that are not so stilted, which is often what we see in the use of the discussion board and the LMS.

Rebecca: I found that too, I use Slack a lot in my classes, because it’s a common platform for designers and people in that realm to communicate professionally, and they love it. It’s convenient, it’s on their phones, takes it away from a clunky interface…

Flower: Sure.

Rebecca: .. some of the LMS’s have and it’s really productive. And they’re able to do that midnight chat with each other.

Flower: Yes, absolutely. Again, let’s think creatively about tools that students already have. I honestly believe that a lot of Learning Management Systems actually raise barriers to student learning because most of them—although this is getting better—most of them don’t have a super robust mobile app and so a student, really to engage with coursework, has to find a place where they can sit down and log into the computer and access the course and jump through a million hoops before they can even get to where the learning is. Whereas if we take some of that learning into apps that they’re already using or things that they’re doing on their phone anyway where it’s in their pocket, we can communicate in real time. Now I need to exercise caution here because many faculty think, “Oh great. I’ll do Slack, and I’ll do VoiceThread, and I’ll do Flipgrid, and I’ll do Twitter, and I’ll do Pinterest, and it’s just going to be so interesting and fun.” Well, if there’s a reason for using some of those tools, absolutely. If those tools are just shiny entertainment—bells and whistles—then you may want to think again. Another important consideration if you’re asking students to use tools that are not in the Learning Management System is whether those tools are fully accessible for students, whether there’s any fee that’s involved, whether students might have to set up a new account with a new password, that might just be a hassle. So really you want to think carefully about what you’re asking students to do. Are the tools fully accessible and usable and cost friendly? Do they support your learning outcomes? And yet, if a tool that you’re thinking about using passes all those tests, then by all means jump right in. This semester I’m using Remind which is the simplest tool on the planet and the most effective. [LAUGHTER] It’s more in use in K-12 currently than in higher ed. It’s simply a text app that anonymizes people’s phone numbers. So I invite my students to sign up for my Remind list. I don’t require it. But then I can easily send a quick little 140 character reminder, “Don’t forget this assessment is due on this particular day,” or “New content has just been released. Login when you get a chance.” The message goes right to where the students are and because I make it optional, nobody is required to have the annoying instructor on their phone all the time. But students who want some additional support with managing deadlines and the class experience really appreciate the use of the simple tool called Remind.

Rebecca: I think one of the things that we know from a lot of evidence-based practices and books that have come out—including Small Teaching—is that frequent feedback is useful. But we also know that frequent feedback can seem really daunting to a faculty member, and time consuming. So are there ways that you would suggest managing some feedback opportunities online, but keeping it easy, quick, and reasonable?

Flower: Sure. Another great question. Another underutilized approach—at least in my experience supporting the faculty that I work with—is the ability to embed feedback into auto-graded multiple choice or true- false types of quizzes within the Learning Management System. So in most of these systems you can design feedback that will show up for students as soon as they submit the quiz. You can set those quizzes to show students which questions they got right or wrong and in the wrong answers you can embed feedback that says, “Please review pages 32 to 35 of this chapter. That is where you’ll find this information.” Similarly, you could encourage or embed challenging feedback and by that I mean, “Great, you totally know this material. If you’re interested in learning more, you may want to check out this website or this resource,” to offer students a range of experiences and engage students at their different levels of experience with the content. To be fair, setting up that kind of embedded feedback takes a little bit of time in the first place, but many of us teach those online courses over and over again, and once you’ve done that work, you can benefit from it time and time again. If you’re not sure how to do that in your Learning Management System, just about every institution has a Learning Management System support team with instructional designers or system admins, help desk folks who can walk you through the creation of that kind of embedded feedback. And it’s timely, it’s right there when the students are thinking about that problem in the first place, it’s relevant, and it’s a great way to automate some useful feedback for student learning.

John: You have a chapter in this forthcoming book on developing as an online instructor. Are there some general suggestions that you can give to faculty who’d like to improve and develop new skills or improve skills as an online instructor? Besides buying the book, of course. [LAUGHTER]

Flower: Right. Great question. Again, what this comes down to for me is that it’s just new. It’s just new for a lot of people. And to be honest, I suspect that many online faculty didn’t really set out to be great online faculty and many faculty are not finding the experience quite as rewarding as they might find the classroom experience. In fact, I have some data to back me up on that. The 2017 survey of faculty and information technology from EDUCAUSE Center of Analysis and Research found that of over 13,000 faculty respondents, 91% said that they don’t prefer to teach online. 9% said, “Great, I love to teach online.” That’s 91% of us who would rather teach anywhere else. [LAUGHTER] So how can we cultivate that joy, that buzz that we get in the classroom? We love teaching. If we didn’t, we wouldn’t be doing it because we don’t get paid enough. How can we cultivate that for ourselves? Now a barrier or a common challenge is time. Who has time to go and learn how to do a whole new skill? It’s different than teaching in person. But there are, again, small things that we can do to increase our awareness. One of the most effective things that you can do as an instructor is to seek an experienced and a thriving online instructor and ask to shadow that class. Ask to be added into that class shell and just observe. How does that person interact with students? What are the structures? What is the teaching? What happens while the class is in session? That can be hugely impactful, it’s usually free, [LAUGHTER] and faculty can invest the amount of time that they have. In fact, this is how I first got started with online teaching over 10 years ago, is before I was going to teach a class. Luckily, this offer was made to me the semester prior to my first online class, just to observe another class and see what happens in there. Simple structure could easily be set up for faculty who are scheduled to teach a new class in the Fall, have them observe or shadow a class in the Spring or the Summer, and yet an often overlooked solution. Certainly there’s lots of online resources. There’s podcasts like this, there are blogs that people are writing about innovative things that they’re doing, but sometimes just finding a thriving online faculty to interact with, shadow, observe, be mentored by, can be the most effective way to learn how to do this better.

John: I even sometimes encourage faculty to join a MOOC, because often you can find some interesting practices there that scale without necessarily requiring much effort on the part of the instructor.

Flower: That’s right. That’s one of my other recommendations and I hope I haven’t given all of the book away here. [LAUGHTER] But one of the other recommendations in that chapter is just to take an online course in whatever form that you can. Whether it’s a MOOC, a lot of organizations like the Online Learning Consortium, Quality Matters, offer online professional development opportunities for faculty. Even if it’s not about teaching online, just go take a class that is online. Or maybe personal interest. Sign up and take Spanish online. And having the experience as an online student is hugely impactful to help you understand what your students are going through. Even as a faculty if you’re taking a course and you’re reading the instructions going: “Now, what am I supposed to do with that?” Immediately, you have much more clarity about what your students might be experiencing and then you can take steps to address those kinds of gaps or areas of concern that might be in your own class… that you may not have previously seen before.

Rebecca: I think the recommendation of taking a course outside of your normal domain or area of expertise is key because you’ve got students who are in an environment they’re not familiar with, with a topic they’re not familiar with. And so to kind of simulate that, I think is key.

Flower: Right?

Rebecca: I know I’ve done that in the past and it’s like, “Oh yes, I forgot what it was like to be a beginner.”

Flower: Absolutely. In fact, I had a really interesting process or experience this past fall semester where I was supporting a redesign in a large cap biology class of liberal studies—or general education biology class—large enrollment. My background is in English literature… the humanities. I don’t think I ever took a hard science class in college because I did an honors program where we could do more sort of ethical concerns related to science. But I went to that class frequently throughout that semester and I clearly remember the first day. 240 students and me and I was sitting in the lecture hall with the students and it was just very, very impactful. Putting me in a situation that was foreign to me—I don’t teach large cap classes, I don’t know a thing about biology—I do now, I know a little more [LAUGHTER]—But I was a novice learner in a very foreign environment and that’s what our students are in our online classes, which is really quite anxiety producing if you think about it. Going into an unknown space, not knowing what’s expected, you don’t know how to get ahold of your faculty member a lot of the time. So just being intentional about helping students be more comfortable and more at ease in our online classes—be more available to them—can make a big difference. And again, you get that insight differently when you choose to place yourself in a situation where you’re a novice, and you’re not really sure what to expect. That’s a great point.

John: Are there any other topics that we should address that we haven’t raised yet? Anything else you’d like to emphasize?

Flower: You know, really only one thing comes to mind and that is an insight that I had literally this past week, which is that I feel like sometimes online faculty—myself included—have somehow developed the notion that we don’t really need to talk with our students. And let me explain what I mean by that. Again, I’m teaching an eight week—it’s an accelerated graduate level course right now—I’m busy. My students are busy. And on a whim a couple of weeks ago, I said, “Well, I know you have this assignment coming up by Sunday night, I’ll be available on Saturday between the hours of 1 to 5pm.” I don’t like to work on Sundays. I tell my students that if you want to just pick up the phone and call me on Saturday, go ahead. So that weekend, I did. I had a student who called me and she was a chatty Cathy, and we stayed on the phone for quite some time, but she got a better understanding of the assignment and how to be successful. Well two weeks later, which was this past weekend, it was my daughter’s 11th birthday and I was right in the middle of finalizing all the food preparation and everything else. And lo and behold, there’s my phone ringing and I can tell that it’s not a connection of mine. And I went, “Uh-oh, it’s one of my students,” [LAUGHTER] because I had said Saturdays 1 to five and that same student who had called me a couple of weeks prior called and we had a great conversation. 15 minutes, I was able to keep chopping the carrots while I was talking with her. And it just occurred to me, that wasn’t really a convenient time for me personally because I was doing that final party prep, but so what? The student needed help in that moment and just taking the time to answer the phone and talking through a couple of quick questions, it was helpful for her, and it just got me thinking about how, you know what, I don’t think a lot of us really talk to our online students, like, literally talk on the phone. I know some faculty have the online office hours, I know people are using video conferencing systems, I’m available, but one of the things I’ve started doing is just saying, “Hey, if you have a quick question, just call me. We’ll talk it through.” And sometimes a five-minute conversation can ease that student’s anxiety and answer a few questions. This happened to me again yesterday where a student was like, “Before I submit tonight, can I please just check in with you?” I talked with her while I was commuting to campus and it’s just a way of talking person-to-person, humanizing the online learning experience. But like I said, I think somewhere along the line personally I had formed this opinion that we don’t actually talk to our online students. And I don’t know why that’s a perception because if you’re teaching in person you talk with your students. If there’s somebody who has a question after class, you stay a few minutes after and answer those questions. But I think for online faculty somehow we’ve missed that connection and it can be a powerful and so simple solution to helping our students thrive and succeed. I think faculty and students both overlook some of those simple solutions. It doesn’t have to be a long, tedious, written interaction in a discussion forum. It could be a phone call, and so much can be conveyed through the tone of voice and emphasis, just as I’m doing here today. And as we all do, when we’re teaching live. Just picking up the phone and calling the students or inviting them to call you. Simple, powerful.

Rebecca: I think you’re pointing to something that I know I’ve experienced even though I don’t teach online regularly. It’s just online communication is always written and it feels daunting and it feels really time consuming. And it feels like, “Oh I got to sit down and dedicate time to do this.” So it’s nice to be reminded that there’s other ways to respond.

Flower: Just in my own work somewhere along the line, I forgot about the phone in my day-to-day job. My full-time job is as an instructional designer and it seems like we never just pick up the phone anymore. It’s always email. And as you said, it just takes longer, especially if you have a little bit of confusion and you’re going back and forth on email. I literally in the past few months, I’ve just remembered how to pick up the phone and call somebody. Have a five-minute conversation, you get your questions answered. And just reminding ourselves of the importance of real- time interactions sometimes, and moving away from the requirement that everything needs to be written all the time. I’m a big fan of video announcements, I do that all the time in my online classes and again, the reason I do it is because tone of voice, inflection, emphasis, and funny faces sometimes, or just emphasis where I might just kind of widen my eyes a little bit to explain that, you know, “This is really important. Pay attention and focus.” Just finding these other forms of communication apart from writing can make a big difference in the online learning experience as well.

John: We always end with the question, what are you doing next?

Flower: Well, I’m not quite done with this project. [LAUGHTER] So I’m wrapping up this book development. But what’s really making me passionate now is to really focus on being a crusader for online education. It’s undervalued. It’s under-supported. I know that faculty don’t see the joy of teaching online and I know that students approach it the same way like, “Well, I have to get this degree and I guess this is a convenient way to do it.” I just want to advocate for how online learning and teaching can be impactful, can be rewarding, and joy giving, and you don’t see that reflected even in the coverage of teaching in higher education. Most of the time, the focus is on what we’re doing in the classroom and that’s so important, but there’s a big gap. What are we doing in our online classrooms? I just want to move into that space and encourage people to think about how they teach in person, and how to do those things in their online classes in ways that are not so daunting that they never get around to it.

Rebecca: This has been really great. I’m looking forward to picking up your book and maybe thinking about teaching online. [LAUGHTER]

Flower: That’s right. And if you don’t mind, Rebecca, I’ll just pick up on that, which is that again, I think a lot of faculty don’t say, “Hey, wow, what a cool opportunity. I totally want to teach online.” For many faculty it’s a daunting prospect, “I don’t know how to do this.” But it can be a really great way to reinvigorate your teaching—to find new ways of finding and addressing those challenges. Keep in mind institutions have the support professionals, instructional designers and such, who can help if you’re thinking about moving into online teaching. Talk with some of those faculty support folks, talk with your colleagues, and jump right in. It’s more fun than a lot of people think.

Rebecca: Well thank you so much for joining us today.

John: Thank you.

Flower: Thank you. What an absolute privilege and honor to be here. Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, and Jacob Alverson.

67. Iterative OER Development

Imagine course materials that are always up to date and evolve continually to become better at supporting student learning. In this, Dr. Steven Greenlaw joins us to discuss how some publishers of open educational resources are trying to set up sustainable practices to achieve these goals. Steve is a Professor of Economics at the University of Mary Washington and the author of the OpenStax Economics textbooks. He has also developed the materials for Lumen Learning’s Waymaker Introductory Economics texts.

Show Notes

Additional Resources

Transcript

John: Imagine course materials that are always up to date and evolve continually to become better at supporting student learning. In this episode, we discuss how some publishers of open educational resources are trying to set up sustainable practices to achieve these goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Steven Greenlaw, a Professor of Economics at the University of Mary Washington and the author of the OpenStax OER Economics textbooks. He has also developed the materials for Lumen Learning’s Waymaker Introductory Economics texts. Welcome, Steve.

Steve: Thank you.

Rebecca: Today teas are:

Steve: I’m drinking coffee. Thank you.

John: …and I have Enchanted Forest Fruits black tea from Epcot which I picked up while I was out there for the OLC conference where I last saw you, Steve.

Rebecca: You’ll never guess what I’m drinking.

John: English afternoon?

Rebecca: Yeah, it’s my favorite.

Steve: Well, honestly, I switched to tea in the afternoon.

Rebecca: See…

Steve: But in the mornings, I tend to drink coffee.

Rebecca: Yeah, you and many other people.

John: What prompted your interest in using and developing OER materials?

Steve: I have to say the developing came first. For a long time, I’ve experimented with textbooks going back into the 1980s, which at least John can remember. And I came to the conclusion that that it didn’t really seem to matter what principles book you used. Students needed a book, particularly for the analytical parts of the course: the models and things like that. But whichever book I used, they seem to learn just as well. And more recently, I’ve come to the conclusion that intro textbooks are commodities, that where companies are going to make their money is in the aftermarket products. But we’re not there yet. At least, the majority of the textbook industry is not there yet. So I had that and I didn’t really pay much attention in the 2000s about what textbook I was using, because I didn’t really think it mattered. But I did notice how high textbook prices were going and it was around that point that I became aware of and interested in OER. Again, this is dating myself, but when I was in college during the mid-1970s, I remember a teacher in my intermediate macro class—John, that’s for you—saying he would never assign texts for a course that collectively cost more than $10. [LAUGHTER] And so that’s sort of my base year. So, I sort of had this in the back of my head, I basically tried to choose around the least expensive textbook that I thought would work. And then out of the blue, OpenStax contacted me and said they had funding to create a principles of micro-macro text, and would I be interested in helping them out. I actually jumped at the opportunity, it sounded like a lot of fun. At that point I had already published one textbook commercially for an upper level course and I knew something about the commercial publishing process. I knew that I didn’t really want to go through that again, but I did want to get my ideas out there. One of the things about commercial publishing is they ask you, “What are all the innovative things you want to do?” and then once they have you on contract, they say, “Oh, but you have to do it like everybody else’s.” So that was the start. A year after the OpenStax book got published, I got contacted by Lumen Learning who said essentially the same thing. They said, “We’re building this digital platform, and we wondered if you would like to be the principals subject matter expert.” That’s the term of art that I become a SME.

John: So could you tell us a little bit about Lumen Learning’s project and the Waymaker version of this?

Steve: Sure.

John: What does it add?

Steve: It adds a lot. So, just to be clear, I wrote the OpenStax principles book. And we can talk about that process later if you want to, especially about peer review and things like that. And then I wrote the Lumen Learning Waymaker version, which was essentially an improved version. When we did the OpenStax principles book, we did it in an incredibly short period of time, I think it was nine months. So, when I did Waymaker for the first time, it allowed me to flesh out some of the things that weren’t ideal in the OpenStax book. And then OpenStax came back to me maybe three years ago and said, “We have funding for a new edition. Would you like to do that?” so I wrote a second formal edition for the OpenStax principles book. And then right after that, I did the same thing for Lumen. So, in my mind, I’ve gone through four versions of this now. And it’s not done and that’s part of the beauty of OER… at least the OER business. So to get back to your question, the OpenStax principles book is a textbook, it’s available in print and a variety of online options. My particular favorite is the phone app. So if I’m in class and a student asked me a question about something, I could literally look it up on my phone. Waymaker is a very different animal. It’s digital courseware so it’s a more immersive, interactive experience for students. And it’s not available in print. For example, how would you show a video or do a simulation in a print textbook? You can’t. The most you could do was provide a URL or something and have the student go out to that. In Waymaker, it’s all in one. So Waymaker, aside from text, it includes video, it includes animations, it includes simulations. Just to give you a specific example, instead of students looking at a graph of supply and demand, they actually get to climb in and take it for a test drive. Students really liked that. Many students seem to get it in a way that’s just looking at a two-dimensional graph, or reading text it is much harder for them.

John: I saw you present on this at the OLC conference…

Steve: Yep.

John: …And you demonstrate this. What software did you use to create those interactive graphs?

Steve: Those little interactives are H5C… maybe… it’s called? [It is H5P]

John: Okay.

Steve: It’s a European company, and it’s open source, and it’s really easy to do. I can say that even though I didn’t create the interactives. That’s the joy of working with a company… they actually have people to do the stuff that you don’t know how to do… unlike my earlier career, when I was the programmer, I was the graphic designer and all of those other things. Talking a little more about Waymaker, it’s more than a source of course content. It’s designed to teach students to study more deeply and more effectively. I don’t know about your students, but my students don’t seem to have learned how to study well. They’re very good at the game of school, but they’re not so good at learning. And I don’t mean that as a criticism. It’s just sort of a fact. They think study means read, highlight, read again, highlight again. When we know a lot from cognitive science now, that learning comes from working with the material. As I like to say, “the best way for students to learn economics is to do economics.” So Waymaker emphasizes mastery learning and personalized tutoring. The tutoring comes both from the software and also from the instructor. It’s designed to give students actionable feedback so that they can make their own decisions about how to allocate their study time. This is a really different way of learning, so I’m going to say it again a little bit differently. Assessment is integral to the learning process, it’s not just or even primarily about the grades. Rather, the assessment is designed to make students interact more deeply with the content and interact in a more intelligent meta-cognitive way. I can go into more detail about what it looks like from the students perspective, if you want.

John: Sure. Could you talk a little bit more about that? It’s a great approach. I tried to do that myself, but it’s always an add-on. Having it integrated is a nice feature, and one of the reasons why I’m planning to adopt your package in the fall.

Steve: This is really different for students, but also for the instructor. I’ve been working on this product for three years. When it finally came out in beta, I thought I knew what was going on, and I was really surprised at how little I knew about how it actually worked. Waymaker is organized into modules, which are analogous to chapters in the text. Students begin each module with the “show what you know,” which is basically a formative assessment. The purpose of that is to identify what content they already know. So, it gives them feedback on how they can efficiently use their study time. So, if there’s stuff that they absolutely already know, they don’t need to read about it again, they can just go into the stuff that they don’t know.

John: And even if they don’t, it activates prior knowledge. And it helps them make connections so that they can learn more effectively…

Steve: …Yes.

John: So there’s a lot of benefits, even for the areas they don’t know.

Steve: Yes. And I’m actually adding a little exercise for my first day of class next week, where I put my students in small groups. Some of whom who’ve had the first semester, and some of them who have not. And I’m going to give them a basically a problem to work with, knowing that some of them won’t really know what to do with it. But I want the groups to start working together. But anyway, I digress. So, as students progress through the content, there are a series of learning activities. The original one is called a “self-check.” It’s basically a short formative quiz. The purpose of the quizzes is not summative assessment. But as I said before, it’s to help students think more about their learning. Think about the idea of a Socratic tutor. The tutor doesn’t ask questions to assess the students’ knowledge, but rather to help them work through the content, help them really understand it. So what happens in Waymaker is: the student reads a page a text, or watches a video, or plays a simulation. And then they’re posed a very short quiz, like one or two questions. If they pass the quiz, the “gate” opens and they move to the next section. If they don’t pass the quiz—and on a one-question quiz, either you get it or you don’t—Waymaker suggests that they review the content before attempting the quiz again. They can take those quizzes as many times as they want to. So they can really build some expertise. There are other sorts of learning activities, but I want to focus on the quizzes today. At the end of the modules, students take a module quiz—essentially a chapter test—which is summative. Again, if they fail to achieve mastery—and the default mastery level is 80%, so it’s pretty high level. As an instructor, you can change that to whatever you want. But I like 80%. So if they don’t achieve 80%, they’re encouraged to study again and they’re given information about what areas to study. And then they can take the module quiz one more time. They’re only allowed to take the module quizzes twice. Now, here’s where it starts to get really interesting from the teacher’s point-of-view. The instructor receives reports from the module quizzes whenever a student fails. So for me, the first really good thing about Waymaker was that I don’t have to go to some website and look at some spreadsheet and see which students are struggling. Rather, anytime a student fails, I get pinged from the software. So it says, “so and so…” Well it’s a little boilerplate language… but basically it says they worked through the module, and they scored a 46 on the module quiz. You might want to reach out to them at that point. So the software is flexible. So you can get these things in real time, you can get them once a day, you can get them once a week, if you want to. I get them once a day. That seems reasonably quick for me. If the students taken the quiz at three in the morning, I’m not up anyway, so it hardly matters. It’s not like I’m going to give them that fast feedback. But what happens is I get that information, and then I get to decide, “What am I going to do about it?” If someone gets a 76 on their first attempt, I generally figure, “Okay, they’re gonna figure this out.” And so I don’t worry about it. If someone gets a 46, then I immediately want to reach out to them and say, “Hey, I see that you’re struggling with this. You know you can take it again. Go back and review the material. And if you’re not sure that you understand it, let me know and I will work with you on this. Because the goal here is mastery. It’s not anything else. Anyway, Waymaker helps me, the instructor, make better, more efficient use of my time. In any given week, Waymaker allows me to know two important things. It allows me to reach out only to those students that need my help. And it lets me know what topics the class is struggling with, so that I can tailor my in-class time to the material where the students need help and not spend it on material where they already know this stuff. Basically, it gives me a better feel for the effectiveness of my teaching and student learning. And that’s really, really important I think as a teacher. I’m embarrassed to think of my early years and teaching, when if I got all the way through the 50 minutes, I counted that as a successful day.

[LAUGHTER]

John: I think many of us started like that.

Rebecca: It ties really nicely to your blog post series that you’ve just recently published. The first one being the critical importance of instructional design…

Steve:Yup…

Rebecca: …where you talk a lot about the instructor’s role is designing the experiences, rather than delivering content. Can you talk a little bit more about how Waymaker helps you do that as an instructor?

Steve: There’s a “just-in-time teaching” element to this. I have a course outline, I know what I’m supposed to be doing on a week-to-week basis. But what happens on any given day depends on the stuff that came before it. I’m absolutely not wedded to the calendar. If the students haven’t figured out what we did on Monday, I’m going to start by spending a little more time on that. But also because of the feedback that I’m getting from Waymaker, there are times when I spend 90% of the class on 10% of the material. Because that’s what I know students are having trouble with. I know that if it’s something analytical, probably what I’m going to want to do is instead of talking to them about it—I mean, certainly I’m going to talk to them about it—I put together some group activities. I do a lot of group activities, small groups, generally two to three people. And then I essentially turn the classroom into a lab experience for that day. They seem to enjoy it more, they seem to get more out of it than me just lecturing over the content. After all the content is in the book. I don’t need to just repeat that stuff. So I guess that’s my short answer to your question.

Rebecca: Can you give an example of the kinds of activities that you’re doing with your students?

Steve: Oh, sure. Supply and demand is the first real model that the students work with. And so one of my learning goals is that they ought to be able to take a scenario… something happens… use supply and demand to analyze the effect on the market for x, gasoline or something like that. Typically what happens is, hopefully they will have read the material in Waymaker. Typically, I spend a day talking about “here’s how you would do it” and then generally what’s going to happen is, I spend a day where I have a couple of problems, like three is all that we’re going to have time to do. And I say, “Get in groups of two or three.” Basically, I count the number of students that showed up that day, because my classes are pretty small. And if it’s divisible by three, I put them in groups of three, if it’s divisible by two, I put them in groups of two. And then I say, “Okay, here’s a problem,” I show them the problem. And I say, “Take 10-minutes to work through this, draw the graphs.” And then they know that I’m going to call some of the groups up to present the results to everyone else. So there’s a little bit of competition. It’s not very stressful. It’s a little stressful for people that don’t like to speak in class, but you’re not there by yourself. You’re there with your group, so it works better that way. So I do a couple of those problems until I’m convinced that most people know what they’re doing. So that would be an example.

John: You also mentioned—when I saw you present at the Online Learning Consortium—how you use some of that feedback to improve the text in your current edition. Could you talk a little bit about that process of revision and creation of the text?

Steve: Sure. While I can’t take all the credit. From the beginning of Waymaker, at least from when I began to get involved… once I realized how integral the assessment process was to Waymaker, I pressed Lumen to make sure that the assessment questions were good. One of the things that I’ve noticed over the years is, test banks seem to be the lowest priority of textbook publishers. Because after all, they’re selling the text but they’re giving away the test bank. So what I want, I guess what we all want, is that the questions in the test bank that Waymaker uses, are discriminating correctly. And that’s harder than you might imagine. To their credit, Lumens put a tremendous amount of effort into this. And more generally, into the design the courseware. This has resulted in a process of continuous improvement. Now, continuous improvement is not a term that excites most faculty. I think that’s a fair statement John? [LAUGHTER]

John: Yes.

Steve: But what it really means is that, Lumen has an ongoing process for improving OER, making it more effective every single semester. And they’ve done this, and we’re now in your five and a half. So how does it work? I have a short answer and a long answer. The short answer is, after every semester Lumen downloads the data from every student who’s given them permission at every school using Waymaker across the country. And then they analyze the data. The analysis identifies where the students are having problems. At that point, we go in and either revise the content to make it clearer, or add some learning activities. Or else we revise the assessments to better capture student learning. We do this a little bit in a panicky way over the winter break, because we only have a month. But we do it intensely every summer. Here’s the longer answer. Over time, we’ve gotten better at doing this more efficiently. Lumen has developed something called “RISE Analysis.” RISE is an acronym. I don’t remember what the letters mean. [LAUGHTER] But basically it asked the question, “Which course materials would benefit the most from improvement?” Or to put it differently, “Which changes would have the greatest impact on learning?” So what we’ve done—and this is all programmed now. So Lumen has dozens of Waymaker courses, not just an economics. Though, I like to think that some of the most interesting stuff is started in the econ Waymaker platform. I’m not just making that up, it’s actually true. [LAUGHTER] So, instead of just doing the aggregate sweep on the data, we particularly look at student learning outcomes. And everything in Waymaker is driven by the student learning outcomes. This is out of order, but let me just throw this in for a minute. The way that Waymaker started is they brought together—I want to say 50 principles instructors from everything from community colleges up to R1s. And we spent four days together. And we asked the question, “What do you have to have in your principles courses?’ And so from that we created a list of primary learning outcomes. And then we drilled down and we now have secondary and tertiary outcomes. So the assessment questions in the test banks are coded down to the third level. So everything is really granular, if you want to think about it in those terms. What we look at is not just which student learning outcomes are students struggling with. But rather, which student learning outcomes where students are doing relatively poorly, are they putting a lot of time and effort into. Because that’s where we’re going to get the biggest bang for the buck in terms of fixing things. So what we do is we look at three things. We look at, “Are the questions badly worded?” We’re mostly done with that at this point. “Are the questions testing what they’re supposed to be testing?” There are some psychometric tests that allow you to do that. And then finally, what we do is—after we’ve exhausted all those—we look at the content and we create new content, or different types of learning activities, and we integrate those into the course. So, the interactives that you saw at OLC John, they were the big new innovation from last summer. So we do this, and then we teach the courses again, and then we start the cycle all over again. So, the process just goes on. It’s not continuous, as in every day, but it’s continuous, as in regular. I’ve used the courseware since the first year, and the courseware has gotten noticeably better. Fewer students are failing to achieve mastery on the module quizzes. And fewer of them are crashing and burning. More of them are in the 60 to 70% range when they fail. But what’s really cool is Lumen has shown no sign that they’re ready to quit, that they’re done with this. As long as they’re willing to do this, I think I’m willing to do this.

Rebecca: I like the iterative process.

Steve: Yeah.

Rebecca: That’s something that, as a designer, I’m very comfortable with… that I do all the time, especially designing online. But one of the things that’s really interesting about this model is that, as the author of the textbook, you don’t just have this finished thing. It’s an ongoing…

Steve: …It is.

Rebecca: … thing. So that’s a really different model of authorship.

Steve: Yes, it is. I think it’s fair to say that we make small changes all the time. And then every summer, we make larger changes. And that’s pretty interesting. Because as a user—as you pointed out—I can see that this is helping.

Rebecca: Yeah, that’s really exciting.

Steve: Right now, the hardest part is getting students to trust the process. Because it’s a very different model of learning. And so one of the things that I’m going to do this semester is, build in opportunities for me to remind them that this is a different process, and that they need to trust the process. One of the things that I did last year, which seemed to help with that was I started using exam wrappers after the midterm exams. And ask them to think about how they were studying, and what they would do differently, and what I could do to help them. It’s real easy to see in 30 seconds, I can tell if they’re taking it seriously or not. And if they’re taking it seriously, I learn a whole lot from what they say. So, anyway, just another little wrinkle.

Rebecca: So we’ve talked a lot about the students and the different learning process for students. You talked a little bit about the different processes being the expert, or the writer of the book. And you also mentioned earlier about the peer-review process for an OER being a bit different. Can you talk a little bit about that?

Steve: Sure, and that’s really important. First of all, people have a wrong idea about how OER is produced. The OER that I have experience with is working with OER publishers. It’s not the loan faculty member working in their spare time in their basement, or something like that. Both the OpenStax and the Lumen experience for me, have been very much a team effort. There have been a lot of people involved. So this is really important because one of the concerns about OER textbooks is their presumed lack of quality. There was an article in The Chronicle about that, today in fact. I have to tell you that the peer-review process that I went through with OpenStax was extensive. The way we did this is, OpenStax purchased a manuscript from Tim Taylor—a prominent economist—as the basis for the first edition. They sent copies of the manuscript out to about two-dozen reviewers all over the country, asking them to identify strengths and weaknesses. Based on those review comments, I rewrote each chapter. Each chapter was then sent out to half a dozen new reviewers. And again, the reviewers were from a range of schools, from community colleges through research universities. I took that feedback and I revised each chapter again before it went through the editorial review and production process. I have to say, this was much more detailed and extensive then when I worked with a commercial publisher. The review process for Lumen was similar, there was a lot of peer review involved. And as I said before, I’ve now written two formal editions of both texts. We’ve gotten lots of feedback from users. I’m pretty happy with that.

Rebecca: Do you find that the difference between OER and a commercial publisher is that you keep getting this feedback from users? And that you’re able to revise based on the use of other faculty, rather than working in a silo?

Steve: If I’d written the principles book for a commercial publisher, I would be better able to answer that. I got no formal feedback on my commercial book. I got a lot of comments from people at conferences and things like that. But we have gotten tons of feedback on the OER books, and that is interesting. You can’t satisfy everybody. Somebody says, “This chapter is too long.” Somebody says the same chapter is too short. But, in general, the feedback has been really, really helpful. And we’ve tried to incorporate it as soon as possible. And with these digital text, it’s really easy to do. I can literally go in and edit if I have five-minutes on the fly. And then it’s out there.

John: While with regular publishers, there’s usually a three-year cycle on intro textbooks.

Steve: Yes. And that’s the other thing that—now I’m not a typical user, but I know that if I want to make a change, it’s going to be done by the next semester. The same thing is generally true of other people who give us feedback. Though, they don’t necessarily know that. W e take that feedback very seriously. And there is no three-year review process. So that’s wonderful.

Rebecca: I love the user-centered design process, like that’s clearly what’s being used.

Steve: Yep, we try.

John: And that iterative process is what we should all be doing with our courses, all the time…

Steve: …Yes.

John: …But the fact that you’re doing it makes it easier for instructors who perhaps, don’t have to do as much of that.

Steve: Yeah. But again, let me just say one thing; Waymaker is not my course, Waymaker is my text. So there’s whole levels to my course that go beyond Waymaker. That’s just one element of it. Not that I’m disagreeing with what you said.

John: I’ve seen you present at conferences on teaching principles, for decades now. And I know you’re constantly changing how you’re teaching your courses and trying new things there. And you’ve been doing a lot of great work for quite a while.

Steve: Thank you.

John: Going back a little bit though, to the question of mastery quizzing. When students take the quizzes at the end of a block, you said there’s one or two questions. When they do it a second time, do they get the same question or different questions?

Steve: No. We are adding questions fairly regularly, and so the test banks are getting larger. From the beginning, I think we started with 2000 questions. But again, that’s across the whole book. The questions are randomly chosen, so the odds are that students would get different questions at the self-check level, at the section level. There’s a different test bank for the self checks than there is for the module quizzes. But there are similar questions. In fact, we wrote two at a time basically when we did that.

John: This is a question more generally about Waymaker. Does it do any type of interleaved practice, where later in the course, does it call back earlier sections? Or is it just based on the current module?

Steve: No, it’s just based on the current module. But my more nuanced response to that is, economics is sort of cumulative. But I have thought about that, we just haven’t thought of a way to build it in yet.

John: In my classes, I’ve been adding that the last couple years where I just randomly pull in questions and the module quizzes from earlier modules. Maybe 10 to 15%, building up to about 20% at the end, just to help do a little bit more spaced practice as well.

Steve: I think I know how you could do that pretty easily. Because instructors have access to the test bank that their students are using, so that you can edit your own questions. But what that also means is that you could move questions from earlier into the course to later in the course. So I think there’s a way to do that.

John: Excellent.

Steve: So John, we learned all this in our graduate training, right?

John: [LAUGHTER] You know, it’s getting a little bit better. Some people are learning these things. We have someone in my department who actually came out of Kentucky where he had a lot of training and teaching and learning. But it’s still pretty uncommon.

Steve: Yep.

John: You mentioned two ways in which, OER materials are developed. Some by primary developers, such as the OpenStax and Lumen. And others, with people working in their basements…

Steve: …Yes.

John: …or working in a dark room somewhere. Which is how I often do a lot of my work. Is that process sustainable? And what role do for-profit publishers such as Lumen play in providing these services, or in continuing the development of OER materials?

Steve: There are a couple questions here. One is, is the development process for published OER materials, or OER materials created by publishers. Is that sustainable? And then the second one is, is the individual scholar model sustainable? And those are very different questions. The individual scholar model, I don’t know if sustainable is the right word. I have a colleague who did this, she did it all on her own. I’m so impressed. She didn’t have any support from the school other than a small summer grant. And she did it without any sort of extrinsic motivators. I think that over time, at least at schools like yours and mine, faculty are going to get credit in tenure and promotion, for creating OER, especially open textbooks. I think that’s really important. I think that people will eventually be able to get sabbatical leaves to create these materials. And I think that’s really important to keep that side of the OER creation process going. As far as revision, I don’t know enough about that to really answer that. But I’m curious. I may have to go talk to my colleague Katie now. As far as the publishers go, and I don’t mean the traditional publishers, every publisher has a plan for how they’re going to do this. Some work better than others. I know something about OpenStax and I know a lot about Lumen, about what their sustainability plan is. OpenStax have develop partnerships with a variety of ancillary publishers like Sapling Learning or Knewton. These people provide aftermarket functionality for the OpenStax books, and in return, they get kickbacks from these ancillary publishers. And by kickbacks, I don’t mean anything pejorative about that. I just mean that they contribute financially. I don’t know any more about how sustainable that model is. I know that that’s what OpenStax has been using. Lumen from the beginning, has been a commercial publisher. It took me two years to figure out how a commercial publisher could make money giving their content away. Maybe others haven’t thought about that, but I sure did. So, the short answer is, Lumen gives the content away, but charges a very modest amount, $25, for the intelligent backend. All the feedback that goes both to the students, and the instructors. Today, you personally, either of you, could go and get a copy of the Lumen Principles and Micro book, or the Principles of Macro book, and it’s yours forever, you can do with it what you want. But if you want to take the full Waymaker course, they charge $25. The idea is, that amount of money is both affordable to students, but also enough to maintain revisions and corrections, and keep the servers running and all of those things. So that’s the answer to that question. And I will say that every semester, I try to be completely transparent, and say, “If you don’t want to pay the $25, you can get all the content for free. But here’s what you lose.” In five years, I’ve never had a student who didn’t pay the $25, because they thought it was like beer money for the weekend, or something. Compared to spending 300 bucks on a traditional text that was nothing to them.

John: What are some of the barriers that you see to faculty adopting OER? You mentioned that people may have this perception of lower quality…

Steve: …Yes .

John: …but there’s quite a bit of evidence that the quality is not weaker in any way. And I think you had done some studies on that a while back, didn’t you?

STEVE. Yes. The number one problem I think is misinformation. The majority of faculty today don’t know what’s available in their discipline. Many of my colleagues have told me, “Yeah, OER sounds like a great idea, but there’s nothing available in my field.” Now, that’s flat out wrong. For your listeners, there is OER available for nearly every Gen-Ed course taught today. So that’s number one, is lack of knowledge of what’s available. Number two is, as you mentioned before, the belief that OER is inferior, that there’s no peer review. And that’s just not true. There’s a couple things here. One is that OER publishers don’t have a sales force, and so it’s going to take longer to get the word out. There’s been a lot of progress over the last few years. But at my school, we’re only in the second-year of our formal OER initiative. So we’ll see how it goes. The other thing that I think gets in the way of adoption of OER is path dependence, and the unwillingness of many faculty to change their textbooks because of the fixed costs involved. “I’m going to have to go through my lecture notes and make sure that I’m using all the same terms as the textbook does,” and that sort of thing. I don’t know the answer to that question. I know that some schools have used financial incentives, fairly modest financial incentives, to get faculty to try to make the switch. As far as my own assessment goes, every summer, I do statistical analysis of the effectiveness of the texts that I’m using. I looked at both the OpenStax Principles book, and also most recently, the Waymaker package. What I’ve looked at is, textbook alone, textbook with ancillary website, digital courseware, and because I used to teach a writing intensive version of the principles courses, I also looked at writing intensive. And what I found is pretty predictable, at least from somebody who has done this for a while. What I found is that there is no significant difference between student learning using OER, with commercial textbooks. I found that using either courseware or an ancillary website improves student learning outcomes, regardless of what the text is that you’re using. And I’ve also found that writing intensive courses seem to work better than non-writing intensive courses, because the students are getting into it in more detail. Over the last two years, I’ve been doing a randomized control trial, where I can really drill down and see what’s going on. And what I found is that using the full Waymaker package seems to have a statistically significant positive impact on student learning. So I’m going to rerun the analysis using last semester data, which I haven’t had a chance to get yet, but I’m anxious to see how that goes too. I believe this stuff works. And so I think sooner or later, more and more publishers—the commercial publishers too—are going to move towards digital courseware type products.

John: I think most of them have started to at least.

Steve: Yes, but it’s like turning the Titanic. Their base is so large that it’s going to take a while before even all of those people get on-board with this.

John: One thing I was wondering is whether you see more collaboration or competition in OER textbooks?

Steve: Initially, there was more collaboration in the early years. And the reason why is because anybody who was doing OER, was increasing the interest in users for everybody’s OER. Now, I think we’re going to see more competition between the users. Especially as more publishers are going to adaptive and personalized learning type courseware. I think that’s a way that publishers are going to be able to say, “Well, yeah, we’re doing that. But we’re doing better in our own particular way.” So I think there’s going to be a fair amount of product differentiation. And it will be harder for faculty, it’s going to take more work to dig in and see exactly what’s going on. I would love to see more published assessment of efficacy on the part of the commercial publishers. They’re only now starting to do that, and the studies that they publish are heavily controlled by them. So it’s not clear that they’re telling us about all of their things, just the ones that work. But at least it’s a start.

John: One of the things I see in most of those studies is comment to the effect that, “Students who use our adaptive learning platform have letter grades on average, one letter grade higher or point eight points higher.”

Steve: Yes, that’s right.

John: And there’s no evidence that they’ve done any control for the students who chose to use it versus those who didn’t.

Steve: That’s right.

John: But it would be nice if we could see more research on that.

Steve: And I think we will. At least I’m hopeful.

John: Earlier you told us a little bit about how your course is structured with some “just-in-time teaching,” and some activities there where you have students work on problems. Could you tell us a little bit more about how you structure your course so that it’s not duplicating the textbook?

Steve: The first thing that I would say is that, my intro course looks like almost anyone else’s Principles of Micro or Macro course. If you look at the course outline, it has all the normal topics in it. A very slight difference is, instead of assigning students chapters to read and problem sets to do, students have modules with content and learning activities to complete. There is some difference between my face-to-face sections and my online sections, because I teach both. My face-to-face sections are pretty much the way I described them to you earlier. My general approach is to do Socratic lecturing with a lot of in-class activities, like the supply and demand problems that I mentioned. I also like to have formal in-class discussions on interesting questions that don’t have a right answer. In the macro class, I spend a day talking about what is money. And I spend the day talking about what is government. And those are things that aren’t done in the same way and the same degree with a textbook, whether it’s Waymaker or something else. My online course is roughly similar. But what I do is I add group and individual activities to the online course to mimic what I do in class. I also have a weekly Google Hangout, a synchronous Google Hangout, where I can give students guidance about what I think they should be doing. And I can give little mini lectures on things that I know students have trouble with. But it also gives them a chance to ask me individual questions in a real time basis, one on one. Not a lot of students come to those Hangouts. I usually have between five and ten, and my classes are about 35. But more than 90% of the students watch the recordings. Google Hangouts are automatically recorded and archived in YouTube. So the students seem to like that a lot.

John: You mentioned that a number of people at Mary Washington have switched over, what proportion, would you say, of the faculty at Mary Washington has moved to using OER?

Steve: Single digits, a handful, probably less than ten at this point. But this semester, I have two new people. So I’m excited about that. And we haven’t yet given them any money or anything to do this. I’ve just been talking to people. I was invited to the College of Business’s summer retreat, and I gave a little talk about OER. And I got two people who expressed an interest in following up. One of whom has already done it. So I think we’re getting there. We just have to be patient.

Rebecca: So we normally wrap up by asking, well, what’s next?

Steve: What’s next for me is I’m continuing to iterate to improve Waymaker. I’m going to continue doing my own statistical analysis. So I get access to the aggregate analysis that Lumen does, but I also have my own analysis. So I can tailor that to my particular students. I also want to do something this semester that I’ve wanted to do for a long time, but have never done it. And that is to write a new non-traditional chapter for the micro book, which is relatively easy to do. It’s just really a question of me sitting down and doing it. So I know it’s doable, but I do want to actually make my version of Waymaker different from the standard version. In part, because it’ll better match the way I teach. But also because I want to see that it’s relatively easy to do so that I can talk about that to faculty.

John: Very good.

Steve: I’m going to the CTREE conference this summer to talk about Waymaker. And this is the first time we’ve actually reached out to a disciplinary conference. So I think that’ll be fun.

John: You know, I always want to go to the CTREE conference, but I teach at Duke in the summer and it runs right into that. So I haven’t been able to go. And we should note that the CTREE conference is a Conference on Teaching and Research and Economic Education.

Steve: I love to talk about this stuff, because I believe it.

Rebecca: Yeah, it was really interesting.

John: Thank you.

Steve: Oh, you’re very welcome. Thanks for the opportunity.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kelly Knight, Kim Fischer, and Jacob Alverson.

[MUSIC]

65. Retrieval Practice

Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, Dr. Michelle Miller joins us to discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Her academic background is in cognitive psychology and her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications.

Show Notes

  • Miller, M. (2014). Minds Online: Teaching Effectively With Technology. Cambridge, MA: Harvard University Press.
  • Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968.
  • Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 1199327.
  • Kahoot
  • Bray, Niki (2018). 43 to 0: How One University Instructor Eliminated Failure Using Gamified Learning. Blog post
  • Retrievalpractice.org
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Pennebaker, J. W., Gosling, S. D., & Ferrell, J. D. (2013). Daily online testing in large classes: Boosting college performance while reducing achievement gaps. PloS one, 8(11), e79774.
  • Pauk, W. (1984). The new SQ4R.
  • Thomas, E. L., & Robinson, H. A. (1972). Improving reading in every class. (a discussion of PQ4R)

Transcript

John: Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, we discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today we’re welcoming back Dr. Michelle Miller. Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications. Welcome back. Michelle.

Michelle: Thank you so much. It’s so great to be here today.

Rebecca: We’re so happy to have you again. Today’s teas are:

Michelle: Well, I’m drinking Coco Loco, which is a blend from a local tea shop here in Flagstaff, Arizona. Steep Leaf Tea. And Coco Loco is a lot like what it sounds like. It’s chocolate and banana. So tea snobs may scoff at my choice, but it’s wonderful.

John: IIt sounds good.

Rebecca: And I think I saw a nice silver teapot that was poured into a green and blue tea mug.

Michelle: Yup.

Rebecca: A nice tall one.

Michelle: That’s what I need.

John: And I’m drinking ginger, peach green tea.

Rebecca: I went with a Christmas tea today. So Michelle, we invited you here today to talk a little bit about retrieval practice. Can you first start with defining that for us and letting us know what it is?

Michelle: Right. So retrieval practice is essentially the act of pulling something out of memory. So that is, in memory research, what we would term retrieval. So something is stored in memory and we want to pull it out so we can actively use that information, have it in our conscious minds and so forth. And so we go through this usually very fast process called retrieval. So retrieval practice is specifically the act of doing it, and we contextualize that with learning. So, when I’m trying to learn something or I’m the process of learning something, to say, “Oh, what was that fact I remembered, or what can I say about this?” When we do that, it produces something that we can call the testing effect. So this is kind of the clearest example… not the only… but the clearest example of retrieval practice in action during learning is when we sit down to take a quiz, take a test or something like that. So all the excitement that’s happened around retrieval practice in higher education, and really in the rest of education today, is around this finding, which has been replicated many, many times: that tests are, as one person put it, not neutral events in learning. When we take a test on something, that has a very powerful effect on our ability to remember it in the future. So, really simplified down to its core, tests help us remember in the future; when we take a test it strengthens our memory. So that’s what retrieval practice is and it can, as we’ll maybe talk about today, take many, many forms in learning settings. And I did want to clarify too, this is something that I definitely don’t want to take any kind of credit for discovering this. This has been around and has been known about for a long, long time. Some of the big names who are associated with this: Jeffrey Karpicke, Robert Bjork. Roddy Roediger, there are quite a few really heavy hitting cognitive scientists and cognitive psychologists who have established this. But there are many, many of us who are out there trying to disseminate this to other teachers around the world so that we can all tap into the power of this. And I have done a little bit of work in this area with my colleague here at Northern Arizona University, Laurie Dixon, who’s another psychologist… and we teamed up some time ago to look at a very practical implementation of retrieval practice in an Introduction to Psychology course that we conducted some years ago and this is a course, you can imagine, where just trying to get students to perform even a little bit better is a big project. So we examined how even something kind of basic… it was very high tech at the time… but just basic web quizzes that came packaged with the textbook. We said, “Well, if we actually assigned students to do these as part of the course, and if they went through and treated these as opportunities to learn, not just assessments in and of themselves, would that have any systematic impact on course performance?” And we found that in fact, there was a significant improvement associated with that. So that’s kind of the landscape of what retrieval practice is, and why we’ve been so interested in discussing this in the psychology of teaching and learning.

John: In fact, I saw you present on that about 11 or 12 years ago in Orlando at one of the NCAT conferences and it convinced me to completely revise how I was giving my classes and it’s made a big difference and resulted in some significant improvements in student learning. Given that we know so much about retrieval practice. Why are faculty resistant to doing this?

Michelle: Wow, that’s a great question, and that is one that I have been really facing a lot these days in my own practice, talking to other faculty members, and disseminating this through some different activities I do in this area. It is easy for those of us who work in this area, cognitive psychologists in particular, but a lot of us who like you heard about this a long time ago. We’ve seen the power of it. We forget that to other faculty, this can be a very off-putting concept. And so it’s really great for us to think about why that is. And I always think that’s really good for me to kind of go back to that and say “Yeah, not everybody is sold on this idea. And there are good reasons for that.” So like with a lot of things that we talk about in teaching and learning, I think that these really break down into two neat categories: there’s the philosophical issues that people have with it, and then there’s more practical and logistical issues with it. So kind of tackling those one at a time. Philosophically, when people say, “Yeah, I understand about the research but this just goes against something that I believe as a teacher or how I want my classes to be. Here are some ways that that can play out. First off is this idea, that I’ve heard in one form or another quite a few times, and that is this notion of superficial learning. So “Okay, sure, there’s a study that showed that maybe people retain something a little bit better. But surely that’s not this deep learning, whatever that is.” That’s a concept that we all want. So do tests and exams… just testing… create just a superficial form of learning? And while, of course, I understand that, and I absolutely applaud faculty for really thinking deeply about that issue, and caring about it. I Here’s the thing… we got to define that. We social scientists, that’s what we do… we have to kind of break things down and say, “Okay, what does deep learning mean?” and I don’t know that anybody has kind of definitively done that. But when I look at that, I say, “Well, this is not just one research study that showed a little improvement in a lab test. There’s quite a few studies that do use realistic types of materials. It’s not all just contrived laboratory studies. Furthermore, there’s also studies that show that when students engage in more quizzing and testing on material that they actually are able to transfer that learning better. And that is a very, very big deal in teaching and learning as a lot of us know is not just getting students to be able to solve a problem in one context or work a concept in one context, but can they do it in the next circumstance? And that very difficult process is aided by quizzing… and to me, what could be deeper learning than learning that transfers? So that’s part of it. Some of it is perceptions around multiple choice quizzes and tests. There’s an assumption too that if we’re talking about quizzing we must be talking about multiple choice questions… and first off, sometimes in larger classes, those are the final assessments… in that Introduction to Psychology course that we studied years back, that’s what the assessments were… so, having students practice in that format I don’t think that we should necessarily dismiss that. And as we can talk about in a little bit, there’s lots of ways to induce retrieval practice that actually don’t involve multiple choice questions. So, there’s a bit of that as well. And something that I’ve talked about with some faculty recently, too, is this baggage around K through 12. And maybe that’s something that’s resonant with you all.

John: Yeah, that’s given the testing effect a somewhat bad name, because high-stakes testing is being used in a lot of what’s going on with K to 12. But that I don’t think is what retrieval practice as you’re suggesting is all about.

Michelle: Right, and I have to be cautious here. I really like how you laid that issue out in K through 12, that there is a reputation problem… and that has happened because of the high-stakes standardized testing policy in the United States. And I got to be careful because I don’t want to represent myself as an expert in K through 12 or in K through 12 education policy. But I don’t think you have to be an expert in that to know that there’s been a lot of same pretty well justified public pushback against over-testing in K through 12. And yeah, I think that we absolutely do have to be aware of that. Students come to us in higher education… that’s a system that many of them have been through and our faculty are very aware and very cognizant of that too. So, nobody’s a blank slate here, not our students, not our fellow faculty. We have assumptions and ideas and experiences about testing that happen. I think those can be addressed. But, yeah, that is definitely another very big barrier. We got to differentiate between high-stakes standardized testing for the reasons it’s done in K through 12, and low-stakes testing and quizzing for learning as proponents of retrieval practice would have it.

Rebecca: Some of the pushback I’ve heard from faculty fall into two categories related to this as well. One is that they assume that retrieval practice is best implemented in 100, 200 level introductory classes instead of upper level 300, 400, graduate-level classes. And then the other area is that paper and pencil tests don’t make sense and all disciplines. And so they assume that a test has to be in a paper/pencil format, which could be online testing, or it could be multiple choice or it could be essay questions. But I think that, from being someone in the arts, like there’s other ways to test beyond that, but we don’t think of those as tests.

Michelle: Right. That’s another great lens through which to look at this issue; that we do need to broaden the definition to draw more attention to this and to make it a more appealing concept. But yes, how can we make it broadly appeal across lots of disparate disciplines? Not only does it not have to be a multiple choice type of exam, maybe it’s not a pencil and paper exam at all. And we as faculty have to think about what makes sense there. You make a good point about the levels concept. I think, these days most of us have heard of, “Well, there’s one particular Bloom’s taxonomy…” which is a wonderful framework for getting us thinking about being systematic about what we’re asking students to do with the information that we’re bringing to a course and trying to do things like align the teaching we do with the assessments that we have. That’s wonderful. However, I think it does ingrain in us that idea that “Well, just knowing things is sort of at the bottom… You sort of get that out of the way, and then we go on to the good stuff.” And from a cognitive perspective, these relationships are much more fluid and much more interdependent, so that yes, absolutely, the higher thinking that is what we want. That is what we should want. Or if we’re in highly applied disciplines (if we’re in the arts, for example), we need students to be able to do things with that information. But they have to have that. So I think it challenges us to think of new ways with that concept as well.

John: One of the barriers I think some people have is they don’t like to grade tests and so forth. But one of the things you mentioned in your book is that the testing effects been known for a long time, but it was really difficult to implement in terms of low-stakes testing, particularly when you’re teaching at a larger scale. But, as you’ve done yourself, and as you suggest in your book, computer technology makes it easy to automate some of this… certainly more easily for multiple choice and free response and similar things. But it makes it a whole lot easier for both students to have multiple attempts at learning something using some type of mastery quizzing and it makes it a whole lot easier for faculty who don’t have to spend all their time grading.

Michelle: Right and that’s absolutely where the practical stuff comes in. So, we’ve worked through some the philosophical objections, that: “No, this does not turn your classroom into some terrible assembly-line concept of learning. It’s not going to create a bad relationship with your students. It’s not going to simply carry on a legacy of bad policy as people perceive it. It’s not going to do those things.” And then, we do get to: “Alright, but what is this going to do to my life as a faculty member?” …and this is important stuff. Those who know me know I’m a big fan of James Lang’s work in his books. One of his more recent books is called Small Teaching and there’s a lot of different takes on it in that book, but it really hits home with respect to “We do have to think about not everybody’s in a position to, nor should we even try sometimes, to just take everything down to the foundation and rebuild.” That we do really need to think about “Do I want to do this, if it’s going to create 800 more questions for me to grade?” Is this a sort of a situation where, “Well, I don’t want multiple choice, so I’m going to have to give these open-ended questions. I’m gonna have to give feedback and I have 200 students, what will happen? If I am going to go with multiple choice questions… well, how am I going to do this? to have to write all of this?” And yes, it’s one of the most powerful outcomes of the educational technology revolution that makes it workable, and scalable in a sense, even with large classes to do these types of things and to bring them in. So, that is definitely a message that I hope faculty think about if they’re on the borderline of wanting to bring in more retrieval practice into their classes.

Rebecca: I’m in a discipline where the multiple-choice questions are using things digitally doesn’t always work for testing and practicing some of these basic things, and there’s not a good way to automatically grade it. But one of the strategies that I’ve used is actually some self grading, which has actually worked pretty well. I just check and I have them write notes about anything that they got wrong. So, it demonstrates that they’ve tried to understand when we go over it, and I give credit based on how thorough those notes are, rather than whether or not they got the question right or wrong. And that’s made a difference in my classes. And had I not come up with that solution, I think I would have abandoned it because it would have been too much work. But it it actually is working pretty well.

John: When we had a reading group on Small Teaching last year, one of the things that was widely adopted by faculty was a very simple form of retrieval practice where they had students at the start of each class reflect back on what they had done in the previous class. And most of them have continued to do that in subsequent classes as well.

Rebecca: One of the other barriers that faculty might raise is the idea that students don’t take low-stakes things seriously, or that they don’t put the same kind of time into it that they might for something that’s high stakes. Can you talk a little bit about how we might help students find value in retrieval practice and subsequently also with the faculty then?

Michelle: Right, so that, ”Well, we can put it out there, but will they do it?” I’ve kind of crossed this philosophical and practical barrier for myself of giving some credit for pretty much anything that I am hoping that students will do. I put the work in to set it up and I do believe as a teacher that there’s reason to believe that will help their performance, that I need to work it into the syllabus somewhere. I don’t think it detracts from learning, necessarily, to say, “Yeah, there’s some points associated with this.” And especially with our students who, for many of us, are going in a million directions at once. They’re juggling jobs, multiple classes, sometimes their own families. So having an incentive in the form of points—having some kind of a payoff—I think, helps them make that decision that this is at least worth the time to do. I think the other thing that we probably can all do more of, and that I’ve done more over the years, is framing and honestly marketing this to students… communicating with them about why. And when students disengage from an activity like this, when they say, “Ah, why do I have to sit down and do this thing? This is just another test. Oh, no.” …really conveying the excitement and the goodwill that we have in setting those things up can go a very, very long way. Of course, a student, if they just look at it inside and they have no context for why this was put into place, they’re going to have them say, “Well, maybe I won’t do that.” But when we can market to students, we can say “There is a lot of research that shows that this is a very, very good use of your time. And hey, you’ve probably taken a lot of tests in your life that were really about measuring or sorting you and figuring out what you know and what you could do. This is a very different kind of test.” So that can go along way and get the C students nodding: “Alright, I get it. I get why she put this assessment right here.” I think a lot of us have hit on the practical strategy too, that the little Easter eggs or goodies that we plant in the form of questions that get re-used on the higher stakes assessment. So most of us will have tests for measurement at some point in our courses. And yes, students really do pick up on it when you use one, two or more of those items that were in, say, the gamified quiz that you ran in class or the reading quiz that they did beforehand. They can see those and say, “Oh, wow, I got feedback on that. I got an opportunity to practice…” and if it draws in a few more students who see it as almost a legitimate form of cheating, honestly…. like a fun and sanctioned form of getting an advance sneak peek at the exam, then great! Then that maybe is an opportunity for them to come in and see that. So there is that. Actually taking it seriously ourselves, not just in the form of saying, “Well, here’s the points I’m going to give you for this,” but spending class time on it. That’s a big bridge for a lot of us to cross, right? Because we as teachers tend to be very focused on “Oh my gosh, class is for covering material. We use all these sort of distance metaphors to talk about what we want to do with our class time. But if I say, “You know what, guys, I believe in this, and I believe in it enough to where we’re going to spend the entire class period before the final exam….” I did that twice last week myself, when running exams. Or we’re going to spend five or 10 minutes of the beginning of every class period doing this, as one project recently published about doing. If you show yourself doing that, and offer them that, I think that also goes a long way towards it. And I guess to just say, well, taking it seriously… here again, what does that look like? What does that mean to different people? And we can kind of a little tongue-in-cheek say, “Well, why do we have to take it so seriously?” Sometimes games and learning can happen when it is presented in a more fun context. So not everything has to be deadly serious or spending hours and hours and hours of stressful time on. There are occasions when a light-hearted approach can be perfectly good and can still get us involved in that really critical activity of retrieval

Rebecca: I can share an example of doing that in my classes. We’ve done design challenges and sometimes we challenge other classes that are happening at the same time. That reinforces some of the basic principles that we think that students should be doing and reminds them of it… and they might work in a team…and then we have a competition. And it’s fun and what have you. And students like those, it breaks up the day, it makes it more fun. And then I’ve also done things where I give class time to do little design challenges in class that might be individual and then they can level up to working with a partner to finish solving a problem or something… and students value that. They recognize that next time they’re trying to do a project on their own,that it’s easier because they’ve had that practice or that opportunity for the retrieval practice. And my students have actually ended up asking for more of those opportunities.

Michelle: Great.

John: Could you go back just a little bit and tell us what you did in those couple of classes last week before your final. We’re recording this, by the way, in early December during finals periods in both of our campuses, but we’ll be releasing it a few weeks later… to put that in context.

Michelle: Oh, okay. Well, I’d love to. …and I’ve been talking about retrieval practice as you pointed out for years, and I still discover new ways to infuse this into courses. And the context for this is my cognitive psychology undergraduate course. It’s a 200 level. So it’s a lower-division course. And it’s about 60 or 70 students. And as you can imagine, it is a bit of a tough sell. For many of the students it’s their first encounter with this side of psychology. It’s not as intuitive as some other areas of psychology. So there’s a lot to learn and a lot of motivation to be done. One of the ways we bring this and in this course is using a technology called Kahoot, that’s spelled K-A-H-O-O-T,, and it’s really very intuitive and functions very smoothly… relatively free of bugs. That’s good stuff. It’s a program for doing gamified quizzes of various kinds. What I did, and at different points in the semester, and then really amped up in the last week of the semester is running these gamified quizzes. And this is something, by the way, that I hit on and got the idea to try based on a colleague named Niki Bray, who’s from Tennessee, and has actually done some really systematic work in reformulating some of her courses around in class quizzing in just really ingenious way. So I saw some of her presentation and I said I’ve got to try this for myself. I went with multiple choice questions. Kahoot does have some parameters… questions do have to be short…very, very short. And to some people that may be off-putting, but you can put together quite a few of these. And so we would put this up on the projector and students have the option of dialing with either their laptop or their smartphone and weighing in on each question. The neat thing about it is it has an algorithm for giving points based on your speed as well as your accuracy and it’s got a little leaderboard so you can actually have a little in-class competition. Now some people who use this do require all students to do it and they actually issue points for performance. Now, I presented it very much as a practice activity… and made it very, very clear because of my philosophy, I’m not going to assume that all students have devices or have smartphones or laptops or that they want to do that. But I said, “Look, remember we’re talking about retrieval practice, guys. So the real meat and potatoes of this is not buzzing it on your phone. That’s fun. But the real benefit of this activity is what you’re doing sitting there in your seat. And you could be doing this with a piece of paper if you want.” And that is what a few students opt to do. They try to answer the questions, they know what they need to go back and review and so on. And it’s nice because it spits out at the end, a whole report that tells me right away… Okay, which questions do we need to revisit? Which ones did students have the hardest time with and so on. That’s one of the things that I just brought in. And yeah, it was a big deal. I sacrifice a chapter of “coverage” so that we would have more time at the end of the semester. But to me, I would rather have students going into the final knowing that they’ve had this retrieval practice and they have a better chance of performing really well. Earning a good grade on this material I care about than honestly cramming in a little bit more mileage in terms of the quantity.

Rebecca: Sounds like fun.

Michelle: You know what, it does really bring a fun factor. There’s been a lot of different variations on in-class polling, and I will admit to this, I actually purchased and am the proud owner of a physical buzz-in quiz device, complete with a whole spaghetti nest of wires and an incredibly abrasive, buzzer sound, and everything. So previous to this, my educational technology did include… I think I could have up to eight intrepid volunteers who would play a quiz game and then I would have to appoint a points keeper and all this… and props are always a lot of fun. So when I say I believe in bringing in retrieval practice, I really do walk that walk. But I will say doing it by a smartphone does allow for more participation, and I don’t have to worry as much about the minutiae of scorekeeping and stuff like that.

John: I played with Kahoot a little bit at my classes at Duke and students have loved it.

Rebecca: We’ve talked a little bit about framing things so that students take the practice seriously. What do we do about the students who just push back, it’s like “This is too much work. This is a lot of extra time…” or that sort of argument.

Michelle: I think that that’s another piece of that barrier to more faculty adopting this, not just the work involved, but realistically, student opinions. Student evaluations matter a lot to faculty life. And of course, we all want to have that wonderfully rewarding semester, not the semester where we feel like we’re at odds with our students. So I do think a little piece of this is we do anticipate sometimes worse and more pushback than what actually happens in the end. So I think we have a fair amount of sort of a dread factor when we go into something new like this. But that said, when students have an issue with more quizzing or more testing, here’s how those come out. I think, first of all, we do have to sometimes separate out the technology aspects of it from the quizzing or testing, per se. So, as John mentioned, this is one of the amazing things that educational technology does. But the flip side of that is, if you’ve ever used technology for education, in any shape or form, you know that it breaks down. And when it breaks down at a big class, that’s a headache for everybody, it’s a misery. So if you’re trying it, and things are not working out, you just got to figure out “Okay, how much of this is the assignment, the activity per se, and how much of it is that wonky quizzing thing that I got from the publisher and it fell apart, or students hanging up when they’re trying to dial in with the poll and fine tuning those. I mean, when I first used kahoot, I decided it would be a lovely idea to put four chapters worth of material into a 40-question quiz. And when you get used to as you know, that they took a long time, I mean, 12 questions can keep you going for a very long time, especially if you’re discussing… and some students got kicked out part way through and then they weren’t on the leaderboard and I could have set the whole thing aside. But really, that was more I needed to get the technology working. So that’s a big piece of it. I think that sometimes it is a perception issue with the timing. Now, those of us who are just all over this as a teaching technique, we like to do reading quizzes before we talk about that material in class. So chapter three is up on the syllabus and your chapter three reading quizzes due on Sunday night before we do that…, and that can provoke a fair amount of confusion and honestly griping with students. They say, “Why did I get tested on this when we didn’t do it… we didn’t cover it in class yet?” And what do you know, that’s another framing and communications issue. Once we know that that is why we’re doing that you get so much less on that side of things. But here too we do also ourselves have to follow along with what we say. If we say you’re doing this reading quiz so that we’re establishing a foundation. we don’t have to teach everything in the class itself. We could spend the time applying. Well, guess what, then you do have to do that. So if I assign you to do the reading quiz on chapter three, and then on that Monday morning I go, “Okay, we’re going to go over chapter three starting at the top. And I’m going to show you all the PowerPoints for all these things that you already read.” Well, yes, then student morale and student support will fall apart at that point.

Rebecca: Those are very good reminders.

John: Yeah, I’ve pretty much adopted this approach all through my class beginning when I first saw you present it. So I have students do the reading in advance, I have them take reading quizzes with repeated attempts allowed on those. And then in class, I have them working on clicker questions, and I and the TAs go around and help them when they get stuck on problems. But there is an adjustment and students, especially when I first started doing it, would generally say, “But you’re not teaching me” …and you do have to sell students on this a bit. One source of resistance is that when students take quizzes, they often get negative feedback. When they read something and they read it over again, it looks more and more familiar. They’ve got that whole fluency illusion thing going and they become comfortable with it, and they feel that they’re learning it… or similarly, if they hear someone give a really clear presentation on a topic, it feels comfortable. They feel like they’re learning it until they get some type of summative assessment where they get negative feedback, and then they feel the test was somehow tricky. But it’s a bit harder to convince them that actually working through retrieval practice, watching videos at their own time and pace, reading material as needed, and then spending class time working through those problems is as effective. I’m getting better at it. But it’s been a long time trying to convince students and I did take a bit of a hit in my evaluations, especially the first few times.

Rebecca: Do you mean learning’s not easy?

John: Students would like learning to be easy.

Michelle: You know, it’s funny, I think almost in a way… see what you think about this idea. But it’s almost a mirror image of our illusions as teachers, right? That I gave a wonderful clear lecture and I assigned wonderful readings and I saw students highlighting them. Therefore, students must have assimilated this knowledge and it must be in there. So I think it challenges our students but it also challenges us as well. It can be quite an eye-opening experience to running something like a Kahoot. And that brilliant point that I gave this great example for…. what do you know…. 7% of the students actually nailed it. And so we can all use a reality check… teachers and the students. And you mentioned re-reading, that’s another one where I’ve had some pretty intensive conversations with other faculty, I’ll kind of say, “Oh, well, and there’s this great research that shows that students tend to re-read when they’re studying and we know that from a memory standpoint, that is really, really ineffective.” And faculty will say, “Whoa, whoa, whoa.I want my students to be re-reading.” Of course, now when they say re-reading, they may be picturing deeply interrogating a text… annotating it… looking at it from a different perspective. And absolutely, that’s a wonderful part of scholarship. But that isn’t what we’re talking about. We’re talking about students re-reading as a study form and mistaking highlighting for deep interrogation. So, just like with the rest of knowledge bases versus higher-order skills. This is another words “both and” it cannot be “either or.” Yes, we want students re-reading in the right ways, but students or teachers cannot mistake that for learning sometimes.

Rebecca: So, let’s say you’ve just convinced all of our listeners that we need to be doing this, how do we bake it into our course designs?

Michelle: Well, I think that, really getting creative with this, and as I talked to faculty when I visit other schools or talk to faculty of my own institution, I just see all of these new ideas all the time about how to do this. Once you do get that critical epiphany of alright, a test doesn’t have to look like a test on the surface… it does not have to be the ritual of “Okay, you’ve got a number 2 pencil that’s breaking and I’m standing over you while you have a panic attack for an hour.” That’s not what it is—that it’s about the retrieval. Once you get that, all kinds of creativity opens up. So I’ve also started sometimes on the very first day with the syllabus quiz. So, especially if you have a smaller class, we all struggle with that “I spent all week writing the syllabus and it’s incredibly important, and then I’m getting questions about stuff that’s on it all the way through the semester.” So, really on the first day, what I do is I take a very light-hearted approach and I divide the students up into teams, just physical teams. Everybody’s got a copy of the syllabus, sometimes I fake them out and I say, “Okay, we’re going to go over the syllabus point by point…” and I say, “No, you can read the syllabus. We’re actually going to do this other thing which research shows will actually help you remember it.” So the task here is that each team formulates a set of questions… you can make just a few. like three… so a little bit of teamwork. So formulate three good questions off the syllabus. And you’d be amazed, they come up with questions that even I can’t answer sometimes without having to cheat. So they talk about it. And then of course, you go around the room in some arrangement and each team gets to ask the other team their questions, and if you really want you could keep score and everybody loves bragging rights for being the team that stumped the other teams and won the most points. So, it can literally start right then. With that idea that I don’t read stuff to you in this class, this is about you. And I’m also not piling a huge amount of work on myself either apart from being the moderator and the MC and having written the thing in the first place. I don’t have to write questions. They’re writing the questions and they’re answering them too. So those are some of the ways that we can do that. I think especially in our larger classes we do want to think about things like peer grading or peer review of open-ended question responses. We do want to take advantage of things like publishers’ test banks to set those up as reading quizzes. And you’d mentioned earlier about “Well, what about this not being suitable for upper-division classes?” I had an upper-division class that just wrap this semester where we had reading quizzes as well. It may have been an upper-division course but it also serve that purpose of “Hey, you take a basic quiz over the chapter on Sunday and that really sets the stage for us to have a more substantive discussion.” All of those things. are ways that we can do this. I think open-ended reflection as well… so tests that look nothing like a test but are still retrieval practice. You’d mentioned about reflecting on what you learned last class period. So this sometimes goes by the name “brain dump.” And I did a bit of this last semester as well. So, this by the way, I do want to credit the great website retrievalpractice.org. So retrieval practice is actually that high profile it has its own website. It’s an absolute treasure trove of ideas. So, with the brain dump the way that I did it, or similar to what it sounds like you did, every now and again we start off class with you writing down everything you remember from last time on a piece of paper. I had students then turn to their neighbor and compare notes and see what they came up with. And I didn’t grade those. This was not a heap of grading for me. But in the end what they did turn into me for accountability and a few points was a very short reflection, just on an index card. So, “What surprised you?” and I review those and they’re very eye-opening but again I’m not there to police or micromanage what they put on their cards. So, that’s retrieval practice too. There’s lots of different flavors that we can bring in.

John: I do something similar with asking students to reflect on the reading in each module we work through. But because I teach a fairly large class, I didn’t want to have to deal with all the index cards. So I just have them fill up a simple Google form. And then I can skim through it and assign grades much more easily than shuffling paper. But it’s the same basic idea and it’s worked quite well.

Michelle: Absolutely.

Rebecca: Do you have any other examples of the kinds of little tests that faculty can run that don’t look like tests?

Michelle: Right? So tests that don’t look like tests. Well, here’s another that was probably a little bit more practical for a small class, and this is how I ran this. It, like the brain-dump exercise the way I ran it, was also very cyclical and very student generated. So we started out each class at the beginning of each week with students generating a set of quiz questions based out of the assigned reading. So we didn’t actually, in this particular example, have pre-quizzes or something like that. But students came in knowing that they can bring their book if they wanted. But they would need to sit down and write for me three questions, whether short-answer or multiple choice. Then I can flip through those and I would really quickly put those together for a quiz that went out the subsequent day. So, we’re alternating between generating questions and then returning to those questions. And then I would have pass out. I told them treat this like a realistic test. Actually try to retrieve everything, but, you know, when times up, you’re going to get to go back to it. And then they would grade it themselves. So I didn’t actually do the grading either. And that is really great for spurring discussion. And “Oh, my gosh, I thought I knew the difference between reliability and validity, but now that I tried to answer it, I realized that I didn’t.” And then you can throw it back out to the student who now has bragging rights for having had their question selected and say, “Hey, what did you mean? What was the right answer? And why did you put that down there?” And at the end of the day, they kept that quiz too. So it was really very much in their own hands to do. So there’s that. Other creative ideas that I’ve run across over the last couple of semesters… There’s a great project out there, run by Bruce Kirchhoff at University of North Carolina at Greensboro. So I got the wonderful pleasure of talking to that group last year. And he and some colleagues working in the area of botany actually put together a freestanding, custom built mobile app that students could take with them that presented different kinds of quizzes over the sorts of things that botany students really need to know like the back of their hands like how to identify different plants and how to discriminate among different examples and they found some empirical evidence that this actually raised performance up quite a bit. I’ve heard too of another Professor put together some surveys in Qualtrics, the surveying software took advantage of its ability to actually text message people and send them the questions. So this was an opt in sort of activity and it’s one that they didn’t just have it run 24-7 because it was a little intrusive, but what it did is it sent students questions that they could answer at different intervals throughout the day, which also takes advantage of another principle from applied memory research which is spacing. So students are getting these unpredictable questions, they have the option to answer them, and they could be happening even when they’re not in class. So those are some other ways. Some of them look like tests, some of them don’t. But those are creative ways to get students engaged in that practice.

Rebecca: As a faculty member, we often advise students and mentor students who might be struggling in other classes, ones that we don’t have control over. Are there ways that we can help those students use this methodology to do well in those classes where it might not be embedded?

Michelle: Right. And that’s so great that you bring that in as well because ultimately that is what we need as teachers and that maybe circles back to yet one other piece of objection that I’ve seen… actually this time in a published article from a few years back that said, well are we doing too much for students? Are we scaffolding them too much so they’re going to grow really dependent on these kinds of aids like reading quizzes, and reading questions. But if we also have in our mind very intentionally, that what we want students to have at the end of the day is also something they can walk away with, I think that we do have to be very mindful of like, “Okay, let’s not create the impression that just because retrieval practice is so important that you have to sit there and wait for me to put together this specific kind of reading quiz for you.” So I think here, the really powerful message is once again that one students take this to heart… once they’ve not just been told this, that “Oh well, you should quiz yourself as a study strategy,” but they’ve seen that I believe in it to the point where I’m going to put time, energy, and work into it as part of my class. And maybe they’ve even seen the results… they’ve now had their own little before and after experience of what happens when I do this… that they can be more likely to take this forward. So having more faculty across the curriculum endorse this is a powerful idea that “Hey, this is how your mind works. This is how your brain works. Your brain doesn’t just soak up stuff that’s in front of it. You soak up stuff that you have to answer questions about.” That I think is going to be a powerful message. And there’s actually another article out there that I just absolutely love it was done by a psychologist named Pennebaker and some colleagues at University of Texas at Austin some time ago. They replaced high-stakes assessments in their Introduction to Psychology course with these mobile in-class quizzes that were done every day. And one of the things that they report in this article is not just that students did better in that class, but that certain subgroups of students actually showed improvement, specifically closing of achievement gaps, really, in classes that they took after the psychology course. And these were classes that we’re not even in psychology. So you think about that for a minute. How does that happen? Now, nobody knows exactly for sure, because this was kind of an unexpected finding in the study was my impression, but a real possibility is that when students have sat in this class and every single day they have seen the power of taking a quiz… of spacing out their learning… and attacking it in a very active learning approach… Once they’ve seen that happen, they’re more likely to go home and say, “You know what? I can do this in my biology course too. I don’t have to sit there with the teacher’s quiz. But, if I attack it in the same way I might get the same results.” So I think with those things, we can have students walking away with that enduring practice that we want them to have. And it is funny too, because it brings up one of the, I’ll just say it was a really heartwarming teacher moment that I had this last semester when I did bring in these Kahoot quizzes. In the run up to the first exam, I had done my fancy little in-class quiz and was kind of patting myself on the back of what a great leader I was. So I came in to, I think it was the class right before this exam, and I come plowing into the classroom and it’s dead quiet. And there is a student at the front of the classroom. He has commandeered the podium and the computer and what has he done? He’s accessed the Kahoots, which I gave them the links. You know, they’re out there for all the students to see. And he is running them and the rest of the students in the class are taking them just as seriously as when I was administering them. So they’re running through the questions again, giving it another shot. I didn’t tell them to do this at all. It was one of these moments where I just backed out and closed the door after me. And I came back in when they were finished. And so I think those are the kind of moments that we can set ourselves up for when we really do bring this into our classes and we get behind it and a really authentic way.

Rebecca: I think one thing Michelle, in the way that I asked the question, I had also asked it from the perspective of a student hadn’t had the experience of doing a retrieval practice in a class. So, if you’re working with a student, maybe outside of class or as an advisor, are there things that we could do to help those students adopt those practices, even if they haven’t seen it modeled for them?

Michelle: So how do you help students when they haven’t had some of these experiences on their own? And I think this is a part of a bigger package of goals that I think a lot of us should have in supporting our students to really put students in the driver’s seat. To say, “Yeah, you don’t have to wait for a certain style of teaching or certain subject material in order to succeed with it.” I see that isn’t really part of a larger package of growth mindset honestly. So what can students do to make themselves the masters of this? Now, I think that there are some old standard, very traditional, approaches that are worthwhile. And when you look at those approaches, sometimes retrieval practice is at the core. So it may not be a matter of trying to get a very unfamiliar set of terminology or anything for students before, but really getting them to look at some of these approaches in a new way. So things like you’ve ever heard the term SSQ4R or PQ4R, there’s a couple of those that have acronyms and the things they have in common are that they tell students when you sit down to study or read a text or prepare for a test, here’s what you do. You don’t just start reading from the top with no goal in mind other than “Oh, I want to get a good grade later.” What you do is first of all, you set yourself up with questions. Which is, after all, a lot like retrieval practice. Start with a question, say, “What do I want to answer? Can I answer that now?” And if not, why not? To read your text or go through your material very intentionally around this questions, and then there’s always this piece of recite or review. That’s what one of those Rs stand for in some of those traditional systems. And that means closing the book. That means closing the book and saying, “Okay, I’ve sat here with this material for this amount of time. What can I actually say about it at this point?” So, sometimes directing students back into some of those and saying, you need to adopt these strategies, which are completely teacher-independent, they’re fairly discipline-independent as well. That can be good. If you’re doing them for the right reasons and with the right approach in mind, that’s very good. Encouraging students to take advantage of the publishers’ companion sites… Now this is a little bit more of an uphill climb. That is where we run into “Well, if there’s no points for it, Why should I do it?” But encouraging students to say “Look, you already paid for this textbook. To be crass about it, you paid all this money, did you know there’s a website over here and if you interact with those materials in this particular way that’s really, really likely to pay off for you?” So, besides just ensuring that our students have what I consider to be these basic foundational pieces of knowledge about the mind and brain: that we remember through testing, we remember through retrieval and an active engagement. All students should have that, but those are some specific things that we can counsel students to use across all their studies that really should pay off.

John: Do you have any other suggestions for those faculty that are thinking about expanding their use of retrieval practice?

Michelle: You know, just to really encourage and support faculty who are starting this journey, as it sounds like that you all have, to really re-examine how we can bring in this incredibly powerful principle, and to really reassure each other that “Yeah, this is not about really just piling so much more work on yourself.” We can sometimes even just re-examine the assessments and assignments that are already in the course and so that’s kind of one last piece of practice that I think that a lot of us can really stand to bring in. And that was a big thing for me as well to say, “Well, if I’m going to administer these tests… we administer tests, we administer midterms anyway… what else can we do to increase their value as learning experiences… as learning events. So here’s another where I’ve brought in various forms of test discussion activities. Instead of standing in front of the class with that deadly the day-after-the-test class period where I say, “Let’s go over it” to realize, you know what? That probably will not work that well. But one of the amazing things that we have learned from the research on retrieval practice is that it’s not just in the moment in the taking of the test, where this advantage to memory happens. It actually creates a sort of a receptive window for learning and for review when we’ve just taken a test on something. So if you’ve got a midterm in your class already, well, hey, why not carve out the time after that test is up to say give it back to the students… like I photocopy the exams before I’ve graded them. They have no feedback on them whatsoever. And I hand them back to students as a group discussion exercise. I say “Alright, here’s a blank copy of the exam, your group can fill out this blank copy together, just knowing what you know, revisiting all those questions, having those good discussions about what you understood and what you didn’t. And I offer a little bit of extra credit for really good performance on that. And there’s other ways to work that out so it actually takes off as a small group exercise in class. But regardless of the specifics of how you make something like that work, the spirit of it is the same. That when you sat down to do this, to try to drag all this information out of memory, that is not an end unto itself. It should be part of a bigger picture of learning in the class. And it’s sort of an untapped vein of potential we have as teachers and that our students have as well and that we can access it regardless of what the discipline is or how the class is setup.

John: …and students see it’s very relevant. I started doing something very similar in my econometrics class last year following a suggestion from Doug McKee who had been doing that in his class. And it worked remarkably well… and turned what was normally a pretty unproductive class period where we’d go over the test… and the people who did well with just be really happy and pretty much ignore any discussion, and the people who did badly were just sitting there unhappy and not really being very receptive. But when they sit there, and they’re explaining it to each other, it seemed like a really ripe time for them to learn the material much more deeply.

Rebecca: I remember the first time that you implemented that. You sent me a text message with a photograph of his students taking a test and it looked very active. [LAUGHTER}

John: …and they were having fun.

Rebecca: Yeah.

John: One of the nice things that came out as I was wandering around listening to their conversations, and I was hearing people say, “Oh, yeah, now I see where that came from.” Or someone would say, “Well, when did we learn this?” And then someone else would say, “Well, remember when we worked on these problems?” …and it just helped them make connections… and the power of peer instruction is so remarkable. I’m going to do it in as many of my classes I can.

Michelle: That’s perfect. That’s what we want as teachers, and that’s what our students benefit from.

Rebecca: So as you know, we always wrap up our podcast by asking what’s next?

Michelle: Oh, wow. So I am looking forward to kind of rebooting a course that I have not taught in some time. My senior capstone course in technology, mind, and brain. And that is a fun one. But it’s one that is going to take a lot of revision, since it is technology and things change so rapidly, as we all know. So that is going to be a big part of next semester. I also have a crop of research projects in various angles on teaching, learning and educational technology that I’m really excited to be moving forward in the next calendar year. One of those… really foremost among them is a project on virtual reality for learning. We have an incredibly creative and dynamic team looking at virtual reality here at Northern Arizona University. They put together an amazing series of interactive exercises that are part of the organic chemistry course here and teach some of the challenging concepts in that extraordinarily challenging gateway course. And so we now have a whole set of data from students who went through and did this at varying points during the semester. We got their feedback, we’ve got all different kinds of psychometric measures that we gathered from them at the time as well. So, I cannot wait to be tackling that and looking at all kinds of angles on how this part of technology is impacting student learning in this course.

Rebecca: Sounds like some exciting adventures.

John: That sounds like a wonderful research project. …looking forward to seeing what you find.

Rebecca: It was a real pleasure to talk to you again, Michelle. Thanks for spending some time with us.

John: It’s wonderful talking to you. Thank you.

Michelle: Oh, likewise, always a pleasure to talk to your listeners.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

64. How Humans Learn

Small children are innately curious about the world around them. This curiosity, though, is often stifled in traditional educational pathways. Dr. Josh Eyler joins us in this episode to discuss how research on how humans learn can help us build a more productive learning environment for all our students. Josh is the Director of the Center for Teaching Excellence and an adjunct Associate Professor of Humanities at Rice University.

Show Notes

  • Eyler, J. R. (2018). How Humans Learn: The Science and Stories behind Effective College Teaching. West Virginia University Press.
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Peters, R. A. (1978). Effects of anxiety, curiosity, and perceived instructor threat on student verbal behavior in the college classroom. Journal of Educational Psychology, 70(3), 388.
  • Bandura, A. (1965). Influence of models’ reinforcement contingencies on the acquisition of imitative responses. Journal of personality and social psychology, 1(6), 589.
  • Bandura, A., Ross, D., & Ross, S. A. (1961). Transmission of aggression through imitation of aggressive models. The Journal of Abnormal and Social Psychology, 63(3), 575.
  • Bandura, A., Ross, D., & Ross, S. A. (1963). Vicarious reinforcement and imitative learning. The Journal of Abnormal and Social Psychology, 67(6), 601.
  • Bandura, A. (1969). Social-learning theory of identificatory processes. Handbook of socialization theory and research, 213, 262.
  • Vygotsky, L. (1987). Zone of proximal development. Mind in society: The development of higher psychological processes, 5291, 157.
  • Bjork, R.A. (1994). “Institutional Impediments to Effective Training”. Learning, remembering, believing: Enhancing human performance.
  • Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
  • Reacting to the Past – This site describes the Reacting to the Past methodology.
  • Yue, C. L., Bjork, E. L., & Bjork, R. A. (2013). Reducing verbal redundancy in multimedia learning: An undesired desirable difficulty?. Journal of Educational Psychology, 105(2), 266.
  • Gopnik, A., Meltzoff, A. N., & Kuhl, P. K. (1999). The scientist in the crib: Minds, brains, and how children learn. William Morrow & Co.
  • Eagleman, D., & Brandt, A. (2017). The Runaway Species: How human creativity remakes the world. Catapult.

Transcript

John: Small children are innately curious about the world around them. This curiosity, though, is often stifled in traditional educational pathways. In this episode, we examine how research on how humans learn can help us build a more productive learning environment for all our students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Josh Eyler, the Director of the Center for Teaching Excellence and an adjunct Associate Professor of Humanities at Rice University. Welcome, Josh.

Josh: Thanks for having me.

Rebecca: We’re really glad to have you. Today’s teas are:

Josh: [LAUGHTER] I’m not drinking tea. I’m afraid I’m more of a coffee person. But I have water today.

Rebecca: We still have to stay hydrated. So waters good.

John: I’m drinking bing cherry tea.

Rebecca: I’m drinking Christmas tea mixed with Prince of Wales tea because I just kind of re-upped with a different tea bag.

Rebecca: On campus, we have a reading group that we have in the fall. So a lot of our faculty have read Minds Online, Make it Stick and Small Teaching. And I was hoping you could address how your book is a bit different in approach to those. I know that you know those authors and have worked with some of them. So can you talk a little bit about how How Humans Learn, which is your new book that just came out is a bit different?

Josh: Sure. Well, I think I’ll start with a similarity. And I think that the one similarity that connects all of them is that we all try to weave in practical suggestions for the classroom as well, taking the research and heading in that direction with it. I do think the one of the things that separates my book from the others is that I look at science very broadly. So the great thing about Make it Stick is that it engages cognitive psychology, the testing effect, desirable difficulties, ways to remember and to encode things in memory more deeply. And I think that’s very important and fascinating. But my book goes in a different direction to look at the evolutionary history of learning, developmental psychology, other biological views on learning. So it takes a different track on science and I think with Small Teaching and Minds Online, they also touch on cognitive science as well, which I do too. But I’m very interested in placing our students into a much larger conversation about the development of learning over time.

John: And you mentioned that your daughter played a role in influencing your decision to write this and investigate this. Could you talk a little bit about that?

Josh: I did. Sure. So right around the time that I began moving into the world of teaching and learning centers, and was doing a lot of research and wondering about why certain teaching practices work, and others don’t. I also became a dad and got to experience the wonder of seeing my daughter explore the world for the first time, a baby trying to figure out and explore. …this curiosity in the flesh… and everything is driven by curiosity. And it just really started to make me wonder about these fundamental ways of experiencing the world and learning about the world. And really, two big questions emerged. Number one, it was pretty clear that some of these were consistent over time, this natural curiosity about the world… we continue to have that and continue to learn from it. The other thing was this what happens to some of these learning strategies that are so prominent when we’re at our youngest ages and It’s clear are so important for learning about the world. How do those shift? Where do they go? How can we utilize them and tap back into them as college instructors. So she was a really important part of this book.

Rebecca: It’s funny that the first thing that I commented to John, after having read some of your book was “Man, this is so fascinating,” because I also have a small child. [LAUGHTER]

John: …and I had two. It was a few decades ago, but I remember seeing the things that you talk about there.

Josh: Right.

John: You mentioned though, this evolutionary perspective, and you use the term Evo Devo to capture that which I think you made a reference to a bad alt-rock band. Can you tell us a little bit about what Evo Devo represents?

Josh: Sure. Evo Devo isn’t my term; it’s one that I, to my great delight, discovered as I was looking at the research on this. And so as a preface to this, in the humanities and exploring fields that I never engaged in before. And so it took me five years to write this book, because I was teaching myself how to read these papers, the methods, etc. It was really important to me that what I said would be credible with scientists. They didn’t have to believe me or agree with me, but they had to find it credible. And so part of the research then led me into this, I don’t want to say is brand new, because it’s been around for a few decades, but in terms of scientific sub disciplines, is pretty new, evolutionary developmental biology. And the practitioners of this call it Evo Devo, for short. And it just struck me as the name of a bad band that you might have heard of in the 90s. But at the core of that research is the study of how developmental processes evolved over time. And so if we think about young children again, why do they do what they do? What evolutionary advantage did it have? What kind of adaptation? Which parts of the brain develop first? And why is that important for understanding cognition? …things like that. So they tackle big questions both in humans and in other animal species. But yeah, it was the name that really kind of jumped out at me.

Rebecca: One of the things I really love about your book is the interdisciplinary nature of it. And that you are tackling all these scientific principles, but really putting it into plain language that faculty from any discipline (and other people who are not faculty) can easily understand. So I immediately got sucked into the language that you were using, because I understand it, I could put it into practice. And so I think other faculty will really enjoy that as well.

Josh: Oh, thank you. I appreciate that. I really was trying hard to take what I saw as really important concepts and make them accessible to all.

John: So you take a number of broad themes and you investigate that and talk about ways in which faculty can use these principles to help improve learning, and the first one you start with is curiosity, which is certainly apparent in small children. Could you talk a little bit about the role of curiosity in learning?

Josh: Definitely, I start with this one for two reasons. The first is that as individual learners, it becomes clear that curiosity is a prime driver for the way we learn new things the way we experience the world. The other reason, though, is that when you begin to look in these various corners of scientific research, it’s also clear that curiosity has been explored in great detail in many different disciplines. And so it’s one of those that I think is a great example of why I wanted to write the book in the first place. Lots of fields are talking about it. And if we just kind of brought those conversations together, we might be able to utilize that information. So curiosity was a fundamental element in the evolution of some of our cognitive processes. It defines us as a species, that we are a curious species, if nothing else; that the way we approach the world and try to figure out the world is driven by needing to know something and seeking out the answer. And then as we see that play out in our own lives, we’ve already talked about children but some of the most famous developmental psychologists (Piaget and others, actually), began a long time ago, studying curiosity by looking at children’s questions like the kinds of questions that they asked. And early on, they were particularly interested in how many questions they asked. So you can see large catalogues of quantitative data on the number of questions kids ask. But more recently, people have really been looking at the kinds of questions and what that might say about different developmental stages. And so what I took from all this, I took several things, but I think the most important thing for college teachers is that the question itself is the unit of curiosity. To what degree can we utilize questions and inquiry to tap back into this natural curiosity that has faded over time as students have learned what they needed to do in educational systems in order to succeed. …and sad to say that curiosity is in some ways a prime casualty of those educational systems… and so, how can we tap back into questions become a really important way for us to do that.

John: In terms of questions, you suggest that when we give students questions to address, that we try to use open-ended questions to allow students perhaps more direction in that. Could you talk about that just a little bit?

Josh: Sure. And open ended questions that can really fascinate them and engage them as well… some of which might come from them. But the distinction I made primarily is between open-ended versus closed-ended questions in designing discussions and closed-ended questions… questions to which there is one distinct answer can shut down discussions pretty quickly. They’re okay for warming up discussions. I think they do have some value. If we really want to get to a place where we’re using questions and discussion to help students collectively generate knowledge, they have to be questions that are open enough to prioritize multiple interpretations or multiple approaches, and in some cases, they can be questions to which we don’t know the answer… that we’re all just sort of exploring possibilities together. And so one of the things I recommend both in the book and in my work with faculty here at Rice, look into questions that you generally ask or have designed for a discussion and see if there are ways to take some of those close-ended questions and push them into more open-ended questions. Not “What is the name of this thing?” but “Why is this thing so important for our understanding?” …approaches like that.

John: You also discussed in this chapter, the trade off between novelty and anxiety. Could you tell us a little bit about that?

Josh: Sure. And there’s some researchers who have built their career looking at this curve, the novelty-anxiety curve. Some novelty, because of our curiosity, is very important. But when we get the curve up to the point where it becomes so new and unfamiliar that it kicks in some anxiety, the learning process begins to shut down… so you start to move back down on the other side of that curve. So anxiety has been shown in a variety of learning settings to shut down the cognitive processes. It’s at an emotional level and the psychological level that if we’re anxious or fearful, we can’t really focus on the subject at hand. It’s adding a cognitive load that just becomes insurmountable in terms of learning. There’s the study that I cite in that chapter from 1978. I still think it’s really fascinating. I’d love to see people try to replicate it. A researcher named Ruth Peters was looking at evidence of curiosity in classes where students had rated their instructors based on how intimidating they found those instructors and even those students who had rated themselves as being relatively high on the curiosity scale, asked fewer questions and engaged in fewer incidence of curiosity in those situations where they were intimidated or felt that the instructor was intimidating. Now an important caveat that I don’t, I think, do as well as I could have done in the book in making is that the issue of intimidation and how students feel about faculty members can differ greatly depending on who the students are and who the instructors are, and how that dynamic plays out could be very gendered. And there are lots of dynamics, I think, that complicate the notion of intimidation that I didn’t dive into that chapter. But they’re very important, I think, to think about. On the whole, though, the point remains, and there’s a lot of consensus on it I would say, that anxiety shuts down curiosity.

John: And one of the points you used to emphasize that is a recommendation to not be scary.

Josh: Right.

John: …which is not a bad suggestion for faculty. Because I think sometimes we can be and making sure that students see us as more open to them could be useful.

Rebecca: …or even as human.

John: …even as humans… [LAUGHTER] … that we could appear human to them. They don’t always see it that way. But it would be nice to encourage that more personal connection.

Rebecca: One of the other topics that you tackle as a theme in your book is tied to failure and risk taking, and I often think about curiosity and failure and risk taking together as someone from the arts. So I often see curiosity feeds risk taking. And if you don’t have failures, you don’t continue to learn and get excited about what you’re doing.

Josh: Right.

Rebecca: But I also know that students often (and all of us often), if we fail, can feel really bad about that and then not continue to propel forward. Can you talk a little bit about the relationship between curiosity and…

John: …the fear of failure.

Rebecca: Yeah.

Josh: Yes, definitely. And I completely agree with you that there’s a link there between those two, because curiosity often takes us into places where we don’t know the answers. This is how great research happens, right? We want to know the answer to a question. So we pursue it but there’s a lot of failure along the way. And so it has the potential to open great doors, but those kinds of intellectual risks that follow curiosity, we haven’t necessarily developed educational systems that value taking those intellectual risks. In fact, as researchers and scholars and artists know, there’s so much trial and error along the way, but we’ve set up the opposite system for our students where they have these high stakes assessments, they get one shot. And that is the kind of environment that will cultivate a fear of any kind of failure or mistake. And so the linkage then, in order to try and help them take those intellectual risks, is to build opportunities in our courses for students to be able to utilize that natural learning process: to make mistakes, to learn from them to get feedback, and to move on. So there’s a variety of ways that we can do it, but it runs counter to traditional modes of teaching and course design.

Rebecca: If we think about being in an institutional structure that imposes certain kinds of systems on us like grades that can cause anxiety, and maybe shut some of this down, what can we do within that system to still foster risk taking?

Josh: Right. That’s in some ways the hardest question to answer and it’s the hardest thing to wrestle with if you’re suggesting to faculty that we can, within our own courses, create opportunities for students to do that. So I have two answers. And the first is that one thing that we can do that will pay students dividends down the line is to help them divest learning from grades. This is the absolute foundation for our work in this area. And one way to do that is to have more assignments with lower stakes. So if you have five assignments each worth 20% of the final grade, breaking that up a little bit more and including smaller assignments in there. The other is to have assignments where you give feedback, but no grades. And this takes some conditioning over time. But eventually students can begin to see learning as an opportunity for feedback, not evaluation. The other thing that I would say that I recommend in the book… The people who are experimenting with alternative grading models, I think, are leading the charge on this question. And so things like contract grading or specifications grading, portfolio grading. There are even some well known folks are experimenting with student self grading, self assessment, peer assessment. And so those are models that shift the value and the meaning of what is a grade. And those systems a grade is not necessarily an evaluation, but it is a culmination of feedback, a culmination of learning over time. The emphasis becomes, in those systems, much more on feedback and development and on evaluation of selected attempts at a task.

John: You also emphasize the roles we have as social creatures in the chapter on sociality and in particular, one of the things you talked about there is imitative learning. Could you talk a little bit about that and how that can be implemented in the classroom?

Josh: Sure, yes. The book began with curiosity because that’s individually how I think we’re driven to learn but sociality, I think it’s one of the most important topics that we can think about with education. We are deeply social beings at heart. And in fact, humans are among the most social creatures in the world. And so that greatly influenced how we developed over time and how we’ve learned. And so a lot of the evolutionary biologists who study, for example, language point back to our imitative gestures. They either preceded language or they co evolved with language. There’s a lot of disagreement about that. But there are fundamental aspects of the way we communicate with people and ultimately teach and learn from each other. And so, the extension of that to our work as teachers? It can be as basic as the gestures that we make and how we communicate the importance of a particular topics. I’m making gestures right now… maybe you can’t see… but that underscores… to hit the emphasis to underscore the importance or to communicate to the students in ways other than verbally, and I was blown away by some of the fascinating research on gestures to prove how important they are to learning. There was one study, for example, of foreign language learning where students were watching videos of a speaker speaking and fully making the normal gestures that you would have in teaching. And then other students were watching videos with no gestures, the speaker was taught to eliminate all gestures as much as possible. And on the assessments afterwards, the students in the gesture condition scored far better than those in the non gesture. So that’s a fundamental level, but there is a social psychologist in the mid 20th century, Albert Bandura, who talks about modeling. And so imitative learning can really connect in an essential way to our modeling for students. They imitate us, even sometimes our manners of speech, but really what I’m most interested in is the way we engage with them. They will often use that as a model for engaging with each other… the way we engage with our subject matter… relative enthusiasm or lack thereof,… they will use us as models for that. And so imitative learning, I’m not going to say that necessarily enhances the content to a large degree, but it does influence the learning processes that are underneath the mastery of the content.

Rebecca: How does sociality relate to group learning and group work that we might do in our classes? So it’s peer instruction, but also other collaborative work that we might do.

Josh: This is a topic that I think is really important to many who teach in college and so it’s one that I wanted to spend some time covering and if we are going to utilize our social natures to maximize learning, we have to design collaborative assignments that, out of necessity, students must work together in order to generate the knowledge in order to fulfill the goals of that assignment. Too often, and I am guilty of this myself, group assignments are designed in such a way that students can “divide and conquer.” And so they say “I’ll take this part, you take this part and you take that part and we’ll come together right before the presentation and we’ll debrief each other.” Certainly they can learn some things individually from that if they’re well designed; that is not making use of our social natures to enhance learning, that is just creating an avenue where they’re talking to each other and making a plan for presentation. The assignments where they have to actually work together to develop the new knowledge, to construct it, those are the kinds of assignments that are taking advantage of our social natures to really enhance it.

Rebecca: Why is it then that students, even in a situation where it might be that they need to collaborate to come up with that new knowledge or information, there’s still this tendency to try to figure out how to divide and conquer even if it doesn’t match up? [LAUGHTER}

Josh: Right.

Rebecca: How do we help students understand the differences in those different scenarios or even how we help faculty understand the differences that happens in committee work and stuff as well?

Josh: Right. Yes, it does. Well, I think some of that is that they’re simply trying to employ strategies that have worked for them in the past. And so even if the assignment isn’t designed for divide and conquer, they’ll try it because that’s what they’ve done in the past. There’s also some reticence often to working together initially. So faculty need to communicate the value of the nature of the assignment as a group assignment: What will you gain from working with each other? I do think there are some examples here in my home campus, and many campuses across the country, have faculty who do a lot of work up front on the team and group dynamics in order to make those assignments and activities more productive. And so bring experts on team dynamics into the classroom to talk to groups and to maximize the way they’ll work together… assessment systems where people are evaluating themselves and each other… warm-up activities or shorter, very small activities over time to help students in the group learn to trust each other before we give them the big assignment… trying to clear some of those social hurdles in order to really use those social elements for their gain.

Rebecca: I think that’s an important piece there… that we put them in groups, or they even put themselves into groups and we just assume that like, “Oh, now they’re a happy little group.” [LAUGHTER] But unless they get to know each other and develop that trust that you just emphasize, really, things can fall flat really quickly.

Josh: Right.

John: But there’s other types of assignments where you can do that such as you mentioned in your book: the Reacting to the Past methodology and many forms or peer instruction where that sort of collaborative work is inherent in the process. When they’re not presenting something they can divide, but when they all have to debate something or present something from different perspectives, it naturally brings them together. So I think you do provide some suggestions on ways to do it, but it doesn’t work with all projects. If they’re writing a paper or presentation, you have to cultivate that. And it may not always work.

Rebecca: …especially if it’s something long term. There’s a difference between peer instruction and class that might happen over a short moment or two versus something that might take weeks.

Josh: Yes, definitely.

John: You also have a chapter on emotion. Could you talk a little bit about how emotion influences learning?

Josh: Yes, in some ways, this is the heart of the book from the start… thinking about our interactions with students at this level. For a long time, psychologists believed that emotion and cognition were entirely separate. And then they thought that they were connected, but one was dominant. But now biologists, psychologists, neuroscientists, they have really shown us that emotion and thinking are completely interlinked. I described them as dance partners in the book and really, that’s what they are. When everything’s going well, the two are working hand in hand, the primed emotions enhance the learning and the learning imbues the emotion with different levels. So, when everything’s working really well, they go hand in hand. And that can look at the span of what we’re talking about, the spectrum of utilizing emotion in our teaching is very broad. And so it can be everything from being enthusiastic about what we’re teaching to using humor, helping students see the joy of the subject matter; it can be that. But it can be also finding the emotion at the heart of the content that we’re teaching. So I’ve used an example a lot of two biology classes both teaching about cancer; one is teaching it entirely about the cellular level and another’s doing that but also showing videos of survivors and talking about the disease at the human level. That kind of scenario primes the students’ emotions which helps them in turn remember that material more effectively… develop better conceptual understanding: “Oh, here’s the long term resonance of this disease and there’s the human impact of this.” And so finding the emotional aspect of our content… And then simply, I think if I had to get one message out about that chapter, and maybe even about the book. is students will learn more if they think we care about them as learners. And that doesn’t mean that we have to tear down professional boundaries that might be important, but it does mean that students have to see us as being actively involved with their learning, as actually caring about their success in the classroom. And if that were the only thing that we did, we’d be making a lot of headway.

John: You mentioned just learning their names could be effective in helping to show that.

Josh: Yes, in fact, often I say that probably the easiest thing we could do to initially show students that we’re invested in them. And often people say rightly that teaching huge classes, it’s really hard to remember names, things like that. …an easy suggestion about that (and I got this from my friend Bethany Usher, George Mason University), hand everyone a table tent… a name card… at the beginning of the semester… have them write their names on it. So even if you can’t memorize all 200 names in your class, you can refer to people by name, which accomplishes in many ways, the same purpose.

Rebecca: Sneaky… It sounds very sneaky. [LAUGHTER]

John: It’s not a bad idea. Actually, when I teach small classes at Duke in the summer, I do that for the first couple of days until I get to know their names. In the class of three to four hundred, it may be a little tougher, at least to get them to come back and do it. I can imagine my students swapping the names just to mess with me a bit. Mybw… I’m not so sure. [LAUGHTER]

Josh: Bethany uses it… and others do for attendance purposes as well. So everyone has to put their name card back on the table when they leave and the date on the inside that they were there. So it’s a way to take attendance too.

John: You also talk about the importance of authenticity, of creating assignments and tasks that are authentic for the students. Could you talk a little bit about that?

Josh: Yeah, and authentic and authenticity have a wide range of meanings in higher ed these days. But what I was particularly looking at is the research on what folks have called cognitive authenticity or situated cognition, and that’s the degree to which our brains pick up on a learning environment as either being artificial or real – authentic. So the degree to which we can help students to engage in work that mirrors the work of scholars in the discipline or has relevance to their lives or application to their careers or application to the world writ large, we’re moving closer to authenticity and the kinds of learning environments that the brain responds to really well. It’s a ruthlessly pragmatic organ that will quickly turn this attention elsewhere if it doesn’t think what is happening is necessary for it. So I talked a lot about undergraduate research or projects or even shorter projects that we do in the classroom where students are doing what actual historians or sociologists or artists or biologists really do. The difference between memorizing the names of a hundreds insects and going out in the field and combining that information with finding them. I think that’s an authentic activity, an authentic learning environment.

John: In your book you Bjork and Bjork’s “desirable difficulties,” which suggest that students learn the most when they’re faced with this feasible challenge… where if something’s really easy, they get bored; If something’s too difficult, you have anxiety and things that interfere with learning.

Josh: Right.

John: One of the problems that people have in implementing that, is the range of backgrounds and skills of students. Can you suggest any ways of trying to reach that zone for all students when students come in with very different prior knowledge and skills.

Josh: Yes, that’s a really great question. It get really happy when I talk about Vygotsky’s zone of proximal development… maybe others don’t, but I think it’s fascinating and deeply important and it ties to desirable difficulties, I think. Simply because what Vygotsky was saying was that each of us for… pick a topic… but for any type of each of us has a zone in which we can learn as individuals, but eventually we will hit a point… the end of that when we need what he called a “knowledgeable other” to help us develop mastery beyond that… and so that’s going to differ depending on topic and depending on individual… and as faculty, if we were in our best position… if we know our students well enough to understand where each of them are within those zones… The “desirable difficulty” by the Bjork’s… things like spaced practice or interleaving… which means don’t study in blocks… study a little bit of topic “A” a little bit of topic “B,” a little bit of topic “C” et cetera… then go back. It’s really hard, but in it’s being hard, the brain encodes ot more effectively… and that matches up well with the zone of proximal development because it is a way for us to design activities where: number one we can see a little bit better where students individually are; number two. though. it helps push past those ending points or the sticking points that students can encounter when they hit the wall with a particular topic.

Rebecca: After writing this book, and spending five years deep into all this material, what have you found most useful as a teacher?

Josh: This has been such an amazing process for me. I’ve learned so much. The chapter on failure, though, has been the most eye opening for me. I’ve certainly redesigned discussions based on curiosity. And I tried to build more stories into my teaching to draw on those emotional social connections. But partly because we’re not taught in graduate school to privilege failure and errors and mistakes… and it’s not necessarily a fundamental element of any pedagogical training that we get even as faculty… I found that research to be extremely important for me as a teacher. And so it has changed the way I designed courses, I have more assignments and more opportunities to just give them feedback in order to manage the load that that brings with it. I have more face-to-face conversations, quicker sessions to give targeted feedback. I have activities like reading responses where all they have to do is complete the set number that I have for them. If you do 12 reading responses over the course of the semester you get the “A” for this activity… really to engage them in the thinking process in ways that will contribute to their work, but without the necessity: “I have to get it right.” I can just think and explore some ideas. In my undergraduate courses. I’ve switched entirely to portfolio grading for all of them. It does take time I have found to help students see that this isn’t a trick or you’re not trying to pull the carpet out from underneath them. But it has really transformed the way I work with students. More of them come see me in office hours because there are no grades. So they just want to talk about the feedback that I’m giving and ways to revise and improve. And I would say that the grad courses that I teach… we team-teach those courses on pedagogy… and those are all contract grading. And so we want the emphasis to be on mentorship for them as new teachers and not evaluation. So that chapter has really made the most difference to me as an individual teacher.

Rebecca: Can you elaborate a little bit more on what it means to portfolio grade.

Josh: Sure.

Rebecca: I think people have a general idea of what that means in like a writing class, something that might not have a clear idea of what it might mean in a different kind of course.

Josh: Right. Sure. And actually, I know that there are very specific models of portfolio grading too… some of which come out of writing studies. I take a very broad look at… actually my wife, teachers art as well… and so I borrow some of the notion of a portfolio from what artists do and how they grade student work over time. And so my own definition of portfolio grading is a developmental approach where all assignments are turned in but only given feedback on… multiple opportunities for revision… and the only thing that gets a grade is the final submission of all the revised work. Even quizzes and exams, you can think about in a portfolio model as adapting and revising answers over time within certain constraints, and then giving a grade on the final product that also includes a reflective introduction. How did I learn over time? How did I improve over time? What areas do I still have left to explore?

Rebecca: Thanks. That’s helpful. I have a question about when you’re writing the book, as someone who’s not a scientist and you’re digging into all this science stuff, it seems interesting that you’re writing about learning where you were probably also actively engaging in all of these things. I’m wondering if that writing process influenced and that experience influenced how you wrote this book?

Josh: It did. Yes. I think part of the process was being in uncomfortable territory, novel territory for myself. In many ways. I was in the same situation as an introductory student in a lot of these disciplines and certainly having patience with myself. As I was writing what I was doing my work with faculty and then with students was understanding (because I was going through it myself) a little bit better how someone from a different discipline might be hearing and responding to another disciplinary approach with an unfamiliarity… not a resistance to… but an unfamiliarity that we need to kind of break down and have a common language for. And so that was a part of it. The other thing, though, was being in that position, kind of like a student myself, I reached out to colleagues here at Rice. So when I had questions about evolutionary biology, I would call up my friend Scott Solomon and say: “Here’s what I’m thinking, am I on the right track?” And he would say: “Well, not really. But why don’t you look at this and this…” and eventually, through those interactions with him, I can get to something that I thought that folks would find credible and that process also helps you to see that students, especially those who are new to a subject, there’s the vulnerability and having to say, “Am I right about this? Do I know what I’m talking about.” And I think that’s worth taking very seriously. And I haven’t been in that role for a very long time. And so it was really helpful, I think, to be in that situation again for my own teaching, and to be able to talk about that with faculty. It was fun to write it. One of the reasons I wanted to move into working at a teaching and learning center was the opportunity to work with faculty from all disciplines. And this really helped me in that role, and that I was learning the contours of a lot of different disciplines. And I learned different kinds of questions to ask and different kind of perspective.

Rebecca: Cool.

John: Yeah, we enjoy that for much too.

Rebecca: Yeah.

John: Are there any aspects of your book that some readers might find controversial? The argument that I make in the book that as adults, we still learn in the same ways that we did when we were children. And I wouldn’t say that that probably will be one the biggest points of argument or discussion over time. Certainly not everyone agrees with that. But what I simply mean by that is that our processes for learning really don’t change that much from the time when we’re very little as we grow older. Our brains certainly mature. We have different life experiences that we’re bringing. Our ability to regulate our emotions is not the same as the three year old that’s very, very different.

John: Usually.

Josh: …at least hopefully. [LAUGHTER] But, the way we learn remains relatively the same. And I draw on the work of a fabulous psychologist named Alison Gopnik, a developmental psychologist, and she’s done a lot of papers on this. But she wrote a pretty popular book with two others, Andrew Meltzoff and someone else (his name is escaping me). And that book’s called The Scientist in the Crib. They make the case very clearly. They say something almost to the effect that scientists can develop the knowledge that they can and experiment the way they can, because they’re utilizing processes and structures that were designed for children. So we are still learning in the same way that we did when we were very young. We’re just approaching it a little bit differently and we’ve matured along the way. So I really hope we can use that information in our work in the classroom to say, “There are things about learning that we know will work and have always worked because this is who we are.” And so if we can use that, if we can build pedagogies that tap into those things that we know that never change about learning, then we will always be serving our students well.

Rebecca: I think that’s a good point.

John: It makes sense. We evolved to learn in a certain way. We take things in from our senses, we encode it, and we’re still using the same processes ultimately.

Josh: Yes, I agree.

John: We always end the podcast with the question, “What are you doing next?”

Josh: I recently gave a talk called “Teaching as a Creative Endeavor.” So kind of the other side of the coin… the creativity that goes into teaching… teaching as an art and some of the things that actually can’t be measured, but that we hope we might be achieving. And so I think that I’ll probably continue down that road… a bigger project on the creative art of teaching… what that means… and the research on creativity and how that can apply. I’m not sure if it’ll be a book yet, but I’m definitely really interested in that aspect of teaching as well.

Rebecca: Yeah, that’s something that faculty here have expressed some interest in, as well… of creativity and what that looks like. So I thank you might have an audience

Josh: Well, that would be great.

John: In our past reading groups,that is one of the questions that came up the most: “How can we encourage the development of creativity?” And I don’t think there’s a lot out there on it that we’ve seen at least

Josh: No, I don’t think that there is. I think that there are faculty in the arts who are thinking a lot about it. A number of books, not related to education, but that have come out in the last 10 years on what is creativity. And in fact, one of my colleagues here, Tony Brandt wrote a book with David Eagleman on creativity and the creative brain and humans as the creative species. So I think there’s a lot of room to really use that information to think about our work as teachers.

Rebecca: Excellent. Look forward to finding out what you end up doing with that information and how you explore it.

Josh: Thank you.

John: Well, thank you for joining us. This was a fascinating conversation and I really loved your book.

Josh: Thank you so much. I really appreciate it. Both those kind of comments and also the opportunity to talk with you today.

Rebecca: Yeah, thanks for taking time.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

60. Inclusive Teaching

Are your class conversations dominated by a small number of voices? In this episode, Dr. Danica Savonick joins us to discuss a variety of class activities that support an inclusive learning environment and promote equity in participation while increasing student learning. Danica is an Assistant Professor of Multi-Ethnic Literature at SUNY Cortland, and a recipient of the K. Patricia Cross Future Leaders award, a Woodrow Wilson Fellowship in Women’s Studies, and a Mellon/ACLS Dissertation Completion Fellowship.

Show Notes

Studies of bias in the classroom:

Transcript

John: Are your class conversations dominated by a small number of voices? In this episode, we explore a variety of class activities that support an inclusive learning environment and promote equity in participation while increasing student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Danica Savonick, an Assistant Professor of Multi-Ethnic Literature at SUNY Cortland. Danica is the recipient of the K. Patricia Cross Future Leaders award, a Woodrow Wilson Fellowship in Women’s Studies and a Mellon/ACLS Dissertation Completion Fellowship. Welcome, Danica.

John: Welcome.

Danica: Thank you. Thanks so much for having me. I’m really excited to be here.

John: Our tea’s today are…

Danica: I’m drinking a coconut lime seltzer.

Rebecca: That sounds pretty good.

Danica: It is.

Rebecca: It’s a good alternative to tea, I suppose.

Danica: I think I’m pretending that I’m on a tropical island or something.

Rebecca: Yeah, the weather around here would make me want to do that, so perhaps it’s the same there.

Danica: How far away are we from from each other? I’m here in Cortland, you’re…

John: About an hour and 45 minutes, I think, by car.

Danica: Okay.

Rebecca: Very rainy today.

Danica: Yeah, and I hear we have some snow coming up in the next 24 hours or so, so should be interesting.

Rebecca: I have the Prince of Wales tea.

John: …and I have a holiday tea from Twinings that I picked up in the tropics in Orlando at the Online Learning Consortium a few weeks back.

Danica: Sounds yummy.

John: It is good.

You’ve written quite a bit on creating a supportive environment for discussing issues of race, class, gender and sexuality. Let’s first talk a little bit about the context in which you address these issues. What courses do you normally teach?

Danica: I generally teach American literature courses. Sometimes those are general education courses, sometimes they are within the English major. I’ve also taught a number of writing classes that are a little bit more interdisciplinary in nature, and regardless of whichever course I’m teaching I like to give them a theme or put my own little twist on them. For instance, if I’m teaching a writing course, this semester the topic is the purpose of education and so we’re drawing from a wide different disciplines… people who’ve been writing about different learning methods and then when I teach English courses, some of the topics I like to do are the arts of dissent and we’ll look at the theme of dissent in American literature. This semester I’m currently teaching Intro to Multicultural Literature, which has been super fun and then next semester I’ll be teaching a graduate course on feminist world-making, which I’m really excited about.

Rebecca: Well that sounds really exciting.

John: What are some of the challenges you face in discussing some of these issues in your classroom and trying to have productive conversations?

Danica: Well, some of the problems that I’ve noticed are consistent regardless of what classroom or what school I’ve been teaching in, but some of them vary according to the student population. But one of the most common problems that I see is just a lack of student participation, or if there is participation it’ll be the same two or three students who dominate the conversation… and actually just this weekend when I was home for the holidays I was talking to my family about this—my aunt is auditing a course at SUNY Purchase—and she was saying that the same one or two students speak every single class period and she’s curious about what the other students have to say and what they’re thinking… and even my grandmother who was at Brooklyn College in the 1950s… she said she remembers feeling too scared to talk in most of her classes… and so it was only one or two of the… I guess… the brightest and most vocal students who would talk in the classes. And then, of course, as I started teaching I started to notice this as well and I think it’s every new instructor’s nightmare probably that “What if nobody talks? What am I gonna do? What am I gonna do with all that silence?” And so, I guess the main problems I’ve been trying to address are not having the same one or two students dominate the conversation but having really every voice be heard in the classroom… and the more I’ve thought about it and the more I’ve come to study classroom dynamics the more I’ve realized it’s not entirely the fault of the students in those situations, and actually quite often it is the shared responsibility of both the professor and the students to create a kind of environment where everyone feels like their voice matters and that they have something that they can say… that they won’t get shot down by the professor… that they’re not intimidated by their peers and whatnot. So a lot of my work has been trying to increase participation in classrooms and also because my focus is often on race and class and gender and sexuality in literature, we have to figure out how to have productive conversations around those really difficult issues. And for a lot of students, it’s their first time talking about these issues and so we’ve had to establish ways that we feel comfortable talking about those important questions and issues.

Rebecca: I was actually just gonna follow up to what you were saying… really curious about the emphasis on the first time students have talked about some of these things, and I think that that’s really important. We’ve been having a reading group on our campus with a book called Race Talk and that’s something that we’ve mentioned pretty frequently: that a lot of these students have never been in a context to have a conversation about race… a lot of the faculty have never been in a situation to have a good conversation about race… So, when it’s someone’s first time, how do you help that be productive and feel safe? Because you have to be vulnerable to be in those situations.

Danica: Definitely. One of the most effective things that I think I’ve done is tell students that we’re inevitably going to mess up in these conversations because our educations have not provided us with the language and the grammar and the vocabulary for talking about conditions of structural inequality… and so I make that the baseline or the premise. We know we’re gonna say the wrong thing and we are likely going to accidentally offend someone and so as a class what we do is establish protocols or ways that we want to collectively address how to handle those situations and we come up with a set of community guidelines and principles and ideas that we agree upon for how to behave when we realize that, “Oh no, I could have said that better. I wish I hadn’t said that…” or if one student feels offended by something and so I think that has really helped, especially for students who are having these conversations for the first time; they know that it’s okay to say the wrong thing and we have an established procedure in place for how to deal with those moments.

Rebecca: Can you talk a little bit about how you set those guidelines up and how students participate in that process?

Danica: Yeah, definitely. This is one of my favorite things to do every semester is have students co-author a set of community guidelines in order to foster inclusive discussions of difference. Because I want every student to understand that their voice matters and I know one of the reactions that you can get is students can start feeling alienated if they say the wrong thing. They can disengage. They can start thinking that I don’t have a place in this conversation. …and so one of the ways that we create that environment is we’ll co-author this set of community guidelines. Rather than having students write them from scratch… I think that can be really difficult… so, instead what I’ll do—it takes about I would say half of a class period to maybe half an hour, could be 40 minutes, it depends on the size of the class—I’ll print out some really basic guidelines, four or five things that I think might work well in the class as principles that we might want to agree to abide by. So things like we won’t make assumptions about anyone in the class’s race, gender, ethnicity, things like that, and we read over them as a class. I usually project them at the front of the class and they also have them in front of them on a piece of paper and we’ll read through them as a class. They can ask questions… they can ask me to define a word they don’t understand, and then I give them about ten minutes to read through them quietly on their own with pen and paper and cross off and edit and add and remove anything that they don’t like about the guidelines… to add additional guidelines… to change the wording of certain guidelines… and then rather than calling on students individually and having to put them on the spot, I have them work in pairs of two to go over some of the amendments and edits and adjustments that they would like to make and then after five minutes or so we go around the class and each pair presents one or two amendments that they would like to make and so it really ranges from adding different adjectives and verbs to adding whole new amendments or saying that they didn’t like one of the ones that I put up, which is totally fine with me. The idea behind not having to ask them to do it from scratch is just that they have something to work with—it’s not that I’m wedded to those particular principles, I just wanted to give them some kind of language and some kind of grammar for how they might formulate the different community guidelines.

Rebecca: It seems like the pair scenario would help to mitigate any issues that might arise from a dominant group dominating the rules.

Danica: Yeah.

Rebecca: That was gonna be my question but then I realized as you were talking that that might actually be how you solve some of that issue.

Danica: Yeah, and it’s pretty egalitarian. We go around the room and each group says something, even if it’s by the time we get to the end sometimes the groups are like, “Well, everyone already said what we were gonna say, so we just wanted to agree that we really liked the amendment that this other group made.” And so that way each pair gets two or three minutes to add something, to say something, and then we move on, and so it’s not like one pair gets to really dominate. The other thing I forgot to mention is students go home, they have at least one or two evenings to think about the guidelines that we came up with. They have access to them. They can open them up at home and it’s not until the beginning of the following class that we ratify them, and often when we come back together at the beginning of the next class they’ll have thought of one or two things that they want to adjust and once we make the final edits and adjustments then we as a class decide that we agree to abide by them.

John: Another nice thing about doing it in pairs is when people are speaking it’s a little safer because they’re representing their group; they don’t have to take a stand and it makes it a little more comfortable perhaps for those who might have been reticent.

Danica: Exactly.

Rebecca: I like that you have the ability to review over a couple of days as well because that also gives students who don’t want to speak up the opportunity to email you or communicate with you separately too, right?

Danica: Yeah, definitely.

John: How well have the guidelines worked? Have students responded well? Do you get more buy-in to the guidelines since they created them?

Danica: Yes, absolutely. I was really surprised the first time I did this. I was like, “This is one of those wacky pedagogical experiments; I might fall flat on my face, they might think that I’m an alien from another planet.” But, they were so enthusiastic and I’ve actually had students from former classes say “That was one of the most meaningful things that we did that semester. I think about that a lot. I wish more teachers did that…” and so I’ve gotten really positive feedback on it and it’s also fun. It’s always one of the best conversations that we have throughout the semester. And you know it turns out that they have a lot to say about the issue. Actually, I often do this assignment when we’re teaching a work of literature called Citizen by author Claudia Rankine, which talks a lot about microaggressions… and students have witnessed and they’ve experienced these microaggressions in the classroom and so they’re eager to have a chance to participate in crafting a classroom that isn’t going to have these kinds of uncomfortable and awkward moments. I also should say that when we do this I share with students beforehand several of the studies that have been done recently on classroom participation and who feels most empowered to speak in the classroom. So, there’s been a lot of studies done on gender and the experiences of students of color and what a lot of these studies have found is that those voices that are most empowered to speak in mainstream media and culture are also the students who feel empowered to take up time in the classroom. And so I share this with students before we begin the community guidelines activity and they’re always really interested. I have the sense that some of them have witnessed or experienced or might have some sense that these things go on, but to actually see the research and to see the findings and to see these massive studies that have been done, they’re just interested in it, and especially because my classes are about race and class and gender and structural inequality, I think it’s fascinating for them to see the way that what we often think of as huge systemic issues can come to influence who speaks and who participates in the classroom as well.

Rebecca: Maybe we could share those citations in our show notes?

Danica: Certainly.

Danica: You’ve used something called Commons in a Box. Could you tell us a little bit about what that is for people who are not familiar with that?

Danica: Sure, it’s a free open-source learning and writing platform. It came out of the CUNY Graduate Center. It’s a combination of WordPress and BuddyPress, and so it’s this easy to install package that allows you to create digital learning spaces, and so different universities have taken it up to do different things. Often I’ll see institutions using it as a space for their professors to host course websites. They might want to have some kind of blog that features student writing. They could use it for digital humanities projects… and it’s free and it’s open source and so all you really need is server space. As often as possible, I’ve tried to host my courses on either Commons in a Box, or currently I’m using an installation of wordpress.org as an alternative to using Blackboard or Canvas and I could talk a little bit about why if you’re interested.

John: Yeah, could you tell us a little bit about what the advantages of this is compared to say one of the common course management systems?

Danica: Sure. I see the primary benefit of these platforms as they help students to develop transferable skills that are going to aid them in the world beyond the classroom, and so I’ll talk a little bit about what I mean by that. WordPress is one of the most common platforms on which the websites in the world are built .The latest statistic that I saw was something like 30% of the world’s websites are built on the WordPress content management system, and so I like to organize my courses on WordPress so that I can familiarize students with how websites are put together… how you can build them… how they think… how they organize information… and so what I try to do throughout the semester is scaffold students’ interaction with the platform. At the beginning it’s pretty user friendly: they create an account, they are able to log in to our site and then gradually they start going into the backend (which WordPress calls the dashboard) and they start creating their own content. So, they get to experience the process of going back and forth between the backend and then the front-end and seeing what that process is like and how information is organized on the WordPress platform. So, they start creating blogs and then what I like to do towards the end of the semester is deconstruct our class website and take it apart and break it and redesign it with students so that they can see, first of all, how easy it is to build a website. A lot of my students are new to this. They’re not necessarily computer science majors. They haven’t taken computer science courses, and so they’ve interacted with a lot of websites but they haven’t really gone in and thought about how they might build their own and so I show them how our course website is built and we redesign it we do all kinds of things and then often for students’ final projects they will have the option of designing a website related to something that we have done in the course and they often choose that option. They like it… they like getting to experiment with WordPress. For most of them it’s their first taste of the platform and several of them have said that they’ve gone on to learn more about WordPress because they’ve become really interested in it and I see this as a really great opportunity for students first to think a little bit more critically about how the internet works and how these pages that we’re constantly interacting with… how they’re constructed… and also to develop a transferable skill that could become a really valuable part of their resume and the skills that they will bring to the work world. Being able to build websites on WordPress is huge and so I find that starting that process early can be really helpful. It also creates an opportunity for us to have conversations like why is our course built on WordPress when all of your other courses are on Blackboard and we get to talk a little bit about what Blackboard is and the different ways that content management systems, especially in higher education, work to structure certain kinds of relationships of teaching and learning.

John: Does the institution host WordPress, or are you hosting your own instance of it?

Danica: Ideally, the university will host it. When I was at CUNY they have a really strong culture around open educational resources and free writing platforms and there’s a big community around that. It might exist at my new institution—I have to do a little bit more work to find it. As far as I know there’s a lot of people that are using Blackboard at my institution. Ideally… best-case scenario… the university would provide server space and then you could have an installation of WordPress or Commons in a Box, but currently I’m using Reclaim Hosting and Domain of One’s Own in order to have a classroom commons installation that I’m using across the three different classes I’m teaching.

John: Do you use the open aspect of that? Is the students’ work public or are you keeping it closed to the classroom, or is that something decided on a case-by-case or class-by-class basis?

Danica: Yeah, that’s a great point, thanks for bringing that up. It varies. Parts of the class web sites are public, parts of it are private… and another benefit of working in a quasi-public, quasi-private space is that it allows us to have conversations about what information students are putting on the Internet and what they want visible. What do they want to become part of their professional digital identity? What do they want to show up in search results versus what do they not want to show up in search results? So, we have a lot of these conversations early on in the semester when they’re establishing their accounts. We talk about the risks and the repercussions versus the benefits of using their real name to do the blogging that they’ll be doing on the site, and then for their final projects… often, but not always, I would encourage them to use their real name because they put a lot of time and effort there carefully revising these projects and they are deliberately constructing them with the idea that they’re going to be writing for a public audience. But, of course, in this climate of anti-immigration that we’re living in, you have to be super careful about what you’re encouraging students to put their name on and so I always have conversations around that. There’s always an option never to use your real name. You can always use a pseudonym for the blogging and for the final projects. You can always submit solely to me instead of publishing to a public audience, because I understand there are severe risks and in some cases they will outweigh the benefits of creating something publicly.

John: And we should note that you have an article in the describing your work here and we’ll include a link to that in the show notes as well.

Rebecca: You mentioned student blogging. Can you talk a little bit about how the student blogging is used in your classes and how that augments student learning and how that might facilitate some of these conversations that might be tricky to have?

Danica: Sure. I love student blogging—I don’t know how I would teach these courses without it. My courses are structured around the blog—it’s one of their major assignments, and so for every single class two or three students are assigned to blog about the assigned reading—I think the requirement is something like 800 words or so—and that they have to do a small close reading… so an analysis of the excerpt of whatever literary text we’re reading and it has to end with two discussion questions, and for every student who isn’t blogging. So, the majority of the class they have to leave a comment on those blogs before our class period starts, and so the blogs are due at noon the day before class and then students have from noon until our class period to leave their comments and then the way the course is structured the same day that those three students are blogging… so they’re each writing a blog… they are also facilitating a class discussion… a ten-minute activity… or it can be a presentation… it can just be more of a conversation. They have ten minutes at the beginning of the next class to do whatever they want, and I encourage them to make it the best lesson plan that they have ever seen or the way that they want their ideal course to be structured, and so I encourage them to try things like think-pair-share or to do interactive activities and it’s really exciting to see, first of all, the things that students choose to blog about, because with the readings that we’re discussing there’s so much that you could talk about. I have certain things I want to talk about but those might not necessarily align with what students are interested in within the text, and so having these open-ended blogs allows students to identify what it is they’re most interested in; it allows them to get feedback on their writing from their peers prior to our class session. One way that I’ve come to think about the blog… that I talk to students about it… is as a rough draft for a paper. They’re putting out a thesis… they’re putting out an interpretation… they’re providing some evidence from the text to support it… and then they have this tremendous opportunity to get feedback from all of their peers… and so in the comments the other students will be like, “I really like this point…” “I have another example that can help you support your point…” They might raise objections; they might raise counter points: “Well, have you thought of this other thing?” And so it’s a really great way for them to increase the quality of their writing and their ideas by getting feedback from their peers. Actually, this happened just in our previous class, a student was using a term “devaluing” to talk about sexuality in one of the books that we were reading and a lot of his fellow classmates were saying that word wasn’t working the way that he thought that it was working. So, in his facilitation he kind of talked through the feedback that he got and as a class we came up with a better word that would more precisely name the kind of relationship that he saw developing in the literary text, and so with the class facilitations it provides students with an opportunity to practice their public speaking and to practice standing up in front of a classroom. A lot of the students say that they’re really nervous at first, but that they’re glad in the end that they did it and they always get through it and we always manage… and so this kind of pairing of the blog with the in-class facilitation really teaches students that they are active knowledge producers and that they have something to contribute to the class and that their voice matters. They know that they’re not allowed to just disappear and sink into the background—they’re actually the ones up there in front of the class leading the lesson and it’s interesting to see actually the ways that it increases their performance once they’re back in the chair of the student, because they know what it feels like to be up at the front and so they’ll put out a question and they then get to experience what it’s like to have no one raise their hand and so they become much better as students and much more engaged once they return to their seats and resume that more traditional role of being a student. I never know what students are gonna do for their facilitation. They don’t have to run it by beforehand, so it’s always exciting. I don’t know what they’re gonna do in class today and it’s really made my role as an educator different and I’ve had to learn to listen really carefully to the things that students are saying when they’re up there presenting and my job becomes connecting what they’re saying to the main ideas and the main skills and the main topics of the class. So, for instance, if a student is giving a presentation I might interject and say that’s a thesis statement,… what you just said… you just made a thesis statement and then they start to recognize learning how to make an argument, how to make a thesis statement is one of the skills of the course, but it takes a long time or they’re not quite sure what I mean by that, but when they’re talking they’ll just do it naturally and so my job becomes pointing out to them that they are already doing the things that we’re learning about and just helping them recognize better the ways that their facilitations are connecting to the themes and the skills of the course.

Rebecca: I wanted to circle back to the leaving comments for other students—so they do the close reading, they post about that and then students comment on it. When they’re commenting, how do you help students learn what a good comment is?

Danica: Yeah, that actually becomes a topic of discussion early on in the semester. They’re given a few guidelines: it should be, I don’t know, a hundred and fifty words or so; it needs to make a contribution to the post; it can’t just be “I liked your post” or “I didn’t like your post,” and then what I’ve tried this semester is we implemented—kind of halfway through the semester—this rule that each comment needs to provide a quote from the text so that the commenter is either supporting providing further evidence that will support the author of the blog’s claim, or providing a counter example. One of my students last class, he said “Conversation makes the best interpretation” and I really loved that because they’re starting to learn through the commenting the ways that all academic writing is a conversation among various viewpoints and that when they’re writing a scholarly paper… when they’re writing a research essay… they are inserting their voices into larger conversations; they’re in dialogue with people. It’s not like you write a paper in a vacuum; it’s actually a synthesis of all these different viewpoints and ideas and so I see the commenting as kind of a rehearsal for class discussion. So when we show up in class, say, for instance, students aren’t being particularly talkative, I can say, well, you said this in your comment, and so I know that they’ve already engaged with the ideas and it allows often our class conversation to reach a higher level because they already know what several of their peers’ interpretations of the text are; they’ve already thought about them; they’ve already thought about the pros and the cons and how we might need to complicate some of these analyses; and so it just takes our class discussions to the next step.

John: Do you do anything to ensure that everyone responds to a certain number of posts to make sure that you don’t see everyone replying just to one other post to make sure you get some balance there? Do you have a mechanism for doing that?

Danica: Yeah, that’s a great question. So, for every class students have to comment on at least one post; they’re welcome to comment on more than one, but the requirement is one comment prior to class. I don’t have a mechanism for ensuring that. The class is 25 students, if we have three bloggers, one blog might get ten comments and the others might get four or five, and one way to kind of address that in the classes is encouraging students to think about their blogs, think about the title of the blog, think about the content of the blog, think about how they’re competing for the attention of their peers. I encourage them to say, “Okay, your peers have three blogs to choose from, how are you gonna get them to read yours?” It’s a way of getting them to think a little bit about audience and what is the function of a title. What is the work that a title can do? …and from that introductory paragraph how can they give their reader a sense of what their blogs gonna be about? How can they convince their reader that there’s gonna be a good payoff that their blog is worth reading? And so it’s interesting to see the different ways that they try to attract the attention of their peers. Because they do want those comments and I find they get excited about the different feedback—they’re not required to respond to the comments, but they do often… which, yeah, it’s always exciting to witness. I try to linger, I lurk a little bit on the blogs and I’m often not interjecting in the conversations, but just kind of reading through them and that’s actually the other benefit is that it ends up serving as a mode of formative assessment because I can see what they have understood from the readings and what might be missing… what might be the things that I need to address in the time that I have—what’s not quite getting through to them, either in terms of aspects of the reading that they overlooked or in terms of the skills. So, if I tell them that your blog needs to have a main point; it needs to have a thesis, and I’m seeing that they’re not quite doing that I can then adjust my lesson plans so that that becomes the focus of the next class and I can use their blogs, their own words as an example to say, “Okay, how could we give this blog a stronger thesis?” …and so it’s quite common that we’ll end up editing or revising some of the blog posts. They get projected up on the screen and students, because we’ve created a culture that they’re constantly giving feedback on each other’s ideas, students feel a lot less embarrassed or they understand that we’re all trying to become better writers and so they’re okay with it if I project their blog post and we talk through “What are some of the pros? What are some of the cons? How could we strengthen this?”

John: And it’s a much more authentic learning experience having them focus on audience and trying to build a strong thesis statement.

Rebecca: It seems like the blog post assignment really primes students well for the final projects that you had mentioned earlier that have a public audience because they’re already practicing writing for a specific audience and it’s another writing for a more general audience, I would assume. Can you talk us through that a little bit?

Danica: The final projects for my class often vary, but they’re usually collaborative… they’re usually digital… they’re usually public… they’re usually some kind of creative student-driven element. It’s usually students identifying the topic and then running with it, whether that’s a research blog or whether that’s currently my students in Intro to Multicultural Literature are co-authoring a glossary of key terms for literary studies—I have never done this before. It is a total experiment—I don’t know if they know that this is my first time doing this, so it’ll be interesting. I don’t know if they’ll hear in this podcast, but whatever, it’s fine. So I’ve done different versions of these collaborative public final projects. They vary sometimes based on the content of the course, students’ level of preparation, what are the aims and objectives of the different courses. It’s a little bit different for a basic writing composition course versus a more advanced literature course… and so one of the format’s I’ve done is have students co-author scholarly articles that they would submit to an actual journal, and so I did this in one of my freshman writing classes. We spent the entire semester talking about contemporary issues in education… so related to technology in the classroom… active learning versus lecturing… conditions of educational equality in segregated schools… and about halfway through the semester they were put into groups and they had to identify a research question. They did an annotated bibliography, they developed a whole research project, and then they made it into an article, a short article that they submitted to the scholarly academic peer-reviewed journal Hybrid Pedagogy to see if they could get it published or not, and I had been in contact with the journal’s editors since the summer before the class, so they knew this was coming. It would not have been possible if I hadn’t been working with them because they knew there was gonna be a really quick turnaround time where the students needed to know if they got revise and resubmit, if they got rejected, or if they got accepted… and I knew that this was a wildly fanciful or an unrealistic expectation to ask students to get a scholarly journal article published—these are basic writing students at Queens College—a lot of them are first-generation students… they work jobs… they are English language learners… and so in addition to reviewing of the conventions of English grammar and how to write a paragraph… how to write an academic paper… all things that were new or needed to be reviewed… they were also trying to get their writing published in a major publication… and so what ended up happening with that is that several students got revise and resubmit. But by the time they did it was the end of the semester and finals were happening and so I tailored the assignment a little bit towards the end. I tweaked it, because all semester I’d been telling them “these blogs are important, these things that you’re writing your research, everything, all of this matters because people are actually going to be reading this and you want them to take it seriously and you want them to listen to you. You don’t want to lose their attention halfway through.” … and so we needed to come up with a way that they would still get published even if they chose not to endure the editorial feedback loop of revise and resubmit, or the accept with minor revisions, and so what we had them do is they took the feedback that they got from the editors and several of the groups chose to post to HASTAC.org, which is a tremendous resource. It is an academic scholarly network of 15,000 plus members of scholars and students and academics and artists and activists and so there’s a special group within HASTAC that showcases and features and highlights the writing of undergraduates. So many of my students ended up submitting their final blogs there, but one group did continue—they kept revising their submission and going through the queries that they were getting from the editors and then the copy editors and just all of these stages of the writing process that were very new to them. This is a required writing course… no one showed up at that course eager to do all these drafts and revision and the skills that we teach in a basic writing course… but they continued in that editorial feedback loop for about a year after our class ended and then in August of 2017 their article was published in Hybrid Pedagogy, which was very exciting and so that is now something that they can put on their resumes and I was just so impressed with them for sticking through it because we know everything that goes into writing a journal article but for them they didn’t even know at the beginning of semester what a peer-reviewed journal article was… and so it was like a huge learning process. So, that’s one of the formats of these collaborative public final projects: submitting something to an established publication, which required a lot of willingness on the part of the journal editors to work within a really quick timeframe and the managing editor Skyped into my class several times and talked to students about the journal, helped them with their submissions, they got to pitch their ideas to him, it was great. Some of the other formats I’ve used that have also been good—I’ve had students write explicitly for HASTAC and that’s an opportunity for them to tailor their writing for a very specific community. So that’s something that we did this semester in the writing class that I’m teaching. We read so much about HASTAC… we read about its history. There was an article in Inside Higher Ed calling it the ethical social network and talking about their commitment to protecting their users’ data; we learned about who is a member of HASTAC, who are the different people who are reading it; how does the website organize information—by topic, by tags, by categories and so they were reading and analyzing the site itself before they even started writing their research blogs. So it was a similar process where they identified a research question and they authored blogs that were specifically going to be then tailored for the HASTAC audience, and so one of the big aims of that assignment was this skill of kairos and figuring out how to tailor your writing for a specific community of readers and figuring out what are the conventions? what are the affordances of this specific writing space? and how can I best get my point across to this very specific audience? So, that was useful in helping us have conversations about audience awareness and tone and how you make an argument; how do you convince someone that your point is right without alienating them. So that’s one example of having students write for HASTAC. …and the nice thing about HASTAC is that it comes with a built-in user community. You have sixteen thousand people who are visiting the site and reading things. The other kind of format for these public projects is what we’re doing now with the keywords. Students are co-authoring these individual keywords, they’ve identified specific words that have emerged that they’re interested in throughout all the readings that we’ve done this semester and through our discussions and through the blogs and the final product for that will be something that is hosted on our course WordPress site… so it’ll be a page or an offshoot—I’m hoping to write some kind of table of contents that will link to each of the student’s posts—hopefully there will be media, and so this will be something that they can then share. They can decide that they want to make it part of their professional identity, part of their portfolio, or they can decide not to—it’s really up to them. …and so one of the things that I’m constantly thinking about in developing these assignments is like how to actually connect students to audiences of readers and people who could actually benefit from a keyword entry on memoir or on ghosts… that’s another keyword, apparently we talk a lot about ghosts in my class… and this is coming out of the research that I’ve done on activist pedagogy and really thinking about the role of the teachers connecting students to audiences and people that could potentially benefit from the writing that they’ve been doing. So, thinking about these projects as both a benefit to the students in the class and also to larger publics and communities.

Rebecca: How have students responded to this sort of work? You know, you mentioned that, you know, some of the classes are required courses—students are not necessarily marching in excited to do these sorts of things… so it sounds like you’ve hooked them a little bit. What is their final response to these?

Danica: In general it’s been really positive. The jury’s still out for the semester. We’re gonna do course evaluations I think next week, so I’ll learn more. But in the past, I do a lot of framing around what we’re doing in part because I was always a very willful student and I did not like being told what to do. But if I understood why I was being told to do that thing then I would get really into it and really excited and so with these student-centered assignments and activities I’m always super explicit: this is why we’re doing this; these are what I see as the benefits of this; this is why we’re authoring a set of Community Guidelines; this is why you’re doing a presentation; you’re doing a presentation because public speaking is one of the most valuable skills that employers look for and so when I’m writing your recommendation letters I want to be able to tell them what a great public speaker you are and that’s why I’m asking you to stand in front of the classroom and facilitate this. Also, I should mention this semester and at SUNY Cortland a lot of the students are going on to become teachers and so it’s important to have these experiences at the front of the classroom. I think that being explicit really helps students, and the other thing I’ll say is that I often am explicit about how frustrating student-centered learning can be, and we talked about how it can feel difficult and how sometimes we just wish that the teacher would give us the answers rather than making us figure it out ourselves or making us work in small groups and so I try to create spaces for students to express those kind of emotions and reactions to things. I also try really hard in designing these student-centered assignments… to design… to create the conditions where for instance, we’re doing a collaborative writing project… I try to give them an assignment that actually requires multiple minds and that if they had tried to do that exact same assignment on their own the final product would not be as good as if they were doing it as a group. So I put a lot of thought into kind of carefully constructing these in a way that they will be oriented to succeed in them. Recently I wrote a blog on collaborative close reading, which is a really, really difficult skill to teach—it takes years, you know, for most of us to learn how to do close reading, but I’ve tried to create this assignment that had students work on it in groups, and so rather than having to notice all of a million different things that are going on in a passage of literature, they had a bunch of different minds put to the task and they were all looking at the same paragraph for 20 minutes and dissecting it and they were all contributing their different insights and so rather than having to go at it alone they were able to learn from the different perspectives that the other students brought to the text, and so I think just being really explicit about why behind everything has helped to ensure that the reactions have generally been positive.

John: How have other faculty responded? Have other people started working on building more productive conversations? Have other people in your department started working more on open pedagogy projects?

Danica: Well, I would hesitate to say anything explicitly about my department because I’m so brand-new. It’s my first semester in the department. But one thing that’s been super exciting for me has been to see people… especially with this recent blog that I wrote on collaborative close reading… it went viral on academic Twitter and people have been reporting back, because that’s one of the things that I asked them to do is let me know how it works… let me know if, you know, you have any suggestions for how to make it better… and almost every day I’m getting tweets from people at universities across the country saying, “I tried collaborative close reading and this is what my students did…” and they’ll post pictures of the passages that their students highlighted and so that makes me feel like I’m part of a community that is bigger than my own institution. So, when I’m running these, of course I hope that they will be helpful to my colleagues, but I’m also… I really feel like I’m part of a bigger academic community and part of that is because I post these blogs to HASTAC, and so it really is, people are a community of 16,000, however many users, but then it can get tweeted out. So, even if you don’t have a HASTAC account you can still read the blogs and there’s so many ideas about scholarship being really isolating, but things like that… and getting to talk to people and discuss pedagogy with people at different institutions makes it feel a lot less isolating. But in terms of your question about reactions of colleagues, I have been super lucky both at CUNY and now at SUNY in terms of support for the kinds of things that I’m interested in doing. These are schools with very strong commitments to education, where people are already interested in and talking about student-centered methods and curious and wanting to learn more. The other day I have my students write found poems—which is the genre of poetry where you take some kind of existing document—often it’s a bureaucratic document—and you make it into a poem by cutting it up and whiting it out and mangling it and turning it into poetry and my students created these awesome, awesome found poems; they were beautiful. So, we spent a day in class, I gave them whiteout… I gave them scissors… I gave them tape… They started as banal documents; they made them into stunning poems. They would bring in their tuition bills or song lyrics with offensive stereotypes in them—one of them brought in the transcript from the Brett Kavanaugh hearing and they took these documents and as a way of thinking about language and power they made them into these gorgeous found poems and so I went to the chair of my department and I said, ”Hey, you know, my students created these poems and I would love to have someplace to display them ‘cause I think they’re really awesome” …and not only did she give me permission to tear down what was on the bulletin board, she helped me do it. We tore down these old flyers that had been up there for decades and we put my students’ found poems on and so now we have this beautiful display in our hallway of student work and several of my colleagues have reported seeing students stop and read the poems and take pictures of it and so they’re excited to see their work has become part of this gallery. In general I’ve just been really lucky and fortunate to work with colleagues who are similarly invested in helping students. I did think of a few ideas for people who might not be so fortunate on ways that they could start doing student centered things. The first would be, and I’ve already mentioned this, creating a free profile on HASTAC.org because there are so many people out there doing really creative and exciting things in their classrooms… and connecting their classrooms to larger movements for social justice… and thinking about how do you engage students in really important discussions about contemporary social issues… and so HASTAC has been a phenomenal place for me to connect with other people who are doing that kind of work. I try to start small, and so something like think-pair-share is so easy… it’s taking an index card, giving students 90 seconds to respond to some kind of open-ended prompt, then they turn to the student sitting next to them, they share their responses and then we go around the classroom and I transcribe each group’s answer to the question on the board… and so their ideas become the material that I then get to teach the course through, and we crowdsource responses to some question related to whatever the topic of discussion is that day and something like that is so easy, it’s so simple—we have gone through… I would say we’re in the thousands of index cards in terms of my courses this semester. Because the students like it and they recognize that that changes the classroom dynamic. They recognize that suddenly it’s not just one or two students dominating the conversations. So, when they get up to the front of the class and they get to facilitate something I would say, I don’t know, 65, 70 percent of them choose to do think-pair-share because they recognize that it really lowers the barriers of anxiety about participating in class—everyone has 90 seconds to scribble something on their index card and it’s only an index card,— it’s tiny—there’s not any kind of pressure to write something beautiful and then that becomes just such an easy way to really transform the dynamics of the classroom… low cost… low time investment… and even when I’m thinking at the bigger scale of assignments and rethinking the research paper so that it’s not just being submitted to the professor, but it’s for a public audience, I try not to overhaul everything at once. So it’s like each semester I’ll try one new assignment. Not throwing everything away and starting from scratch each time, because it takes a lot of energy to do these things. And so thinking about how we can make small changes and experiments but not overwhelm ourselves or our students… and the other thing that I would suggest if somebody finds themself in a situation where they want to start trying these things but might not have the kinds of tremendous support that I’ve been lucky to have is… there’s just so much research out there on the effectiveness of student-centered pedagogy. We’ve read a lot of it in my course this semester on writing and education. We read a lot of the studies that have shown how positive of an impact it can have on students to discover ideas for themselves and to work in small groups and solve problems and arrive at answers rather than sitting and listening to a lecture. Just kind of having some of those studies in my back pocket—I’ve always felt that if I was called upon… you know, “Why did you do that? I can’t believe you let the students help assess each other’s papers.” I would have some things that I can cite, that I could go back to to say “Well, actually, it’s been shown that asking students to metacognitively reflect on the implications of their writing is a great strategy.” So that kind of thing has been really helpful for me in terms of thinking about relationships to colleagues and different reactions to this kind of pedagogy.

Rebecca: We normally wrap up by asking, what next?

Danica: Well, at the small scale, I guess, we have this digital glossary of keywords that will be coming out from my multicultural literature students… going to learn all about ghosts and power and assimilation and why these words are important for how we think about and analyze literature… so really excited to see what they do with that… and then I guess at the bigger scale I’m working on a book on the activist pedagogy of teacher poets from the 1960s and 1970s, and I’m hoping that some of that work will help us really understand the ways that a lot of contemporary student-centered practices… things that we’ve talked about today… a lot of them emerged in the 60s and 70s and especially in relation to the critiques of power emerging from the social movements of that era from the women’s movement and the civil rights movement and protests against the Vietnam War… and so I’ve been thinking a lot about how those critiques of power necessitated new relationships of teaching and learning and this was especially happening in the work of poets who I’m interested in and so that book is also considering the ways that interactions with students shaped American literature in ways that we rarely consider and also the tremendous role that poets and authors and especially feminist poets have played in creating a lot of the contemporary student-centered pedagogy that we know today to be so effective.

Rebecca: That sounds really exciting.

Danica: Yeah, it’s really fun to work on.

John: In some of your posts you’ve listed a large variety of techniques that people can try and we’ll include links to those in the show notes as well. Thank you, this has been a fascinating discussion and we look forward to hearing more about what you’re doing.

Danica: Thank you so much for having me, this has been really fun.

Rebecca: Yeah, thanks for joining us. [Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

59. Gatekeeping in Math Ed

Teachers at all levels often play an important role in influencing the educational and career paths of our students. In this episode, Dr. Marcia Burrell joins us to discuss how math teachers play a critical role as gatekeepers who may either welcome students to or provide a barrier to student success in all STEM fields. Marcia is the Chair of the Curriculum and Instruction Department at SUNY Oswego.

Show Notes

  • National Council of Teachers of Math (NCTM)
  • Budapest Semesters in Math Education
  • Polya, G. (1973). How to solve it: A new aspect of mathematical method. Princeton: Princeton University Press.
  • The Polya Approach Used at the University of Idaho
  • Bjork, R.A. (1994). “Institutional Impediments to Effective Training”. Learning, remembering, believing: Enhancing human performance.
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Miller, L. & Spiegel, A. (Hosts). (2015, January 23).Invisibilia: How to become Batman pt. 1 [Radio broadcast episode].
  • National Research Council, & Mathematics Learning Study Committee. (2001). Adding it up: Helping children learn mathematics. National Academies Press.
  • Brandsford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. National Academy Press.

Other resources:

  • Larson, M. (2016). The Need to Make Homework Comprehensible. National Council of Teachers of Mathematics.
  • Stinson, D.W (2004). Mathematics as gate-keeper: Three theoretical perspectives that aim toward empowering all children with a Key to the Gate, The Mathematics Educator14 (1), 8–18.
  • Burrell, Marcia (2016) Gatekeeping in Mathematics TEDx talk at OCC. January 29, 2016.

Transcript

John: Teachers at all levels often play an important role in influencing the educational and career paths of our students. In this episode, we examine how math teachers play a critical role as gatekeepers who may either welcome students to or provide a barrier to student success in all STEM fields.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Marcia Burrell, the Chair of the Curriculum and Instruction Department at SUNY Oswego. Welcome, Marcia.

Marcia: Thank you.

Rebecca: Welcome. Today our teas are…

Marcia: Earl Grey with caffeine.

Rebecca: Extra caffeine. [LAUGHTER]

John: Mine is just a pure peppermint tea today.

Rebecca: And I have a jasmine green tea.

John: We’ve invited you here to talk a little bit about the work you’ve done on math instructors as gatekeepers. What does it mean to be a gatekeeper?

Marcia: Well, I like to use the word gatekeeping because sometimes gatekeeping has to do with an open gate, where you can just slide right through, or someone gives you the key, or they’ve given you the secret password, or it’s a barrier, where if you don’t really know what the hidden curriculum is about passing through the gate then you could stay there and be turned away. And in mathematics a lot of times people are afraid of math or they’ve been socialized to think they cannot do math and it’s really a gate that’s been created either by themselves through socialization or it’s been created by a math person or by someone like a parent who said, “oh, don’t worry, I wasn’t good at math either.” So, when I think about gatekeeping and mathematics it’s really about barriers that are created by us or barriers that are created by others, or for people who are really successful in mathematics, they have an opportunity to open the gate; there are certain things that they can do that will make people pass through the gate more easily.

Rebecca: I think our students can empathize with the idea of gatekeeping when it comes to mathematics—you hear them talking about these stories of certain situations where the barriers have been in place for them, or sometimes that’s faculty. For example, I’ve heard many times in creative fields where the creative faculty might say, “yeah, we know you’re not great at math but you have to take math,” or I had a situation when I was a kid in middle school—I remember distinctly middle school teachers saying “the women in this class aren’t going to do as well” and then I remember the few of us banding together and then we got really good grades on this final exam that we were told that we wouldn’t do well in. I think that those narratives are certainly there and it’s interesting to think about it not only from the person coming to the gate but also from the gatekeeper perspective, which leads me to the question of, what are some things that gatekeepers do that keep people out?

Marcia: I’m gonna focus on math people mostly, where sometimes they say things like maybe in a beginning level math course, “Why didn’t you know that? You should’ve learned that before. I don’t understand why you can’t do fractions.” So, there’s vocabulary built into a lot of us where we send out messages which get people to realize, “Oh, there’s something wrong with me; I should know how to do this.” So, they start imposing those same messages on themselves. The other thing that I think is important is in mathematics there’s always been a stratification about who can do math or who should do math and who can be successful in math. Often, as you just said, women have stories about fighting to get into a advanced math class because they didn’t do very well on some class but they were willing to work hard. So, certain populations are harmed because they’re socialized that way that when women have trouble in mathematics we say, “Oh, we should make it easier; you should do a group of courses that are not gonna lead you to calculus in high school,” but sometimes when men struggle we go, “Oh, struggles perfectly fine.” In the U.S., teachers make it easier for students to learn; they give them answers, they work out all the details. When I say give answers, I mean they work out all of the problems so that it’s really just rote, as opposed to in other countries, struggle is actually honored—hard work and struggle is part of the mathematics learning process, where in the U.S. sometimes we don’t allow people to struggle. If you got a B in Algebra I, well, you don’t really need to take Algebra II because the minimum requirement in New York state is Algebra I, and the fact is struggle is a part of the learning process. Historically, we’ve always stratified who is successful in math or who can take math and the level of courses that people can take. Plato 2,300 years ago believed that everybody needed arithmetic, but the advanced math was relegated to philosopher guardians, and in the 1920s the National Council of Teachers of Mathematics argued to have mathematics part of the curriculum, and between 1890 and the 1940s there was a growth in public schools and the perception was that sometimes they weren’t sure that students had the intellectual capability of doing some of the mathematics that NCTM thought was important. But remember in the 1950s the business world and industry said, “What are you guys doing in schools? The people that you’re putting out there can’t do mathematics.” Well, that was mathematics for a purpose and then Sputnik happened and all of a sudden math became this subject that we wanted to make sure people had. But think about how many English classes do people take—one or two in high school, but in high school students often take four or five math courses. I’m not saying they’re not important, but it really forces us to think about mathematics as an elite subject when gatekeeping from my perspective is it’s not about an elite subject, it’s everyone can do math; people are born mathematical and everyone should have an opportunity to do the subject and not fail at it, but struggle and make movements towards whatever learning they need to do.

John: So a lot of this sounds like our society is creating or emphasizing or encouraging the development of fixed mindsets in math where many messages coming through (as you both have mentioned) in early childhood discourage people from thinking that they’re able to do math and only the elite can get through. Is that common in other cultures?

Marcia: I mentioned earlier that in other cultures hard work and struggle are honored and I witnessed in Budapest, when I was visiting there as part of my sabbatical, that students were asked to go up to the board and struggle through a problem, even if they had no idea. And we do a lot less of that because either you know the answer or you don’t; that doesn’t really work that way—it is an iterative process. I used to work on problems and maybe get a little frustrated, put it away and the next day I’d look at it and I go, “Oh, now I get it.” It’s really about process. The NCTM standards talk about process and product, and if you want people to learn mathematics then you really have to emphasize process, working in teams, giving people a chance to try things and fail but also collaborate with others to ensure that maybe there are multiple ways of approaching a problem, but if you’re not allowing students to talk with one another and work it through, then sometimes they think there’s only one way to do it and it really doesn’t improve their mathematical abilities. Mathematicians are about process—there are certain skills that mathematicians use. Good mathematicians persevere through problem-solving. They check their answers using different methods, they plan how to solve a problem versus jumping into a solution, and they justify the answers and communicate with others. Good mathematicians don’t just know the answer; it’s a process, and there’s even collaboration between mathematicians, but when we teach it on the K-12 level, we say, “This is what you need to learn and you need to learn it in a specified amount of time,” and so a lot of times students are turned off by the way we teach mathematics. Opening the gate is really about helping teachers rethink how they actually teach mathematics. We have a lot of data about how to successfully teach math, and it’s about problem-solving, reasoning, communication, connections and representations, but if you’re just gonna stand at the board and write the answer to a problem, that doesn’t help people really connect to how you came to that problem. So, gatekeeping is about getting teachers to rethink how they’re teaching mathematics and what they think is important. Process and products are important, but process is actually more important.

Rebecca: You mentioned mathematics as a collaborative process, but in my experience in K-12 I don’t think I ever worked with another person once.

Marcia: It’s funny you mention that. Again, the stratification stuff is huge. I attended a program called Budapest Semesters in Math Education and it’s geared for Americans, Canadians to come to this program. They’re interested in both juniors and seniors to come and learn about the problem-solving approach to mathematics. These are students who are mostly math majors, but they could be math ed majors, and they are sent to these schools where they’ve selected the top students in mathematics to use a problem-solving approach and what happens is they give them a problem with no background and they ask them to work out these problems. They can use their textbooks, they can use calculators, but the fact is our students—Americans and Canadians—get to witness students almost trying anything to work out these high-level mathematics problems—sometimes they’re theoretical, sometimes they’re applied. But what the students say who are in this particular program—and I got to be in these classes with them—was, “Why can’t we have all students use some of these processes?” And the processes are really just the things we already know that good mathematicians are supposed to do, sort of George Pólya, you know: analyze the problem, look at all the facts, try something, test your answer. But you actually get to witness that. So, when you asked me “None of the classes I ever went to that were collaborative and problem-based and working in teams,” well we seem to have an idea that only the gifted and talented or special programs will allow kids who already show aptitude to do mathematics in that particular way, and the fact is I visited a school in Budapest where this teacher who’s been working with the gifted and talented students got permission from the parents to try this problem-solving approach for a ninth grade through 12th grade. They had to get sign-offs by parents, because of course, in our system, if kids don’t know certain things by the end of certain grades then their opportunities—another gate—for getting into the university and going through the career path are cut off. So, these parents had to sign off that they were going to risk that what she was gonna to do over the next four years was gonna be helpful to their students and that they wouldn’t be harmed by doing this problem-solving approach. I witnessed several math classes where this teacher had been working as part of her dissertation to have students go through this problem-solving approach—it’s not just Pólya; there are other… Pósa there’s a Pósa method—I met this gentleman who, he was in his 80s and he invented the Pósa method and he’s one of the top mathematicians in his age… in his day, but he devoted his life to teaching problem-solving to kindergarten through grade 12. But the point that I’m making is, I witness students who had been through this process, and they were explaining problems to their peers on the board in ways that I haven’t seen good math teachers explain. But they built these kids up from start to finish to be confident about what they knew, to work in groups, not be afraid to make mistakes, and I think that we can do more of allowing students to learn not just at their own pace, but learn what mathematicians do—the process of engaging with one another if we weren’t so afraid of the whole accountability—what do kids know at the end of 12th grade? What do they know at the end of 11th grade? It’s recursive. Some things they learned in ninth grade in Algebra I will come back in Algebra II and when they’re college students they’re gonna pull the algebra and geometry together, if we allow it, as opposed to looking at these areas as completely separate things. One of the things about gatekeeping is that teachers have to think about students as already being competent; they’ve got to provide students with scaffolding so that students that are in different places have an opportunity to demonstrate what they know. I also think that we have to have high expectations, but we have to let students understand that they can extend the learning if they take some risks; that’s what good mathematicians do, and then we have to exhibit in depth knowledge as well as subject matter knowledge. So there are certain things that gatekeepers—math teachers—can do, but they’ve got to trust that students can learn, and we’ve got to keep the expectations high, but also scaffold for them so that they’re successful.

Rebecca: …a lot of evidence-based practices.

John: Yes, I was just going to say a lot of what you’re talking about, there’s a tremendous amount of research supporting that, not just in math instruction but across the board. In terms of providing students with challenging problems—you have the desirable difficulties of Bjork and Bjork, for example, and in terms of learning from mistakes, that shows up in all of the research on teaching and learning and it’s something that Ken Bain talked about when he summarized some of this research in What the Best College Teachers Do, and it’s also shown up in several of the books we’ve used in our reading groups, Make It Stick and Minds Online, for example: that retrieval practice, low stakes testing, where students can make mistakes and learn from mistakes, is effective in all types of instruction. So, these are really good practices that seem to be mostly neglected in math instruction.

Rebecca: I was expecting John to also mention something about growth mindset. [LAUGHTER]

John: I think I already did a while back, but treating math as something you’re either good at or not good at by teachers and by families and by our culture discourages the development of a growth mindset, and that’s really important. This year I’ve completely flipped my large microeconomics class and one of the things I had them do is before each class I asked them to do some readings and then I asked them to work through some problems in the readings; I have students submit a short Google form, where I ask them just two questions before each class. The first question is: “What have they learned from this reading assignment before that day’s class?” And also, “What are they still struggling with or what don’t they fully understand?” And half to two-thirds of them before each and every class list, “I have trouble interpreting graphs;” “I have trouble understanding graphs;” or that “I have trouble computing these things,” and that’s all basic math, and of course they have trouble doing it when it’s the first time they see it, but they see it as a barrier— “I’m just not good at it,” and every day in class I’ve been trying to encourage them to say, “Well, you may not do it now, but you can get better at this;” “You haven’t yet mastered this;” “You’re not yet good at this, but the more you do it the easier it gets,” and we’re not always seeing that happen, and we see that in lots of areas.

Marcia: Yeah, I think that students are more willing to say “I’m not good at math; I don’t have any experience with math,” but they would never say, “I can’t read; I’m not good at reading.” They might say it, but it’s socially acceptable to say “I’m not good at mathematics,” and the fact is when you look at a group of kindergarteners and they’re in a classroom, they’re all learning for the first time how to add and subtract and they slowly… I’m sorry, through some of their elementary school teachers who often are afraid of mathematics, and they say little things, “Oh, don’t worry about that, it’s okay to not be able to do that, we’ll work on that later on,” but they say it in a way that sometimes gives students permission to say, “Oh, I don’t have to learn that—I’m a girl, I’m a student of color, I don’t have to learn that because the teacher said she doesn’t know it either,” and so one of the concerns that I have for how we train childhood educators is we force them through, at least on our campus, these two math classes where they go kicking and screaming, but the fact is we almost need to reprogram them to think about the things that they can do mathematically and then build curriculum around them. It’s not always about the fact that the way you learn is the same way that all the kids that you’re teaching learn; it’s more about how do you change your perceptions about mathematics. There’s something on NPR, and I’ll have to find the reference a little bit later on, where this young man who was blind learned how to ride a bike, was sent to school, and people couldn’t even really understand why he was able to do all of these things as a blind person—well, his mother decided to treat him like he was a sighted person and it’s a Batman series, where the fact is, if you convince someone that they can do something and you believe it, then all of the things that you do to work on their perceptions about their capacity will come through. But first the teacher has to believe it and then they have to do all of these things to scaffold it. The fact is that, and again, I’ll have to find the researcher, but he did this study where he told all of his researchers that these mice were smart mice… these mice were everyday the same mice… what happened is the researchers came in and they treated those mice like they were smart—they handled them differently, they had them run through whatever people do in psychology with mice, and then he came back later on and said “All of these mice have exactly the same capabilities.” Well, that works in exactly the same way in the math classroom; students come, and if we believe that they’re capable and we come off and treat them with respect about what they have learned and how to build on that, then we’re gonna see better progress in their learning. I have to come back to the gate because the teacher has a lot of power to make the gate accessible or make the gate a barrier, and the barrier is really just the messages that the teacher says to the students and to herself about success in mathematics, and we lose entire generations of people when the gate is closed to them mainly because of perception.

Rebecca: So much discussion of gates it should be important to note that in front of Marcia is this picture of so many different kinds of gates in our conversation. Can you talk a little bit about the gates that you have in front of you?

Marcia: Yeah, I decided to Google different kinds of gates and when you think about the Brandenburg Gate or you think about gates like this one —remind me what this is called; this is in Cincinnati—the arch; this is really a gate, but this shows an opening to something, so when you think about gatekeeping in mathematics, I want us to think about people being gatekeepers for accessibility. So when you look at those pictures you think of when you’re going through the turnstile to pay with an EZ Pass. That is a barrier. If you don’t have money, you don’t have an EZ Pass, you’re not getting through, but if you look at the door to no return like in Benin it’s an opening to the next world just like certain pictures of gates just have you think differently about openings and closings.

Rebecca: There’s some like the dog pen where there is no way in or out; it looks like that one’s just closed forever.

Marcia: Yeah, which one is it? This one or this one? Right, I mean this has a gate, but often people are closed inside of thinking that they can’t do math and they can’t be successful. The job of a teacher would be to help them jump over that particular gate or find a different way to think about opening that particular gate. If you’re a dog and you’re inside of a pen, I think you’re just gonna need somebody to lift you up over that gate, and I think about that with teachers that what they have to do with each individual student is completely different, but their responsibility is to help them understand that they’re all mathematicians and they all have capacity for success in mathematics.

Rebecca: You’ve talked a little bit about how gatekeepers can open the gate or provide the leg up over the wall, or whatever it is, right, that’s there. Can you talk a little bit more about how to be inclusive and how faculty and teachers can really support this environment that would allow for problem-solving and allowing students to fail and try again and to iterate and eventually succeed?

Marcia: I’ve thought a lot about elementary school students and middle school students, where you’ve probably heard about the Montessori Method. The Montessori Method, you work with individuals to build from what their interests are and it turns out that students without a lot of direct instruction can complete whatever the curriculum is for that grade level by mapping to their interests, their strengths, and projects that they do where they’re learning the mathematics in ways that might be considered non-traditional. In the Montessori Method, they’re not just looking at memorizing times tables; they’re looking at multiplication as repeated addition, they’re looking at visualizations instead of just looking at a text. And the fact is that sometimes, I think, that if we allowed students to individualize their learning, especially in middle school and high school, that there’d be more progress than forcing students through the curriculum where each week they’re expected to learn something but they’re not learning it, they’re sort of just being dragged through the mud. And I have a lot of respect for my peers who are math teachers. I was a math teacher where I felt like I know what that kid needs, I need to take time to help that kid through what they need, but I didn’t have the courage to stop what I was doing and figure out how to individualize or make them work in small groups. I was a successful K-12 teacher, but I feel like I started to figure out what was needed when I made the decision to leave. So, part of my job as a math educator is to help our candidates who are gonna be teachers in schools to have the courage to do what they know is right: think about their love of mathematics and give kids problems that are theoretical and have them try it; give them applied problems, give them things where they have to use visualizations and not just know the procedures, but also understand the concept.

John: And also perhaps to use peer instruction, as you talked about, where students explaining things to each other reinforces learning for each student.

Marcia: Yeah, and sometimes the things that we expect of what we call the gifted and talented are exactly the things that other students can do but we’re afraid to take a risk, and I met earlier this afternoon with one of our adjuncts that’s teaching math methods to our graduate students and she said her job is to teach her candidates how to be good teachers, and sometimes that means forgoing what they think they wanted accomplished on that day and building something fun that’s gonna get students to see that math has many openings, not just following things through rote or through memorization. So, I had a really nice conversation with her because she does work in the school systems, but she’s teaching a course for us and she uses constructivist approaches. I have many peers that are still engaged in this math war that it has to be rote, it has to be step-by-step. In the constructivist approach, you care more about the process that students engage in and there’s a program that I listen to on Sunday morning it’s on NPR where it’s a puzzle and the puzzle is usually related to a vocabulary puzzle as opposed to a math puzzle, but the type of thinking that you have to engage to solve those puzzles really is mathematical thinking, so I love those puzzles, but they’re all couched in word puzzles… but it’s really mathematical thinking… and so I think the teachers need to use more of those word puzzles to bring people in so that they understand that they’re engaged in mathematical thinking—it’s just not called mathematical thinking. One of the other things that I wanted to mention before I run out of time is we are heavily tracking students into particular tracks. Sometimes you’re in the track where you’re just going to do Algebra I, and sometimes you’re in the track where you’re gonna get to do Algebra I and Algebra II, and maybe you’ll get to do Geometry, but some of the best learning occurs when there’s heterogeneous grouping and there’s less tracking. This gate stuff, these gatekeeping, really reinforces tracking, which when students come to SUNY Oswego and they’re in a remedial class and don’t know why they’re in the remedial class, because they may have been tracked in a particular way and cut off many, many job opportunities or majors because they were tracked in a particular way, and that is gatekeeping that occurs in fourth grade. And again our responsibility for our childhood educators is to get kids to think more broadly about what mathematics is; it’s not just arithmetic, it’s not just geometry, it’s not just theoretical problems; there are many types of problems that childhood people could engage students in that wouldn’t shut the door to possibilities 10 or 12 years later when students find out that they were tracked in a way that makes it so that they could never do graphic design or they could never do engineering or something else that they didn’t really understand was possible because somebody closed the gate early on.

John: …and that’s really important because most of the growth in income inequality is due to differences in educational attainment and the returns to education. And the returns to education in the STEM fields is far above the returns in other areas as well. So, keeping people out of those areas means that the people with those areas end up doing really well, but the people without those skills end up in jobs that are perhaps overcrowded with lower job prospects, lower prospects of growth and it helps reduce social mobility and economic mobility. It’s a serious problem in our society; it’s the worst we’ve ever seen it in the U.S.

Marcia: Yeah, I can’t connect it completely to perceptions, but a long time ago I taught a remedial math course at Clinton Community College and I had a student in that class and she was a smart person—I think everyone is smart—but I walked through how to study math, how to approach it: you are capable, work hard, keep asking questions, and about 10 years later I got a postcard from her—this flabbergasted me; she was in a remedial class and she had entered a PhD program in mathematics and she said it was just about the fact that somebody finally showed her how to study math—it was read the textbook, try the problems, come to class, listen maybe to the lecture, don’t be afraid to make mistakes; when you’re tired take a break. There are certain things We know that people can be successful in mathematics but we keep thinking that it’s this magic wand thing; it’s not a magic wand thing. We actually know —there’s research from Adding it Up —where we know exactly how people learn math well. The stuff research from Bransford, which how to study mathematics, how to learn mathematics, it’s written in black and white from large-scale studies, but then we return to the rote memorization, follow these steps and that’s not the beauty of mathematics at all.

Rebecca: One of the things that I think is really interesting about what you’re saying is that societally we might think, “Oh, fourth grade teacher… not really gonna have a big impact,” but you’re really talking about this fourth grade teacher is not a gatekeeper of the little gate around the garden; this is like the gate to the universe.

Marcia: Absolutely, and most of our math candidates who are not math concentrates—they’ve got to take these two four-credit math courses—will say, “I just need to get through this class; I hate math.” If you hate math it comes through loud and clear in your teaching; it’s really difficult to mask that. I taught a math for diverse learners course that the School of Education and Arts and Sciences Math Department and Curriculum and Instruction collaborated on and it was a math for diverse learner, so some of the things that I’ve been talking about here was in a full graduate course, and students would say, “Well, I never really thought about that; I thought everybody was gonna learn math the same way I learned math”—you’re a math person, I shouldn’t even say that. You’re a math person—you came through the system and you were successful in the current system, but if you want to build the next generation you’ve got to think about some of these other factors—you’re gonna be in a system, and as we’ve talked about systems, you are part of the system and you do have power to make changes to it, even if it’s perceptions, even if it’s just giving students the perception that you care about their learning and that they can succeed, and so this is really important to me. There are three principles: teachers must engage student misconceptions, understanding requires factual knowledge and conceptual understanding, and a metacognitive approach enables students self-monitoring. If I think about gatekeeping, if teachers kept those three principles in mind, they’re not mine—it’s in the research. This is sort of revolutionary because we don’t want to restrict people to thinking that only certain people can do mathematics, but if math teachers, whether they’re childhood or adolescence, or university teachers think about what good mathematicians do, they’ll follow these three principles and it might move us forward. I know it’s a big deal because the successful people want to keep what they have to themselves, but I think we miss out on the potential of entire generations if we don’t give them access to opening the gate through mathematics. When the Common Core came out teachers had the perception that they had to give these problems to students and parents would call and complain—“I can’t even do these problems; these aren’t the problems that I did when I was a kid”—well, the fact is we weren’t supposed to be sending these problems home; we were supposed to be doing those problems in class, and so a lot of the Common Core mathematics was supposed to be using manipulatives and getting kids to talk about how they think through the arithmetic problem. They were sending home problems and parents were complaining they were spending two or three hours to work through these problems, and there was an article put out—it was an NCTM—where they said what is your problem? No, don’t send these problems home for kids to fight with their parents, ‘cause that’s just gonna reinforce, “Oh, I couldn’t do math either;” it was supposed to be completely done in the classroom in collaborative groups, but we’re still not interested in teaching in that way. So, we sent home the homework—well, you could have been sending home memorize these timetables just as we did 20 years ago or 30 years ago, so finally NCTM put something out to help math teachers in the K-12 area not to send home these problems that would take parents two to three hours, but to rethink the organization of their classrooms where students could work on problems and have fun with mathematics, and the fact is that there are reforms that mathematicians fight about; there are a whole host of mathematicians that said Common Core was bad; Common Core is not bad, the way it was implemented was bad, so now we’ve done some backtracking to think about the fact that when you carry, when you’re subtracting or you’re adding, why do we do that? And the Common Core got students to make sense out of place value and make sense out of what it means when we carry this is about the tens place or the hundreds place and whenever you have new curriculum, Common Core or what was the curriculum in the ‘50s, I can’t remember… the new math… there’s always new math, it’s just an approach to make it more inclusive, but sometimes the way we roll things out makes it difficult, at least for the next generation of teachers, so I’m pro-reform movements, but we have to take the time and the energy to implement it in a way that’s actually gonna be useful—we just keep going back to the way we taught math a hundred years ago.

Rebecca: It sounds like what happened was faculty who knew how to do things a particular way get handed something that’s different but not a way of demonstrating or doing the different, right, like…

John: …without the professional development needed to allow them to implement it effectively.

Marcia: Correct. That’s correct.

Rebecca: The method doesn’t match the material.

Marcia: Exactly. At the same time they were putting out that students have to take a main assessment in fourth grade and eighth grade, but those assessments didn’t really align to this new Common Core curriculum, and so lots of things have changed over the last, I’d say seven to ten years, and we’re sort of coming out of that. When students come to the university level we still expect them to know mathematics. Do you remember twenty-five years ago they changed the math curriculum to be Math A and B, Course I, II, and III? New York state was the only state that was really thinking more globally about, “Wow, it doesn’t always have to be about algebra—it could be about statistics, it can be about more applied,” but the fact is universities didn’t change and we were still expecting students to know this narrow curriculum but it did broaden what people thought about mathematics, but it didn’t really help a lot of those students because then they were closed out of particular career areas because they might have been in a school that embraced applied math or embraced business math or something that might not connect to what they would do at the university level.

John: You’ve also been involved with Project Smart here at Oswego. Could you you tell us a little bit about that and how it relates to math instruction.

Marcia: Project Smart was a thirty-year project where teachers came to SUNY Oswego for summers to do professional development, math, science, technology. There are some teachers retiring over the last couple of years that came to Project Smart right from the beginning. We brought people in like Damian Schofield in the early days to learn about human-computer interaction. We brought people in from music and from art to help teachers integrate other things into their teaching, so they used to come for three weeks, then they came for two weeks, then they came for one week, then we built it into the department where faculty got released time to go into schools and work with teachers from the bottom up to think about how to improve teaching in their classroom. Project Smart really honored the work that teachers did because we would say, “What do you want to improve in your classroom? Are there particular things that you know students are struggling with?” This past year, funding for Project Smart ended, but the institution is still supporting individual faculty to go into schools and work with teachers to build classrooms that connect with the learners that they have in front of them. It’s more connected to what’s called a professional development school, where at the university we have the latest about how to teach, whether it’s math or English or social studies or modern language, and then we go into schools where they’re dealing with kids every single day and we try to help them figure out how to improve as a teacher; we meet them where they are; we build from there, so Project Smart is over—I’m not gonna say it’s dead, but we have a different system to work on professional development schools but just in a different way.

John: So you’re still doing the same thing even though it’s not under that official title?

Marcia: Correct. Correct.

Rebecca: We always wrap up our episodes by asking, what next?

Marcia: Oh my goodness, thank you for asking what next. After returning from my sabbatical, where I had the opportunity to be part of Budapest Semesters in Math Education where I got to see classrooms where students were using Pólya’s problem-solving approach in addition to something called the Pósa method, I worked with Josh McKeown, who’s from international education to reduce the cost of the Budapest program, so we’re working to recruit math students, both childhood and adolescence teacher candidates, as well as straight math candidates to consider going to Budapest over a winter course for one or two weeks over winter session or during spring break. What would they experience if they went to a short course? They would visit classrooms using the Pósa method, they would sit in on some of the math courses at BSME, where teachers are actually showing how to use a problem-solving approach in mathematics, where sometimes our students say “You talk about problem solving, you talk about the constructive approach, but no one is doing it so we don’t really know what it is.” The next step is to work with international ed to get a group of students to do the BSME program.

Rebecca: That’s really incredible.

Marcia: I’m excited about it too and I hope to also re-institute my math for diverse learners course because through that course I reinforce that I believe students should have access to high-quality, engaging math instruction. I believe all students should have mathematically rich curriculum. I believe all students should have high expectations and strong support, and we’re all gatekeepers— we are change agents and we control the gate. I think it’s ambitious because many people don’t agree with me saying that mathematics needs to be more inclusive, but that’s what I’ve been working for my entire career and I hope to continue that way.

Rebecca: Your work is incredible and we’re really excited that you’re doing that work.

Marcia: Thank you.

Rebecca: I know as someone who’s in a field that you don’t always associate with math—I believe in math and so I hope we can all help support your initiative.

John: It’s a major social justice issue.

Marcia: It’s a huge social justice issue because, again, what happens is often students of color, students that come from poor families may or may not have had the best math instruction. I mean, it’s a big cycle, and when they come here we should be able to help not just convince them, but this is a public institution. We should be able to provide access for them to reach whatever goals they hope to. We should be able to take students where they are and help them achieve whatever their focus is, whether it’s math related or not.

John: Well, thank you.

Rebecca: Thank you so much.
[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

58. Role-play

Do your students sometimes settle for a superficial understanding of your course content? Role-playing activities can provide an opportunity for students to become more fully immersed in the academic dialog of your discipline. In this episode, Jill Peterfeso joins us discuss a variety of role-playing activities that can be implemented into a single class session or over a more extended period of time. Jill is an Assistant Professor in and the chair of the Department of Religious Studies at Guilford College.

Show Notes

Transcript

John: Do your students sometimes settle for a superficial understanding of your course course content? Role-playing activities can provide an opportunity for students to become more fully immersed in the academic dialog of your discipline. In this episode, we’ll discuss a variety of role-playing activities that can be implemented into a single class session or over a more extended period of time.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Jill Peterfeso, assistant professor in and the chair of the Department of Religious Studies at Guilford College. Welcome, Jill.

John: Welcome, Jill.

Jill: Hi, nice to be here.

John: Our teas today are…

Jill: I am drinking candy cane tea; it’s a black tea. If you like peppermint tea, this takes it up a notch with even more sweetness. It’s really delicious.

John: What brand is that?

Jill: Adagio.

Rebecca: That sounds right up John’s alley, actually.

Jill: Oh really? For Christmas a couple years ago I asked for some and my parents are like, “Oh, what size?” And I was like, “You know what? I don’t know. A pound.” Well, you know how much a pound of tea is? [LAUGHTER] So I have enough to last me like a decade.

John: I had a mix of tea where it was peppermint, spearmint and tarragon and I got a pound of peppermint, a pound of spearmint and a pound of tarragon.

Jill: My gosh. [LAUGHTER] Yeah.

Rebecca: Speaking of lifetime supplies. I have English afternoon.

John: Again?

Rebecca: Yeah.

John: And I have blueberry green tea. We invited you here to talk a bit about how you’ve been using role-play in your classes. Could you give us some examples of what you’ve been doing with this and in what context?

Jill: Yeah, absolutely. This idea of working with role-play comes from my own interest as a theatre person in high school and college and even into my adult years and also just this memory I have of doing theatre where stepping into the role of another person opens up your mind in really different ways. I’ve devised a number of different things that I do in the class, sometimes borrowing from others, sometimes doing completely experimental assignments. So, I think that it’s sort of three different levels of immersion into role-play. A level one thing, for instance, might be I use dialog tests where I have students imagine dialogues. For instance, historical figures John Winthrop and Anne Hutchinson are having a conversation about conversion… what do they say to one another? And so instead of writing an essay on the thesis of conversion in Puritan New England, I have students imagine a conversation between these two historical actors. So, that would be an example of something that’s level one. Something that’s more level two… I often invite students to take on the voices and the ideas of authors, theologians and theorists that we’re studying. For instance, I teach an upper-level Holocaust class where we often read a lot of excerpts from very dense critical theory, for instance. I will assign students to different authors we’ve read… someone will get Habermas, someone will get Adorno, et cetera, et cetera, and then we come together in a colloquium setting and they need to speak in the discussion as that “author” or that character… and then sort of the deeper level of immersion would be something like reacting to the past, which is a very well established pedagogical role-play method in historical game that comes out of Barnard College and about 20 years old now—is developed by history professor named Mark Carnes—and in that students literally are assigned historical characters and then they play out some event from the past. Games that I have used in my classes include the Narrative Life of Frederick Douglass, Anne Hutchinson (who I mentioned earlier), the Council of Nicaea; I’ve done those in my classes, but they have them for all sorts of disciplines in all different time periods. So, that one, it’s several days. Students give speeches and form teams and do some politicking behind the scenes to come together and play their characters in order to see, like, literally playing with history.

John: In that second level of role-play, when they’re in the role of characters do you have students discuss contemporary issues or issues of the historical period?

Jill: When they are speaking as theorists or authors then I have them sort of in the secondary source mode so that they are speaking as contemporaries, even if it’s an Adorno or Habermas —who weren’t so much contemporary for us anymore—they’re still able to speak about contemporary issues. If we’re talking about Holocaust Studies, for instance, they are able to bring some of that to bear on whatever is happening here and now. I do something similar in a feminist theologies class where we read various feminist theologians over the course of the semester and one student is assigned a theologian each semester, one student per one theologian, and when we discuss that theologian the student speaks not as a student but as the theologian. So, it’s this extra meta-level that they that one student wears in this one moment or I should say in this one class period and what that lets them do is have this dexterity where they are connecting the text that they have a certain intimacy with as the “author” but then they’re also able to connect with their classmates who might be bringing up some of these different issues, like how this reading in feminist theology might connect to some of the reproductive issues that are going on now in politics or issues around concern for the planet, et cetera, et cetera. Really to your question, John, the way I see it, especially in that second level, is a hinging where they’re able to sort of pivot between the creation of the text and the application of the text and that’s one of the nice things about it because it keeps them again hinged where they’re connected to both parts and they’re aware of the fact that they’re swiveling, if that makes some sense.

John: It does. It sounds like they’re making some really deep connections.

Jill: Hopefully.

John: It’s a form in a sense of peer instruction.

Jill: Absolutely, yeah, thank you. That’s one of the things that I try to get them to do with role-play… with other activities I do in the class… but the role-play specifically… is to get students to realize that they can be instructors of their peers and just as successfully as I could be in some instances. It lets them feel that they are experts in what they are speaking on. That is ideally very empowering, but it also gives them—and I found this constantly with role-play, and this is something your audience might find interesting—is that when students are wearing a “mask” of someone else’s ideas or someone else’s character, they are much more willing to be directive with their peers and sort of challenge their peers if their peers are not thinking very critically or very clearly… and I’ve heard this from my students for the theologian activity in feminist theology… they get more annoyed if their classmates are not really stepping up and not really engaging “their ideas.” So, they’re able to say, “Well, wait a second, that’s not what I wrote. Look at the bottom of page 36…” and yet it works because no one really feels attacked by someone playing the persona of Elisabeth Schüssler Fiorenza. So, it works really nicely because that framing… that mask… however you want to think about it… creates the opportunity to step into a liminal space where it’s a little safer to push those boundaries, and students tend to do that and that allows them to do that peer instruction even more so than they would otherwise. I don’t think they think of it that way until after when I have them reflect, then they’re like, “Oh, yeah, I felt more defensive of these ideas and I also felt like I knew where they were coming from.” There was a material, historical context that gave rise to the need for me to write this theology or me to write this theory and they felt that attachment to it. To hear them say that I’m like, “Yes, that’s exactly what I would hope would happen.” As somebody who’s done acting and done theater, that’s the best part for me, immersing yourself empathetically in another experience and so it seems to work for students intellectually in a scholarly way.

Rebecca: …seems really powerful, but I can imagine that telling your students that they’re gonna role-play could be really intimidating. So, how do you prepare students for that experience?

Jill: Yeah, this is one of these things that I’m constantly trying to get better about. I tell them early and often. Whoever signs up for early on—so, again, right now I’m thinking about the feminist theology class where they have to step into and embody these ideas one at a time over the course of the semester—what I do is try to get some of my stronger students who might know me or have done this before in other classes to go first and I make sure to give them a special amount of direction and leeway and then after one or two students will go I will do a reflection like a stop, okay, what’s working. Students who are not doing this but were in the classroom discussion with the theologian, what are you noticing? And students who did this, what advice do you have for others? So, then again, the peers become the instructor. When it comes to other things, I mentioned earlier in my Holocaust class, and we do this sometimes in feminist theology, we do this in my Jesus in Film and Pop Culture class, where we really will be in a circle discussion and I mostly teach seminar. Disclaimer: most of my classes are 10 to 25 students, so this works really nicely. We’ll be in a circle and we’ll be looking at each other and channeling historians and scholars of the historical Jesus or Holocaust theory around memory studies. We might get into it and I’ll need to stop and say, “Okay, I want to pause. I noticed some of you are not speaking in the first person. Remember, I want you to be speaking as you’re scholar. Some of you are doing a really nice job with this, but I don’t hear you using quotes from the text, so remember the text is your foundation. The text is what gives you a platform. You don’t have to make up everything. You are using the text as a springboard to merge with your own ideas.” Constantly of doing that, modeling some of it for students as well and then affirming them. I think this all ultimately plays to where the majority of students get it at some point. But, Rebecca, I will tell you, I mean, you’re right. Some students never really get into this. They think it’s too strange or it’s too uncomfortable, or they’re really good students in the traditional way of doing things and they don’t think that this is something that they need or is helpful. That’s fine; not everything is going to work for everybody. What I love about some of these different liminal activities is that they will reach students who otherwise would feel that they can’t step into discussion in the traditional way because they don’t think they’re good at it, but giving them this additional costume of intellectual ideas to wear is liberating for some students, and that’s enough for me to do it once or twice a semester in some classes because it’s gonna invite in people who might not feel invited another way.

John: How long do these activities run? Is it a one day thing or multiple days?

Jill: It all depends again on which activities. When I do the symposia type models where we’re all together that’s usually at the lowest… it would be our 75 minute classes; sometimes I do it in our three hour classes, then it’s more about two and a half hours with a break in between. I’m getting ready to do one of these symposia in my Catholicism course and we’re gonna do it over two 75-minute classes, totaled about two and a half hours. What that does… and this ties to your question, John… what that does is it allows me to not be anxious that, “Oh my gosh, we have so much to cover and we’re not doing it,” and it really pushes me to the side, which is another key issue with this role-play is I as the professor in an ideal world create the settings, create the condition, give the instruction and then get the heck out of their way and let them stumble a little bit, let them struggle with some silence, let them look awkwardly at each other, let them look pleadingly at me but then turn to each other and realize, “okay, this has got to be us.” I encourage folks who want to try these sorts of things to give time because just investing in time means you’re gonna let the silence happen. Some things are much longer, so reacting to the past, for instance, which I’ve been playing with for the past year or more, that’s several weeks. We did the Anne Hutchinson game in my Religion in the U.S. class just last month and that was five 75-minute days and we’re gearing up for the Frederick Douglass game that starts next week. That’s gonna be six days. So, six days of game playing and then prep on the beginning and prep at the end. Doing that role-play meant completely redoing my syllabus for that course. There were reasons that that made sense given my teaching condition for that class, which I can get into if you’re interested, but that was a real total revamping. Everything from little bits to larger bits, depending on what you’re willing to invest in and what you’re looking to do with your students, what kinds of skills you’re trying to emphasize.

Rebecca: Jill, for someone who doesn’t have a background in theater, but…

Jill: Yeah.

Rebecca: …might find this to be a really interesting idea, what would you advise them to look at or how to start or an activity that they might do the first time out to just get their feet wet?

Jill: Reacting to the Past is a premade pedagogy. There are so many games I would recommend anyone who’s listening to this and thinking that sounds interesting to go to their website ‘cause those folks who run the Reacting Consortium will help you and there are so many games. I go to Reacting as a theater person who’s like, “Oh, won’t it be great if we all just sort of immerse ourselves in these characters of this historical moment and then give speeches as these characters,” this is like Jill in high school who did murder mystery weekends with her friend, like it’s getting these characters in and improving dialogue and a relationship and it’s just so fun, but that’s what gets me stuck on Reacting. A lot of folks who do Reacting are more gamers, they like that there are victory conditions and points for winning, or they’re historians who like this different way of doing history. That’s just my hook, but that’s not everybody’s hook. There are plenty of people I’ve met in the Reacting world who would never have thought of themselves as “a theater person.” So, I think that’s a safe one. Reacting has games as short as a day, as long as ten days. It’s good because it’s pre-made and you can go to conferences where you get to play some of the games, so that’s a good place to start. As far as some of the smaller ones, I think a safe and fun place to begin if you’re intrigued by this idea would be the dialogue assignments or the dialogue tests, sort of like I alluded to earlier, inviting students to put authors in conversation with each other… maybe across historical moments… maybe across religious traditions, in my case… maybe inserting themselves as a student into the conversation. And why is this valuable? Well, because when we want academic writing to happen, ideally students are putting different ideas “in conversation with one another”—juxtaposing different ideas—and so with these dialogue tests they were like literally doing that in a dialogue format as opposed to just writing a traditional paper where they may not be so aware that that’s what they’re doing. So… something very meta about all of this role-play stuff where they are with me—with the professor—the students are aware that they are trying on a different voice and that often for students makes something click. This is a different way of engaging. By the end of going through the process they’re like, “Oh, yeah, like I’ve made these discoveries that I didn’t think I would have permission to make otherwise.” Again, it gives them a permission to see and do something differently.

John: Could you tell us a little bit about the two reacting to the past scenarios that you’re running… in terms of what the main issues are that the students will be addressing?

Jill: The first one is the Anne Hutchinson game, which Anne Hutchinson was a Puritan woman… 1630s… Massachusetts Bay Colony… shows up… John Winthrop is the governor and they’re there to be the city on a hill to show the world this is what a true God-dedicated colony should look like; they’re gonna turn the eyes of the world to them and everything’s going to flow smoothly, and yet things start happening and people start getting religious ideas that aren’t quite in line with the orthodox and Anne Hutchinson is one of them… and she’s this woman… she’s a midwife… she starts having prayer meetings in her own home and, come to find out, that she’s having these visions and hearing these voices coming from the Bible and she believes God is speaking to her. And so doing she’s shifting the theology of the colony and people are starting to listen to her—that makes her dangerous. So this whole trial happened in the 1630s in Puritan, New England, and she is ultimately kicked out of the colony. The game scenario that we play in Reacting to the Past—again, I did not write this; I borrow it, I adapt it—much credit to the authors of the game. This game’s been running for many years. What happens is it’s this counterfactual where we imagine that there’s a second trial; she’s been banished but we’re gonna give her another try. What happens is you have the faction that is against Anne Hutchinson and then you have the faction that is Pro-Anne Hutchinson and then you have these indeterminate—this is a pretty standard format for reacting: you get one side, another side, and these indeterminates. The indeterminates are the ones you need to persuade; basically we have about three or four days of debate among these different groups trying to figure out what did she do wrong and do we want to let her back in, and the indeterminates are these immigrants who are arriving from England who want to get in the church. Before they can vote on Anne Hutchinson, they have to get into the church. So, they have their own objective of getting in but once they’re in both sides want these new immigrants to vote with them. It becomes this very fun game and what happens—this happened in my class just last month—a lot of students find themselves making arguments that they don’t agree with. They will step outside of class and say to me like, “I do not agree with what I’m saying; I can’t believe I’m arguing that Anne Hutchinson shouldn’t be speaking because she’s a woman. I believe that women have rights and should be able to have religious ideas and speak to men” and I’m like, “Yes, but you’re in a different historical context, so you need to be able to separate yourself from that.” I love—and I don’t say to students—I love that you’re trying to hold in tension what you think and feel with what you say and isn’t that how we sometimes have to act in the world? They did a really lovely job with that this semester. So, that’s the Anne Hutchinson game. The one we’re launching next week is the Frederick Douglass game which takes place in 1845. Abolitionism is really coming to its own as the political force. Slave owners are getting even more anxious and holding on to their power and the country is really in turmoil, specifically around the publication of the new autobiography by a man named Frederick Douglass. In this game there are even more historical characters—we have students who are about to be assigned the roles of John C. Calhoun and Senator Henry Clay and William Lloyd Garrison and Sojourner Truth and Angelina Grimké and they’re gonna go at it around issues of slavery, what the Constitution does and doesn’t allow about slavery and there will be indeterminates as well who can see the wisdom in both sides. They may not like slavery but is it politically advantageous to go against it at this point? This is a controversial game in some context, as you can imagine, because there will be students playing slave owners, there will be students playing former slaves, so you have to tread really carefully. That’s another thing with role-play, depending on your institutional context and who your students are, you do have to be careful as you ask students to step into roles that are not their own identities, but, again, that takes a lot of prep upfront. I think it can be done, you just have to be delicate with it. So with Frederick Douglass, for instance—I mean, who’s going to be John C. Calhoun? He was one of the biggest, baddest, most racist (by our terms) slave owners in the 19th century. Well, I’ve sold him this way to students. They’re gonna get input on the characters and I have an African American male in the class who said, “I want to play that part; that’s a part that I want,” and in fact when I did this last spring I had an African American male who wanted that part. I talked to them and I say, “are you sure? Why do you want this? And that’s awesome.” They say “I want to understand where they’re coming from, because that’s the worst thinking I can imagine and I want to know more about it.” So my student who did it last spring came to class in costume every single day as John C. Calhoun. I have pictures of the students wearing this wig and invariably he also wore some symbol of Africa on his person, like a shirt or a medallion around his neck. It was great what he was doing… how he was showing resistance to these messages that he was speaking in class—I mean, it was actually deeply profound. It also liberated other students to argue in the voices of pro-slavery advocates to have students of color in the class be willing to do that work too, and frequently at the end of classes—again, Frederick Douglass was about six days—like we would stop maybe five minutes early; I don’t know that the Reacting people would approve of us, but we’d stop and say, “Okay, this is getting heavy—how can we support one another around some of these really difficult conversations? How can we continue to support the pro-slavery students in the class…” because what would happen is the indeterminates in the Frederick Douglass game would just be like, “Oh, well slavery is bad; I know it’s bad ‘cause it’s 2018, slavery’s bad”—we just kept having to note that’s not where you are, but the water in which you swim as a mid-19th century fish is one in which slavery’s just accepted. You can’t don your 21st century hat and argue from that way. So, that’s also part of the learning objective. I think it’s probably clear that student collaboration is a big part of this… students having to work together in teams, having to come together around strategies and how to make an argument and who to target on the other team to try to turn their mind around.

Rebecca: Can you talk a little bit about the prep work before any of these role-playing instances? Clearly there needs to be some groundwork laid before you have a whole class period dedicated to any of these activities that you’re talking about.

Jill: Yeah, definitely. Letting them know in advance it’s coming is a big one. With Reacting it’s on the syllabus. I send out the syllabus about two or three weeks before the semester starts and I’m always like, “Hey, this is coming, so please look at this two-page document I’m sending you about what this entails; if this sounds great to you, wonderful; if it sounds miserable, let’s talk, because I don’t want you to be immersed in something that’s really unpleasant.” With the symposia that we do that’s a heads up in advance on the syllabus and then a constant reminder going into it—I tend to tell them within the days before to start making notes as you’re reading—read with the intent of thinking about how you would talk as this theorist. How would you channel these ideas? …and I also always for the symposia start class, ‘cause usually I can’t just have like one student as one person because they’re too many students, so I usually have two teams, like so there’s the Adorno team, whatever. What then happens is I give them about ten minutes to start, just to work together, to come, to brainstorm some ideas and maybe pull some quotes from the reading, also to prepare questions—that’s always the other thing, so it’s not just what are you going to say, but what are you going to ask of one another to keep the conversation going. Part of being in discussion is knowing how to ask questions and when to ask questions. Also encouraging them to draw connections between what different groups are saying—I never want the role-play to be an opportunity for every group to grandstand and then pass the torch to another group of grandstanders who aren’t really making connection. How does that work with role-play? That… either I’ve been modeling that all semester or I haven’t, but something I consciously do in class, I try to tell students, you know, when we’re building on each other’s comments let’s not change the subject without trying to bring in what has come before; if you’re going to change the subject, announce it and explain why you’re changing the subject. That’s part of the modeling that I try to do really consciously throughout the semester so that students get in the groove of how to have a conversation and then these other things just sort of kick it up several notches where then I’m more out of the way and they are hopefully building on tools they’ve been given and are ready to run with it.

Rebecca: You mentioned reflections earlier; can you talk about what that reflection process looks like in a little more detail?

Jill: Yeah, for the Reacting games, a whole day of debrief is built in; that’s one of the things that the Reacting people are pretty insistent on and I do not disagree with them. That tends to be a “Hey student, we just did this whole several days of this historical event. We changed history a little bit because in history this would not have happened and this would not have happened, so here’s what really happened and here’s why.” The debrief is very important for connecting back to larger class theme because, again, when you are sort of stepping outside of normal classroom behavior for a while it’s good to remind students as they gently re-enter why you’ve done what you’ve done and how it connects to these larger class themes, and that’s what the debrief is able to do. So, what you called reflection, Rebecca, I think of also as a debriefing. I also always have students write about these experiences. With Reacting they write a couple paragraphs reflection; I give them some very directed questions. After we did Anne Hutchinson, for instance, in my 101 class, I said, “What did you do well and what are you going to do better next time? Because we have Frederick Douglass coming up in a month.” …and then I share those with the students, like, “Okay, your classmates are really proud of how you all did this and here are the requests that people have for the class going forward. Many of you would like your classmates to prepare better; many of you would like your classmates to show up on time so you’re ready to give your speech; many of you would like your classmates to put more effort into writing your speeches and then delivering them with more confidence and poise.” In this way I don’t become that naggy teacher saying, “Okay, remember, we watched videos on public speaking before you delivered your speeches, but you are still not standing with much confidence and you are still reading from your paper.” Then it becomes the students doing it. Again, it goes back to John’s point before about peer instruction—the self or peer critique—and students don’t want to look foolish in front of their peers, so that sort of ups the ante there. With, for instance, the be the theologian assignment, and even the dialogue test, I always either give a journal assignment or even at the end of a dialogue test on a test say, “Okay, in three or four sentences what did you learn writing this as a dialogue that’s different from writing it as an essay? Or you’ve just performed the role in class of Mary Daly in feminist theology. What did you learn from being Mary Daly that’s different from you as Caitlyn talking about Mary Daly.” So, I think reflection is always such an important part of putting the lid on the assignment, really making it a full, complete thing so that it’s not that weird thing we did once in class but something that “Oh, like giving them the opportunity to make their own connections.” That’s why creating conditions for them to make discoveries for themselves and the reflection is sort of the last chance to do that and I don’t squander that opportunity. So, I think asking those questions and giving them space to reflect is really key.

John: There’s a lot of research that certainly supports that. Sounds like a great collection of activities. You mentioned of some concerns that students have. But, in general, how have students responded to these activities?

Jill: Yeah, a range of things. Some students, I will admit, seem confused. I’m thinking about the last time I did the be the theologian activity—I would say like the first month of class students were like, “What are we doing? Why are we doing this?” And I was trying to be patient and then I’m like, “What have I not been clear about?” And at some point it clicked and it seems to happen with role-play: at some point it clicked and it usually comes with one or two students and then like a lightbulb goes off and they get it and then everyone starts to get it. So I will say for anybody who’s thinking about these things or any creative pedagogy really from my experience: do it more than once, because the first time might not work, but that doesn’t mean that the pedagogy is not right; it might just mean that students are gonna need a little more time. Some students really thrive in it; they feel—I’ve talked about this earlier—they feel free to do college in a way that they haven’t felt free before and that’s really awesome to see because some of these are students who don’t speak. With Reacting, for instance, sometimes I’ve been in class with these students for a month or two months and suddenly we do this different thing and they just come to life and it’s really exciting. You get to see a different part of their personality. What is also exciting is how they then carry some of what they learned and some of the collaborative work that they did into future things like, “Oh, we really work together on this one game that we did, like maybe we can do our group project together at the end.” They respond really interestingly in that way. What I love is when I see then in their written work going forward how they make allusions to the role-play, even if it’s indirect. They start using some of the language and some of the teaching tools and some of the terms, it’s like they actually got it. So, it’s a real range. I’ve had some of my very best students not love it, but, yeah, I think those are the students who you pull aside and you talk to them about why because you can usually show them why you’re doing this pedagogy and why you’re doing something so different and they tend to have some really interesting ideas too, ultimately, and then they can sometimes help you reframe things. One of the things with this role-play stuff that I’ve been working on the past few years is I try to be creative but also humble. I’m not afraid—I try to not be afraid when students have critiques and suggestions ‘cause often they have some of the best ideas. They’re the ones who are doing it and so I invite them to do it; I think that goes to the reflection part that Rebecca had asked about earlier. Sometimes reflection means how would we do this differently? How can we do this better? …and sometimes that’s not just about students and their peers but also about me and the way the assignment is written.

Rebecca: How have your colleagues responded to what you’ve been doing?

Jill: Oh, good question. I’m fortunate because this year at Guilford we have a Center for Principled Problem Solving and they have faculty fellowships for a year and I was lucky to get one for the 2018-2019 school year focused on this performance and pedagogy stuff and specifically around trying to bring some of these ideas to my faculty colleagues. I should say again, I’m never an evangelist for these kinds of ideas because I think everybody should do them at all; I’m really an evangelist for teachers doing things that they think are cool and might work for their students and, while I’m not trying to force anything on anybody, but I am trying to help some of my colleagues just as they’re helping me to come up with new and creative ways to engage students and engage material and make what we do exciting to us. We’re going through a pretty significant curriculum and schedule revision at Guilford that’s gonna kick off next fall; we’ve got a lot of faculty who are rethinking their courses and course designs and activities and there’s not a small amount of anxiety about this change. So, one of the things I’m saying is, “Hey, this is a good opportunity to do some things that are more experimental and even experiential.” One of the things I did was, with the help of faculty development, brought in a Reacting to the Past Consortium board member who came and did a workshop for faculty development in September, and it was awesome. He was really engaging, gave us a lot to think about, and hopefully he’ll be back in January to do a small Reacting game. Reacting has some micro games that last an hour and a half. I believe some of Guilford faculty are going to go to a regional Reacting workshop in March. So, I’m trying to just invite people in—nobody’s being forced to do anything. I don’t have that kind of power, nor do I want it. I’m just trying to give people some ideas that have worked for me that I think are fun and that students seem to respond to and it helps our students. So, Guilford student population… we have traditional age students. we have very diverse, like ethnically… racially… in terms of class… we have a lot of diversity. We also have an adult population and then we have some high school students that take college students at Guilford in one of the best high schools in the state of North Carolina, so we have so much diversity, so how can we reach everybody? How can we invite everybody to the conversation? And this is one way that’s gotten people to sort of break down their walls. I think my colleagues are—some of them are suspicious and they should be—nobody should listen to this and be like, “Oh, this is brilliant, perfect, like, no, it’s not perfect.” Reacting to the Past is well-established, it’s not perfect. Some of my ideas aren’t perfect, but it’s a starting point and we can keep honing and keep working together to fix some of these ideas and that’s certainly what I’m doing. A lot of this work started with a fellowship I had a couple summers ago with the Wabash Center for Teaching and Learning in Religion and Theology; I got to spend the summer researching performance and pedagogy and that’s where I started to develop a lot of these ideas and some of the folks that run the teaching journal in conjunction with the Wabash Center, it’s called Teaching Theology and Religion—TTR. They’re excited about this; they think it’s worth hearing about so I’m working on an article with them—I’ve already published a few things smaller with them and I’m gonna work on some bigger pieces. There seems to be enthusiasm because I think we’re at a place where we want students to be engaged. The population of students seems to be changing in terms of their preparation for college, what they find interesting, what they’re willing to sit through in class. So, this is just one of many ways to get students trying a new way of learning. I don’t think if everybody did it that it would be awesome; I think it’s fun because they’re gonna remember in five years, “oh, yeah, in Jill Peterfeso’s class we did that really weird thing. It was weird, but it was also really cool.” I’m alright with that, I’m totally okay with that. Yes, I just say also like “I’m not afraid to be a dork about these things” and I think that’s disarming and students respond to that, ‘cause I’m like, “You guys are gonna get to role-play and I don’t get to play, but I get to watch, and do you want invite your friends?” and there at first they’re like, “no” and then by the end of the class, like we did this in the spring with Anne Hutchinson, I said, “So you want to invite like your professors or some of your friends?” and they’re like, “no, no” and then with Frederick Douglass they’re like, “We could invite everybody, like let’s put a message in the college newsletter,” like they got so into it. So, that’s learning and that self confidence and that’s not being afraid of trying new things and that transition over the course of a semester… something’s going on. I haven’t measured it and assessed it yet—don’t tell the administration—but it’s doing something and they’re learning because I read their reflections and what they come up with is pretty profound.

Rebecca: Sounds pretty incredible.

John: It does. I know in my own class I went from having students write papers to have them do a poster session and I asked if they wanted to invite other people and they were thrilled to have people from the department come in and the Dean came over and visited them and they were so much more excited and engaged about it. Small changes can make a big difference.

Jill: Yeah, I think that’s my thing whether you’re doing my level one immersion, level two, level three—those are just my categories—those small changes can mean a lot ‘cause even a little bit of reframing get students’ brains working differently and gets their hearts engage in different ways, so I totally agree with you, John.

Rebecca: I’m just sitting here contemplating how I can add role-playing into my Three Little Pigs exercise.

Jill: Aren’t you already doing it? [LAUGHTER]

Rebecca: A little bit, but I’m thinking about how maybe the students can do it more. I usually have someone come in and be the client and role-play the client role in my design class but the students are still acting as designers as humans but maybe they need to be characters in the Three Little Pigs or something for my assignment.

John: Actually, I was thinking of that—our second most popular episode has a title “The Three Little Pigs” and I can imagine all these parents playing it for their kids and finding out that it was really an exercise for a design class. [LAUGHTER]

Rebecca: Yeah.”

John: Much of what you’re describing in terms of being in this third party role is exactly the same type of thing, where students are able to see things much more clearly and are able to address issues that they’d be really cautious to approach if they were doing it in their own persona and I can see that connection and the benefits of that.

Rebecca: Somehow it’s just okay to embody that…

Jill: Yeah.

Rebecca: …other that they don’t feel connected to and explore and develop empathy and those sorts of things which is pretty powerful but I think the actual acting it out or writing the dialogues would really strengthen some of the things that I was already doing.

Jill: For sure.

John: How did you prepare to introduce this activity?

Jill: When I had my summer fellowship about performance and pedagogy I spent a lot of time doing research, starting to see who’s doing this and where and I was frustrated by the lack of what I could find in humanities classes or in sort of your more traditional classes—you get a lot of great activities coming out of theater classes or some more of the arts classes, but like high school classes or elementary school, like most of the role-play books I was finding were not geared toward college students doing the material that I wanted to do. So, I think there is room still for exploration and creativity here, that’s what I like about this. Reacting to the Past is certainly a place—Mark Carnes who designed it has written this great book called Minds on Fire, which I would recommend to anybody interested in this. There are other books about reacting that really do some pedagogical analysis of student experience in what’s going on. But, I think then within our individual discipline there’s a lot we can still do and I’m trying to think about that for myself as a religious studies scholar. I think there’s got to be stuff with empathy there and belief—I mean it’s really hard for students to try to understand beliefs of religious groups that they don’t subscribe to. This seems to be a way where they can at least intellectually be trying on beliefs of others just as they would an idea and I think that also shifts the location of some of these ideas where students are I find, “okay, I can understand that someone else may have this idea but to think somebody else may have this belief is like not about the head but the heart.” They’re more uncomfortable with that. So, trying to push those ideas of the heart as they see it up into the head, I think, could be really rich and beneficial for them. I’m sort of just riffing as I’m discovering this year but when we were talking about Puritan Theology… this Anne Hutchinson game… I just kept reminding them Puritans were intellectuals. These are highly educated people, so their beliefs weren’t just of an experience of God—it was well researched and reasoned with their relationship with scripture. So, I think there’s got to be some of that too… to think that where our emotions and our motivations come from connects heart and head; there isn’t some bifurcation of the two. I think that might help us as a society moving forward as we think about where some of our ideas and inspirations come from. I hope that what they take from some of these role-plays they’re able to put in other parts of their lives, that’s really the idea, ‘cause it’s more authentic than a classroom environment, this kind of here, I have some ideas and now I’m gonna improv conversation and ask questions and try not to step on toes… that’s life.

Rebecca: Sounds like a lot of interesting research can come out of what you just said. Sounds like you’ve got lots of plans. [LAUGHTER]

Jill: That’s what I’m thinking through right now. I guess that’s hopefully the next paper for the TTR Journal.

John: We always close with the question, what are you going to do next?

Jill: Next… So, I’m currently designing a new class. I alluded earlier to the new schedule that we’re doing, so we’re going to have three-week classes—some three-week classes, some twelve—and one of my three-week classes is going to be a new class called Religion, Voice and Performance, where I’m gonna use one or two Reacting games… we’ll see… in the service of helping students think through some of the things I was just talking about with empathy, compassion, belief, reason, rationality and relief, discovering voice, whether it’s claiming your own voice while speaking for another through Reacting and role-play or whether it’s trying to figure out who you are. I think that’s another beautiful thing about theater and acting is it invites you to figure out who you are while you are dancing around in somebody else’s shoes—that’s one of the things I’m working on now, which hopefully I’ll get to teach next year. Working on this article for Teaching Theology and Religion and I’m getting ready to keep working on these assignments that I’ve designed from the past and keep making them better. There’s always room to improve them, so those are my three things right now… and helping my faculty colleagues, as they may or may not want to try some of this stuff, so four goals.

Rebecca: It’s really exciting work, I’m glad that you were able to share it with us today.

John: Yes, thank you.

Jill: Thank you; thank you for inviting me.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

54. SOTL

As faculty, we face a tradeoff between spending time on  teaching and on research activities. In this episode, Dr. Regan Gurung joins us to explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.  Regan is the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research.

Show Notes

Show Notes

John: As faculty, we face a tradeoff between spending time on teaching and on research activities. In this episode, we explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.

[MUSIC]

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Regan Gurung, the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research. Welcome.

John: Welcome.

Regan: Thanks a lot, Rebecca and John.

John: Our teas today are…

Rebecca: I’m drinking Prince of Wales today.

Regan: Alright.

John: I’m drinking ginger tea.

Regan: Ooh, now you’re making me want to. [LAUGHTER]

John: We’ve invited you here today to talk about research in the scholarship of teaching and learning, or SOTL. You’ve conducted a lot of research on teaching and learning as well as research within your discipline. In most disciplines there has been an increase in the journals devoted to teaching and learning and an increase in research in teaching and learning, but it hasn’t reached everywhere yet. SOTL research is often not discussed in graduate programs and is sometimes devalued by campus colleagues. Why does that occur?

Regan: So. I think there are multiple reasons why the—and I’m going to start with the devaluing. I think there’s a lot of uncertainty about what it exactly it is, so on one hand, when people say a scholarship of teaching and learning… very often if it’s somebody who hasn’t really read up on it recently the sense is, oh, you know, that’s research on teaching; that’s not as good as your regular research. Now, I think that’s a misperception and once upon a time, and here I mean maybe even 15 years ago, there was some scholarship of teaching and learning that wasn’t done very well and I think people have heard about that in the past and that’s why there’s that knee-jerk reaction. Far too often it’s seen as something where it’s not as rigorous, perhaps, or it’s not done in the same way and most of that is wrong. What I like to tell folks who see that is, if you think the scholarship of teaching and learning is not rigorous, well, you haven’t tried to submit something to a journal recently. I co-edit a journal on the scholarship of teaching and learning in psychology and I can actually see some people submit poor work and I send it right back; I do the classic desk rejection and I say, look, this is just not good enough. So my favorite tip for “How do you write for a scholarship of teaching journal?” is very simple: just like you write anything else. There’s a lot of baggage, but I think that as you alluded to, John, it has changed more recently and I think part of what you notice now or what I’ve been seeing is that this kind of work, this kind of examination is being called different things. For example, a term that I’m hearing more and more often is DBER: disciplinary based educational research. And I’m hearing this come out of medical schools and engineering schools and social work schools and many professional programs where they’re doing DBER, which is essentially what the scholarship of teaching and learning is. So, I think because of that baggage with the term, people are calling it different things but in general the work is getting much more rigorous.

John: Excellent, and if changing the name is sufficient to do that, it’s a valuable step.

Regan: I think that’s why, when I talk about it I like to talk about it as: “Do you want to know if your students are learning? Do you want to know if your teaching is effective?” Well, then you should do some research on it. You can call it what you want. I started really calling it pedagogical research because that’s what it was, but it’s truly a rose by any name.

John: And that’s something that Carl Wieman has emphasized.

Regan: Absolutely, yup.

John: In the sciences, you test hypotheses and there’s no reason we couldn’t do the same thing in our teaching.

Regan: Exactly.

John: And that’s starting to happen, or it’s happening more and more.

Rebecca: In some disciplines, the scholarship of teaching and learning is not accepted as being part of their tenure and promotion file, for example. What would you recommend faculty do in a department like that if they really want to get started in SOTL?

Regan: Well, so, Rebecca, let me take you a half step back.

Rebecca: Yeah.

Regan: When you say “in some disciplines it isn’t as accepted.” What has surprised me is that most disciplines have actually been doing the scholarship of teaching and learning and publishing it for the longest time. I mean, if you take a look at chemistry, it goes back, gosh, seventy years or so. Almost every discipline out there has a journal that publishes the scholarship of teaching and learning, but, and here’s the big but: most of us in our normal training never run into it. So, I’ll take my own case. In psychology, the Teaching of Psychology Journal has been around for 46 years, yet all through grad school, all through my post-doc I never even knew the journal existed. Why? Because the programs that I went through weren’t focused on teaching the individuals—wonderful as they may be—who I worked with didn’t do that kind of work, so they didn’t know about it. So I think that’s a really important fine-tune there: there is a journal in almost every discipline—almost every discipline—for the scholarship of teaching and learning. So, it’s just a question of discovering it… it’s a question of finding it. Now, that said, where can they start? I think I can answer your question from a conceptual level and from a practical level, so I’ll start with the practical. The easiest place to start, there are lots of compilations of how to do it. For example, I think both of you have my website. On my website I have a simple tab called SOTL. On that tab is a list of places to get going, and I’ve organized it so that there’s a brief introduction to SOTL, there are journals, there are resources, there are little handouts. So, if a faculty member has even ten minutes, go to my website, hit SOTL, scroll through. That’s the more practical, that’s the easiest way to get started. From a conceptual standpoint it really starts with the question, what aspect of your teaching or your student learning are you curious about? John, I know you do some work in large-class instruction in economics. Why is this assignment not working? Can I get my students to remember certain concepts better if I change how I present information? It starts with a question. And you don’t have to read anything, you don’t have to look at any manual. If you look at your class and you go, “Hmmm, why isn’t this working, or why isn’t that working?” That’s where it begins, and from there you follow the same route that we always do: go look at what’s been published in it, fine-tune your question, design, think about what do you want to change and so on and so forth. I think it’ll help if I give you my working definition of the scholarship of teaching and learning, and when I think about it I think of SOTL as encompassing those theoretical underpinnings of how we learn. And more specifically, I see it as the intentional and systematic modifications of pedagogy and here’s the important part: the assessment of the resulting changes in learning. So that’s the key: you intentionally, you systematically, modify what you’re doing and then you measure whether it worked or not. That’s it. I could say that nonchalantly. There’s a technique , there’s a robustness to it, but at the heart, where do you start? You start by asking the question.

Rebecca: I think one of the things that I hear you saying is not much different than someone has a really reflective teaching practice—they’re doing it but not in that systematic way?

Regan:Yeah, absolutely right. There’s a term called scholarly teaching and in this kind of literature there’s a distinction made between scholarly teaching and the scholarship of teaching and learning, and all the distinction is is that scholarly teacher is reflecting on their work and then you’re right, you’re absolutely right; making those intentional systemic changes. That’s scholarly teaching. When it becomes the scholarship of teaching and learning is when you present it or you publish it, preferably through peer-reviewed ways, but you’re absolutely right; at the heart of it it’s scholarly teaching. It’s reflective intentional systematic changes.

John: One of the barriers, that people who are considering doing research in the scholarship of teaching and learning, is going through IRB approval, and in many disciplines that’s something they haven’t experienced before. It’s common in psychology. It’s less common in economics and perhaps in art.

Rebecca: It doesn’t exist in design. [LAUGHTER]

John: Could you tell us a little bit about that process?

Regan: Sure. Every university has an institutional review board and essentially what that board does is it’s in place to make sure that any research that’s being done isn’t harmful. Now, normally when we think about harmful we think about a drug or a food substance being tested, but here it just means any research that’s being done, and so when you do the scholarship of teaching and learning or when you’re examining your classes, yes, you could just look at your exams and see if exam scores are changing, but, if you do want to publish that, if you do want to share that, you really should go through institutional review board review. Now, the key thing here: it does sound like this whole new world, and it is, but at the heart of it is a very simple process. Now, there are three levels of review and I think knowing about the levels helps. For example, the first level is called an exempt review. The next level is called an expedited review, and the third level is called a full board review. I don’t think I’ve run into scholarship of teaching and learning that has gone through a full board review, because we’re not doing things that are more than minimum level of stress. Now when you say, hey, hang on, I didn’t know they were stress involved. Well, anytime you ask anybody to fill out a survey, there’s a minimal level of stress. And when you’re asking your students to reflect on their learning, well that’s a minimum level of stress. Every university has its own procedure. SUNY Oswego probably has a forum online. It’s a short forum; you’re basically telling this board what you plan on doing, what you plan on doing with the information, and most importantly, in these kind of cases, you are letting the board know whether or not students will be put under duress. What the IRB is going to look for is are you the instructor in some way forcing your students to do things that normally wouldn’t be done in the normal course of the educational process. But at the heart of it, all you’re doing is you’re sharing with this board whether or not you can do it and most scholarship of teaching and learning is at that exempt level. That exempt level essentially translates to exempt from further review. It doesn’t mean exempt from being reviewed; it just means this is mundane and low stress enough that it’s exempt from further review. Now that second level, expedited. If you do want to measure or keep track of names, if you want to look at how certain names relate to scores down the line—and that’s actually some really key research—well that’s expedited review. Now, even there it’s reviewed by one person. Both the expedited and the exempt review are reviewed by one person, often the chair. It often takes no longer than a week, and by doing that you just know that all your t’s are crossed and your i’s are dotted and it’s the ethical thing to do. So, whenever people say: “Oh, this is really mundane and I’m not really doing much more than just measuring student learning,” I still sa y if there’s any chance you want to present it or publish it make sure you go through the IRB.

John: And many journals will require evidence of completion of the IRB process.

Regan: Oh, absolutely. The moment you want to publish it you have to sign off saying that you got IRB review..

John: We do use an expedited review process on our campus. I was going to say, though, that we’re recording this a bit early because we’ve recorded a few things in advance, so we’re recording this in late October, but just yesterday I read that Rice University has introduced a streamlined expedited review process or IRB and apparently that’s something that’s been happening at more and more campuses. Are you familiar with that?

Regan: You know, not as much, because right now there’s so much up in the air with the IRB because national guidelines are changing. They were supposed to have changed in January, then it was moved to July. The latest I heard is it’s moved to next January. So, for the most part actual regulations are changing. Even on our own campus we switch from one form of human subjects training to another form, but this so called short-form expedited process will definitely help. That said, even the regular expedited, it’s a very easy process and I think the neat thing about this—and I tell students this when I’m teaching research methods, too—as the instructor or the researcher, just going through that IRB form really reminds you of some key things that you may have otherwise forgotten about, so, yes.

Rebecca: Do you talk a little bit about your own research to give people an overview of what project might look like from the beginning to the end?

Regan: Sure. What really got me interested in this is I teach large introductory psychology classes, the class is 250 individuals and I was struck by how when publisher reps come into my office and try to convince me to adopt one book over the other they would talk about the pedagogical aids in the textbook; “oh, look, our book has this and our book has that.” And that really got me started studying textbooks and how students use textbooks. So the umbrella under which I do research is student studying: What’s the optimal way for students to study? …and I use both a social psychology and a cognitive psychology lens or approach to it and it really started with looking at how they use textbook pedagogical aids. So, for example, in one of my really first studies I measured which of the different aids in a textbook the student uses and then I used their usage to predict their exam scores. Now, what I found, and this is what really surprised me and got me doing this even more, is that even those students were using and focusing on key terms a lot. Now, mind you, I’ll take a half step back—you may not be surprised to know that students use bold terms, they use italics, that’s what they focus a lot on. But students in my study also said that they use key terms a lot. Now if you’re studying key terms that should be good. If you’re making flashcards and studying those key terms that should be good, but what I found is that the more students use key terms the worse their exam scores. There was this negative correlation and that’s completely counterintuitive. Why would they go the opposite direction? So, I dug into it some more and I realized that students spend so much time on key terms or so much time on flashcards that they’re not studying in any other way. So even though they’re using flashcards, they’re so intent on memorizing and surface-level processing that they’re not doing deeper level processing. So, that was some years ago and I’ve been building on that, trying to unpack how students study. My most recent study… that’s actually under review right now… a colleague, Kate Burns, and I took two of the most recommended cognitive psychology study techniques, which is repeated practice or testing yourself frequently and spacing out your practice or spacing out your studying, and we took both of these and across nine different campuses divided up classes such that the students in those classes were either using high or low levels of each of these. So, in one study across multiple campuses we tested is there a main effect of one of these types of studying or is there an interaction? And what we found is that there is an interaction and the critical component seems to be spacing out your studying. Not so much even repeating your studying, but really spacing out your studying, and I think what’s interesting here is the reason this is happening is the students who said that they were testing themselves repeatedly, that sounds great, and if you’re a cognitive psychologist you say, hey, the lab says repeat testing is great; the problem is in the classroom a lot of students who were repeatedly testing themselves were repeatedly testing themselves during a really short period of time.

John: Right, I’ve seen that myself.

Regan: And I think that’s the issue, but because we had both these factors in the study, we could actually tease that out. So that’s the kind of work that I do… is take a look at what the cognitive lab says is important; let’s see how it works in the actual classroom.

John: Now was this a controlled experiment? Or was this based on the students’ behavior?

Regan: So, yes and no, okay. [LAUGHTER] I love this study because of a number of reasons. Number one, we tested two different techniques in the same thing. Number two, we did it at multiple institutions, so it’s not just my classroom. A lot of SOTL is one class. So, here we went beyond to try and generalize. But, to get to your question, we actually used a true experimental design. So we recruited these different campuses and we assigned a classroom. So, for example, I’d say, “Hey John, thanks for taking part. If you can have your students do high repetition and high spacing?” “Hey Rebecca, thanks for taking part. Could you have your students do high repetition and low spacing?” And that’s how we spread it out. We had about two campuses in each of these cells. That’s the true experiment on paper. To get to the other part of what you said… in reality, that’s not exactly what students always did. And you know students; we can tell them to do something but a whole bunch of things gets in the way. Fortunately, of course, we measured self reports of what students said they actually did and it was relatively close to the study cells, but even though it varied a little bit we could still control for it. So, yes, it was close to a controlled study as much as you could control something in the real world across nine campuses.

John: That brings us to the general question of how you construct controls. Suppose that you make a change in your class; how do you get the counterfactual?

Regan: Right.

John: What would be some examples for people designing an experiment?

Regan: The word control, especially in research, has the true connotation of the word control group and that’s controlling for factors as different from having a control group. Optimally we’d love a control group. The problem with the control group is that it means no treatment. So, very often a true control group means this group of students is not getting something. From a philosophical and an ethical standpoint, I don’t like the notion of one group not getting something. So, the word I like to use is comparison group. So, your question still holds, but what’s the comparison group? I think here’s where if you’re fortunate enough to teach multiple sections, well one of the sections can be the comparison group. If you’re not fortunate enough to have multiple sections, you compare the students this semester with the students the last semester when you weren’t doing that new, funky innovation. So, there are a bunch of different ways to gather the comparison group, but you’re absolutely right: having a comparison group is important. Most commonly in scholarship of teaching and learning, the comparison is the students before that intervention, so it’s a classic pre- and post- measure. I’ll give you this quiz before I’ve introduced the material, I give you an equivalent quiz after, let’s see if there are changes in learning. And that’s the most common comparison; you’re comparing them with them before but optimally again you want a different section, you want a group of students, a different semester, or so on, and so on.

John: And it’s best if you have some other controls…

Regan: Absolutely.

John: for student ability and characteristics.

Regan: You nailed one of the key—my two favorite are effort and ability. As much as possible, measure their GPA. If they’re first-year students, measure their high school ACT scores or their high school GPA and then you have to measure ability, and I think those two are probably the usual suspects for control. And again, a lot of SOTL doesn’t do that and it should.

Rebecca: I think one thing that comes up a lot for me (and maybe some others who are in disciplines maybe more similar to my own) is that the kind of research that we do is not this kind of research generally, but we’re really interested in what’s happening in our classrooms. So, for faculty who might be in the arts or some other area where we’re doing really different kinds of research, how would you recommend being able to partner or do this kind of work without that background?

Regan: And I think implicit in your question is the “Do I need to have a certain methodological tool bag?” and I remember I was at a conference once and somebody accosted me and said “Hey, is it true that you have to be a social scientist to do this work?” And the answer is no, and I wrote a pretty funky essay called “Get Foxy,” which is how social scientists can benefit from the methodologies of the humanists and vice versa. But, you’re right; you can collaborate if you need to do that kind of work, but there are a lot of questions even within your discipline… and when I think about SOTL I think about answering questions about teaching and learning with the tools of your discipline. Now, I’ll give you an example: a good friend of mine was an art and her project, or something that she wanted to dig into, was to improve student critiques in an art class. Here we have students learning how to do art (and I think it was drawing or jewelry making) and across the course of the semester everybody had to present their work and then critique each other’s work… and those critiques, they just didn’t have the teeth that she wanted them to, so she was giving them skills and how to do it. So here’s a case of how did she know whether or not the critiquing tools were increasing? Well, she came up with a simple rubric and to score them against and look at if the scores changed. Now, you may say, well, we very often in the arts and theater you don’t get skills to do that, which is true, but that’s where I think collaboration comes in and that’s why what’s really neat about scholarship of teaching and learning is very often there are class collaborations. I have a historian on my campus who wanted to change the quality of his essays and he and David Voelker changed how he was teaching and wanted to see it roll out and had students on their essays use teams in a different way. Well, he compared, and John this goes back to your point, he compared essays from before the change with essays from after the change, counted up the number of teams students had and then, Rebecca, to your point went over to my colleague in psychology and said, hey, can you tell me if this is statistically different. So, he didn’t even bother with doing the stats; he just said, “Hey look, I don’t need to do the stats.” But you can, in a click, and literally within minutes my colleague in psychology had done the stats for him. I think that’s the kind of stuff that can happen to truly get at those answers if you go, “You know, I don’t know how to do that.” But, you’d be surprised… the basic skills for SOTL can give you enough to test questions pretty well.

Rebecca: I think John and I have also found in the teaching center that it’s really exciting when faculty from different disciplines start talking about their research when they’re looking at learning because there’s things that we can learn from each other and the more that we’re talking across disciplines can be really valuable as well.

Regan: Right, and I think this is where reading the rich literature that exists in your discipline or even across disciplines on scholarship on teaching and learning really gives you the leg up, because I find now when I do workshops and somebody says, “You know, I’ve got this question; I don’t know how to start.” More often than not it’ll remind me of a study that I can say, hey, here’s what you can do. And it’s just because I read a lot and I’ve got all that in my head and I just matched to that question and it’s pretty easy. I mean, very rarely do we have to invent something from scratch. We go, “Hey, yeah, you know what? Here’s the study that’s pretty close to the question you have, let’s use that methodology.”

Rebecca: So, how do we build a culture of the scholarship of teaching and learning—the departments who might have faculty who are resistant to the idea of their colleagues spending their time doing that? How do we start changing minds and really building a culture that embraces the idea of the scholarship of teaching and learning?

Regan: Well, I think you’ve got to attack it from two different levels. You definitely want a champion in the administration who is educated enough about the scholarship of teaching and learning and how it can be done robustly. If you can convince somebody of it’s worth and then if you go “How do you do that?” …well that’s where you need to make sure you have at your fingertips, as a teaching and learning center, the exemplars of really robust work… and I think if you have that really robust work at your fingertips, that’s definitely a key place to start. One of my favorite examples along those lines of trying to convince (especially administrators) about the worth of scholarship of teaching and learning, I recommend a 2011 publication by Hutchings, Huber, and Ciccone, it’s called A Scholarship of Teaching and Learning Reconsidered and this 2011 publication is a great collection. It does your homework for you. That one book pulls together evidence for why scholarship of teaching and learning helps students, helps faculty, helps institutions. So that’s where the top down—get your administrators to check that book out and go, “Oh yeah, look, there is actually some good research.” Coming at it from the other angle—I know this for a fact—there are people on your campus doing some of that work, but often they may be isolated, they may be a small group. You want to strengthen them so that they can spread that to their circles, and that’s really how it starts. On my campus, when Scott was the Dean at Green Bay, we did a lot to develop scholarship of teaching and learning through the teaching center. There was one year where we had 14 faculty who got together every month and talked about their projects. Now you may say, well, that’s 14 and you had 160 faculty. You know what, you do 10 of working every year and colleagues see the value of the work those 10 or 14 are doing, pretty soon you’re gonna have a culture where people recognize it more and appreciate it more. So I think that’s how it goes… you put your efforts on those people who are already doing it to make them stronger and that’s gonna spill over and pretty soon you’re gonna win over folks.

John: We generally had support from the upper administration and there’s often been a lot of faculty who are new, interested in doing it; it’s usually the promotions and tenure committees that have served as a barrier in some departments, but we’ll work on that and we need to keep working on that.

Regan: Well, just along those lines on our campus we felt so strongly about the scholarship of teaching and learning that the Faculty Senate actually passed a resolution recognizing the importance of scholarship of teaching and learning. Now again, it still gave department chairs some leeway, but at least the faculty voted on it as something that the university values and that goes a really long way to having especially junior faculty say, you know, I can do this.

Rebecca: Certainly makes faculty, especially junior faculty, feel supported when the Senate is saying, “Yes, we believe in this” and it’s not just one person saying we don’t.

Regan: Absolutely. And they’ll be naysayers. We started off this conversation with “There are people out there who think it’s not good enough” and there are people out there but I’ve had conversations with such people on my campus where sharing some information, sharing things about how it’s done goes a long way towards changing minds.

John: In my department, it’s helped that I’ve been the chair of our search committee for a few decades now. We’ve generally hired people who are interested in this, but that’s not the case in all of our departments yet, but we’re hoping that’ll change. For those who have small classes or may not be interested in doing research in their own classes, one other option is meta-analysis. Could you talk a little bit about that?

Regan: So meta-analysis, where one study is taking a look at a lot of different studies, there is the mother of all meta analyses… is one that we should talk about because I think the interested person can run to it. John Hattie, now at the University of Melbourne, did a meta-analysis where actually he did a meta-meta-analysis; took 900 meta analyses and then synthesized the data from those 900 studies that had already synthesized data, and the reason I like talking about that is the sample size when you take all those 900 meta analyses is a quarter of a billion with a “b”; that’s a lot of data points, it’s a lot of students. And what’s neat about meta analyses is that instead of just being one study at one place it’s now multiple studies over multiple contexts, and if you can find an effect over multiple contexts, that’s really saying something because a lot of single studies are so geared into the local context of where that place is that if you run into a meta analysis, so even if anybody listening pulls up an educational journal or an SOTL journal and sees meta analysis in the title, I would spend more time reading that one because it’s gonna be more likely to generalize from that. So, I think it’s statistical and methodological advances now mean that there are more meta analyses around and more meta, meta analyses around as well.

Rebecca: As an advocate for the scholarship of teaching and learning, where do you hope the scholarship of teaching and learning goes in the next five years?

Regan: Honestly, I think it should be a part of every teacher’s repertoire. When I think about a model teacher, and it’s not just when I think about it—I’ve published on evidence-based college and university teaching and when my co-authors and I looked at all the evidence out there and what makes a successful university teacher… one of those components, and we found six… I mean, it wasn’t just student evaluations, no, it was your syllabi, it was your course design, but one big element was doing the scholarship of teaching and learning… and to answer your question, I think if in five years from now we can see it be part of teacher training to look at your class with that intentional systematic lens, I think that’s where the field needs to get to.

John: At the very least it would get people to start considering evidence-based teaching practices instead of just replicating whatever was done to them in graduate school.

Regan: Absolutely. People would be surprised at how much good SOTL there is out there, and I always like sending folks to the Kennesaw State Center for Teaching and Learning where they have a list of journals in SOTL in essentially every field. You will scroll through that list for ages and it is just mind-boggling to realize that, “Wow, SOTL has been going on for a very long time.” And Rebecca, you mentioned art and performance arts and theater and music… not as much, but even there there is a fair amount and I think it’s just a question of getting folks making those resources more available to individuals and that’s why whenever I interact with teaching and learning centers I have a short list of key resources to look at. And again, that’s on my SOTL link. But, even that small list is an eye-opener to most people who never knew this existed, and I think once they realize it’s there they will start seeing it everywhere and once you start doing it it really energizes you. For those of us who’ve been teaching for 20-plus years to look at our classes with that new eye of how can I change something, how can I make it better and then seeing the positive effects of those changes, that’s invigorating.

Rebecca: I’m energized after having this conversation.

Regan: It is good stuff.

Rebecca: Yeah.

Regan: I just got back from a three-day conference and all we did was sit around and talk about cool SOTL. And you’re right …came back and sitting on the plane I was texting people with study ideas to collaborate on. It was that exciting.

Rebecca: The more you talk… collaborate… the more it happens.

Regan: There you go.

Rebecca: So, we always wrap up by asking, what’s next?

Regan: You know, I think I like getting the bang for my buck and you mentioned this in the intro: right now I’m working on the American Psych Association’s Introductory Psychology Initiative and what’s next is basically two years of really focusing on the introductory psychology course. It’s taken by close to a 1.5 million students a year and I’d like to make sure we can make that course the best learning experience for our students as possible, so that’s where my energy is gonna be for the next little bit.

John: That’s a big task and a very useful one.

Rebecca: And definitely worthwhile. Well, thank you so much for spending some time with us this afternoon. it’s been eye-opening and exciting… energizing. I can’t wait to look through some of the resources.

Regan: You know, is there anything else that you’d like, get in touch and I welcome anybody listening to get in touch as well.

John: Thank you, and we’ll share links to the resources you mentioned in the show notes.

Regan: Sounds good.

[MUSIC]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.

[MUSIC]

49. Closing the performance gap

Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, Dr. Angela Bauer, the chair of the Biology Department at High Point University, joins us to discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

Show Notes

  • “Success for all Students: TOSS workshops” – Inside UW-Green Bay News (This includes a short video clip in which Dr. Bauer describes TOSS workshops)
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc.
  • Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative learning techniques: A handbook for college faculty. John Wiley & Sons.
  • Life Sciences Education
  • Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.
  • Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6), 613.
  • The Teaching Lab Podcast – Angela Bauer’s new podcast series. (Coming soon to iTunes and other podcast services)

Transcript

Coming Soon!