Category: peer instruction

70. Dynamic lecturing

Posted in active learning, attention, higher education, learning, lecturing, pedagogy, peer instruction, teaching
February 27, 2019

The lecture has dominated instructional practice for several centuries. In the last few decades, though, the lecture mode of instruction has often been criticized by advocates of active learning approaches. In this episode, Dr. Christine Harrington joins us to discuss evidence on the effectiveness of lectures and how we can create lectures that better support student learning. Christine is an associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing and several other books related to teaching, learning, and student success. Christine has been the Executive Director of the Student Success Center at the New Jersey Council of County Colleges.

Show Notes

  • Dr. Christine Harrington Associate Professor of Educational Leadership at New Jersey City University (NJCU) Previously served as Executive Director of the Center for Student Success at the New Jersey Council of County Colleges (NJCCC)
  • Todd Zakrajsek – Co-author of Dynamic Learning
  • Dr. Neil Bradbury – Professor of Physiology and Biophysics at Rosalind Franklin University of Science and Medicine
  • Bradbury, N. A. (2016). Attention span during lectures: 8 seconds, 10 minutes, or more? Advances in Physiology Education,40(4), 509-513.
  • Richard Mayer- Professor of Psychology and Multimedia Learning at University of California at Santa Barbara
  • Mayer, R. (2019). How Multimedia Can Improve Learning and Instruction. In J. Dunlosky & K. Rawson (Eds.), The Cambridge Handbook of Cognition and Education (Cambridge Handbooks in Psychology, pp. 460-479). Cambridge: Cambridge University Press. doi:10.1017/9781108235631.019
  • New Jersey City University Ed.D. in Community College Leadership program
  • Dr. Harrington’s book, Dynamic Lecturing can be purchased from Stylus Publishing and listeners can use promo code: “ETS20” (excellent teaching series) to receive a 20% discount on the Dynamic Lecturing book or Dr. Harrington’s other book, Designing a Motivational Syllabus

Tea For Teaching episodes referenced

Student Feedback tools

Transcript

John: The lecture has dominated instructional practice for several centuries. In the last few decades, though, the lecture mode of instruction has often been criticized by advocates of active learning approaches. In this episode, we examine evidence on the effectiveness of lectures and how we can create lectures that better support student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Christine Harrington, an associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing and several other books related to teaching, learning, and student success. Christine has been the Executive Director of the Student Success Center at the New Jersey Council of County Colleges. Welcome Christine.

Christine: Thanks.

John: Welcome.

Rebecca: Actually it should be “welcome back.”

Christine: Thank you for having me again. I’m looking forward to a new conversation.

John: Our teas today are:

Christine: I am not today.

Rebecca: I am drinking Lady Grey.

John: And I’m drinking ginger peach black tea today. We’ve invited you here this time to talk about one of your other books on dynamic lecturing. It’s not uncommon for people to argue that lecturing is ineffective, but it’s still one of the most common forms of instructional delivery. Why is lecturing so often discouraged?

Christine: This is an interesting question. I’m not really sure where this stems from but I think that the push for active learning started to pit the lecture against the active learning approaches, the collaboration. And I really do believe that there is significant value in both approaches and I’m not sure why it became an “either or” kind of situation. But unfortunately, it really has, and one of the reasons that I decided to write this book was because the lecturing is the most common method, you know, it’s still the tried and true method of faculty rely on all of the time. And yet, there are very few resources or support to help faculty be effective at lecturing. If you go to a professional development conference, a teaching learning conference, you’re hard pressed to find a session—unless I’m there, I guess – on lecturing. I actually haven’t seen another one yet—so there really are not any resources for faculty on this and it’s not really fair that it got such a bad reputation, because there’s not validity in that thinking.

Rebecca: So that leads to a good question. What is the research on effective lecturing?

Christine: So in terms of what is effective, I think one of the first questions that we need to ask ourselves is, “Well what are we talking about?” because much of the research that exists out there, if you’re going to research a boring, monotonous lecturer… who’s got the old yellow papers and just is reading and not even looking at the students and engaging them, or you’re talking about a lecture that is dynamic and the presenter is passionate and excited about the topic. We’re not always measuring the same concept. And that’s true in group work as well. So really, when we talk about teaching and learning practices, and we try to look at the literature about what works and what doesn’t, it’s very complicated because of the complexities associated with the teaching and learning processes. However, there is research out there that does support the lecture. What much of the research really points to is that the lecture is most effective for novice learners. For students who have very little background knowledge in the subject matter, they need to have someone who’s an expert present that information in a way that they can take it in so that they are developing that expertise and hopefully learning that content. If you ask them to just engage in what’s been called inquiry-based learning or case-based learning, the research really shows that that approach is not as effective if you don’t have the background knowledge. So what in essence happens is that well-intentioned faculty and teachers use that approach and end up wasting a lot of precious learning time, because the students in the groups aren’t equipped yet to be able to tackle those high-level questions and to figure it out without the guidance. There’s some interesting research out there on the importance of it being done well, but also making sure that the lecture is done before the group work is done, so that the foundational knowledge kind of sets the stage for some of those more what we call traditional active collaborative learning experiences. So if you want to look at novice learners, you’re going to see a strong correlation between the lecture working and student success outcomes. And then there’s something called the expertise reversal effect. What happens there is the more that you know about the subject matter, the less helpful the lecture is and the more helpful those more active collaborative learning group exercises are. I still today learn from TED Talk or a great lecture. But I’m also going to really get a lot of value—especially in my area of expertise—out of dialoguing with other experts and engaging those conversations because I have a strong foundational background. So it’s kind of interesting when we think about “Does the lecture work or does it not work?” it depends on who you’re talking about, in what subject, for what purpose, under what conditions. So it’s not as simple as a yes or no, but I will tell you that there is a significant body of research that says the lecture is effective. It’s not that we need to throw the baby out with the bathwater, we just need to make sure we’re using that method effectively.

Rebecca: One thing that you hinted at, Christine, is that with lecture from an expert, there seems to be an opportunity to ask questions, which is why just reading about stuff maybe isn’t always sufficient. If you have that opportunity to ask questions of the lecturer or am I reading into something that I shouldn’t be reading into?

Christine: No, I think you’re right on track there. And actually, I think that’s part of the reason why live lectures are more effective than online lectures… because in a live lecture, you as the expert get to see the puzzled looks on the faces. So even if they’re not offering up the questions, you can say, “I think I need to throw in another example.” And students can ask for clarification along the way, so that you’re not just going on without them following and getting the concepts that you’re discussing. Lectures are not one-way kinds of teaching methods, but they really are two-way processes. Even though it seems heavier on the expert delivering to the student, the student can be very engaged and also asked to be engaged through questioning and other activities during the lecture.

John: And it’s not necessarily an “either-or” condition… that you can embed active learning activities in the lecture, right?

Christine: And that’s exactly how I would define a dynamic lecture: that you would incorporate what I’ll call brief active learning breaks into the lecture because we don’t want to just talk at students for a really long time… and let’s face it, as faculty, we could do that. And quite honestly, it’s the easiest thing to do, especially when you know your subject really well. And you can just walk in and start talking about your subject matter. But that’s not necessarily going to lead to the highest levels of learning. So when we look at what our learning outcomes are for the course, we need to structure our course in a way that’s going to help students learn and achieve those learning outcomes. So in order to do that, it’s really powerful to build in some brief reflective opportunities for students to engage with the content. And Mayer has really done a lot of work in this space, saying that it’s a cognitive focus that needs to be emphasized more so than an interactive focus. So the breaks don’t always need to be a social or group break. It doesn’t have to be a partner activity or a small group activity. Some of them can be independent activities. The key is that once you get a certain amount of information, you need to process that information and digest it and there are a variety of techniques that you can use to help students really learn that content. One of my favorite studies—which is a little bit disturbing, and in fact, I always say it hurts our feelings when we find out what it actually says—there’s a research study that compares students who had no pause in their lecture. So that the professor just kept talking the entire time sharing all the expertise in an effective strategy and an effective way. And then another condition where the instructor paused three times for two minutes each. So we’re talking about a total of six minutes during the class period that the professor stopped talking and during that time the research study was set up such that the students had to do what’s called a “compare and share” of their notes. They had two minutes to take a look at what their partner wrote down, fill in any gaps that they had, and engage in that. And at the end of the day, what they found out was that the students in the pause condition really outperform significantly students in the no-pause condition and it sounds exciting at first until you realize that “You mean, If I stopped talking for six minutes my students learn more?” like that kind of hurts their feelings. [LAUGHTER] But it’s true. Sometimes especially at the end of a class, we start talking faster and faster and trying to give more information as if that’s going to lead to high levels of learning. We need to keep in mind that students need time to digest and process. And it’s really important that we strategically and intentionally build in those opportunities for our students during our lecture.

John: And as they’re building their own mental models, just giving them a little time to process it and to compare notes takes advantage of peer instruction. You’ve got some reflection going on there, and you’ve got a little bit of retrieval going on there. So there’s a lot of evidence-based strategies that are embedded in that basic activity.

Rebecca: I think it also helps students who might start feeling panicked because they didn’t get everything in their notes… give them a second to maybe fill it in and then they don’t feel so panicked and they can focus again. If you get anxious because you feel like you’re behind, it’s really hard to focus.

Christine: Absolutely, and it is human nature for us. Our attention wanders, no matter how wonderful a lecture is, life is happening to you. Sometimes it’s easy to have mind wandering happen and you don’t want students to be penalized for that happening for a brief moment. So giving them opportunities to get back on track and refocus I think is really important. And John, as you mentioned before, there are several of these great learning breaks that you can use that are very, very much grounded in the research. You mentioned retrieval practice, for instance and we all know about the testing effect and how powerful that is. And I think we focus primarily on taking tests and encouraging quizzing—and that’s definitely an important component of what we should be doing in the way that we structure our classes so that students get to benefit from the testing effect—but quite honestly, we don’t have to grade everything and not everything needs to be called a quiz or a test. But if you ask students to do the classic, one-minute paper, for instance, that really is retrieving the content that they just learned and giving them practice at doing that will make it more likely that they transfer those actions into their world outside of the classroom, so that when they’re studying, they also engage in those same kinds of evidence-based practices.

Rebecca: I think a lot of times when you hear good lecturing, people think about TED Talks and maybe some of the storytelling and things that happen or the visual strategies that are used in those talks. Are there elements of those that come into strong dynamic lecturing in the classroom? Are there things that are missing from those that we should be thinking about in our own classroom?

Christine: I think you talked about something that’s really important: Storytelling. For ages and ages storytelling has been a way that we have learned and I think we have all been on the edge of our seats in a lecture that was based on storytelling. We want to know what’s going to happen next. And the lecture really can become the story of our discipline. And we can weave in personal stories and examples and things to make the content come alive for our students. It really puts it into context for them and helps them identify and see the relevance of the material that’s being discussed in their real world application. So to me, I think storytelling is probably one of the reasons why lecturing is so effective if it’s done well and you are weaving that in and mesmerizing your students with the chapter content. The the key element is doing that effectively. So I would say, “Absolutely, that is great.” Although TED Talks, when you think about them, obviously they’re online and they’re one directional still. So stories can be more just told by the storyteller and not have audience participation or they’re stories where you think back to your days in elementary school, where the teacher would pause and ask for you to get engaged in the story and maybe predict what would happen next, and to think about examples from your own world. And I think that’s what we can do in the live lecture, sitting there with students face to face we can give them those opportunities to do a prediction. “What do you think this research study is going to find? What is the key finding going to be? You heard how the study was set up, what are the implications of that?” You’re getting them to think about it and to be really engaged in the story and participants in the story, I think is one of the areas where we can as faculty enhance the effectiveness of the lecture.

Rebecca: How does a faculty member learn to be a better storyteller?

Christine: I think that some of that’s natural, I think some of us are more naturally better storytellers than others. But one of the strategies that I suggest to faculty is for you as the expert to take a step back and to think about what are the key elements of your story? Or what are your big ideas of your lecture? Because it’s all natural, and it all flows to you as an expert, because you know, this material so well. But for your students who are getting exposed to it for the first time—or maybe on the second or third time hearing this content—they don’t know what the big ideas are. So I think that one of the strategies that faculty can use to become better storytellers is to almost map out what are the many chapters in this book that I’m telling, right? Who are the characters and the main players? What are the big theorists that we’re going to talk about? Or, what are the big researchers that we’re going to discuss? And what are the key variables—or the factors really—that are going to comprise this story? I think one of the most helpful things you can do as a faculty member to strengthen your lecture, is to step back and identify what are the three big ideas or major elements of this lecture for today. And if you’re able to do that, and then clearly communicate those to the students in the beginning and throughout every time that those big ideas are getting introduced, that will really help students hone in on the most important elements versus getting lost in some of the details of the story.

John: That’s one of the ways it helps reduce the cognitive load of the students so they can focus on those key points, without getting lost in the details that they’re not quite ready to incorporate into their models. What do we know about student attention and how we can keep student attention during a lecture? I know sometimes when I have a large class of 3 to 400 some odd students, sometimes their attention will wander. What can be done to try to keep that, a more constant level of attention and focus?

Christine: Well I don’t know what you’re doing, John, I don’t have that problem. [LAUGHTER] No, only kidding. [LAUGHTER] Attention during a lecture as an interesting topic. You know, I’ve been going around the country doing a lot of presentations on dynamic lecturing. And as you know, my colleague and co author, Todd Zakrajsek speaks on this topic quite a bit. He said, “Christine, try this out when you go present. I want you to ask the audience how long a student can pay attention during the lecture.” He said, “Just throw the question out there and see what they say.” And I have done this and he has done this, you know, we’ve compared and shared notes of ourselves and immediately people are throwing out numbers. It doesn’t take very long at all, the numbers usually start out like and hover in that 15 to 20 minute range —although I get, you know, some wise guys in the audience a 90 seconds and some others who are more optimistic, saying larger amounts — but immediately they’re throwing numbers out. So I said to Todd when I was writing the book, “I don’t know what’s wrong with me, but I’m usually pretty good at finding the research. I can’t find the research on this topic.” I said, “I heard this over and over. I’ve been to professional conferences where the presenter has said 15 to 20 minutes is all that folks can maintain their attention for, so you’re going to have to change things up if you want them to stay attentive and I couldn’t find it.” And we had this really deep conversation about it. And he says, “Now go and ask those audiences, how long would a student be able to pay attention if they were reading” and immediately you hear silence for a minute, because they’re processing and they’re trying to decide how long it would be for reading. So you don’t get answers right away. What almost always happens is somebody says, “It depends” pretty quickly. Or if they do say any numbers — and they do sometimes — they’re usually saying it in a much more tentative voice, lower, not as loud and enthusiastically, and not with confidence. Todd and I will say to our audiences is, “Why is that? Well doesn’t it depend for both the lecture and the reading?” It depends a lot on whether I liked the book, if it’s written in a way that I can understand it, am I able to take it in — if I’m reading a really dense, heavy textbook in a subject I don’t know and I don’t understand what I’m reading — I’m going to be done with that in a couple of seconds. But if I’m reading something that I really enjoy, and I have some background knowledge on and I’m able to take in and I care about, then I’m going to read for hours at end. I mean, you could be on the beach reading all day long, right? The same is true in a lecture. There really is no magical number about how many minutes a student can pay attention to. It depends on how much they care and their personal variables as much as the professor variables. Obviously, the attention span would be longer in an interactive, engaging presentation versus a monotonous boring presentation. The folks who talk about the 15 to 20 minute mark, it’s really not based on the research. However, there is some mind-wandering research that does say that students report higher levels of mind wandering in the second half a class as compared to the first half of class. It seems that there is a small drop around that 20 minute mark, that may have been where it came from. But, I have to be honest with you, some areas I can tell you those robust data or we have like hundreds of studies — this is a handful of studies, we don’t have an enormous body of research in this space — but the good news is, is that it started to get people thinking about: “I guess, if we need to keep students attentive, then we need to switch things up.” And I don’t think that was bad practice. Despite it not being founded on good research, I think that the result was probably positive if faculty were in fact incorporating some active learning break that could be advantageous for student learning.

John: Actually, this is a topic we talked about in an earlier podcast. In Episode 16, one of the people we interviewed, Neil Bradbury, had written a paper on attention span during lecture because he was faced with the same thing. He kept being told that you should keep your lectures or videos to 10 to 15 minutes or so at the most. And so he went to try to find the research and he published this in the Advances in Physiological Education a couple years ago. He found that it was based on a study that was really just looking at note taking and it was done in one class and it was based on an analysis of students’ notes and how much they wrote during different periods of the lecture, which had very little relationship necessarily to the importance of what was being discussed and so forth. And that became cited over and over again in other studies, and then people just started repeating it without ever seeming to go back and analyze that. And I think there was another podcast where someone had looked at attention on videos, he was looking at how effective videos of different links were on student learning — it was in chemistry, I believe. There really isn’t much research on a student attention span. And that obviously will vary, as you said, with the quality of the presentation. The students are willing to spend hours watching movies, we don’t see them walking out or starting to chat with their friends 10 or 15 minutes in, normally.

Christine: Right. And I think that some of the online video research that people rely on is sometimes is for non-educational purposes, so they’re looking at the attention span of someone watching a video from a marketing perspective. But when there’s no grade attached, we’re in a different situation. Hopefully we’re with an audience that has some motivation, they’re in the class, they’re in college, so their motivational level, I think, is very different than a consumer. I think it’s problematic that we’re trying to bring all of this really heavy, deep content into these like very brief news clips. But don’t think that this is the way that students are going to learn best. I remember I was working with someone and they were convinced they had to be two minutes or less. I’m like, “What can I accomplish in two minutes or less?” I mean, I can give you a quick news flash, but if you want to have deep learning, we’re going to have to have a deeper conversation. And I suspect you’re going to want additional examples from me, and you’re going to want me to share the relevance and that will actually help you. So, I think online videos are not as engaging as in-person videos. We probably do need to have them maybe in shorter chunks, but the key is is trying to bring their attention back and your initial question was, “How do you maintain their attention throughout?” And one of the strategies I think that we’re all very aware of is that we need a hook or something at the beginning of a presentation or beginning of a class. But we don’t really think about the hook throughout the class. So I advocate for faculty to go back to those three big ideas that I asked them to identify and identify a hook or an attention grabber before you introduce each big idea. It can vary, I get kind of silly in my classes sometimes, and we’ll use hand gestures and things of that nature. But for the faculty member who’s not comfortable doing that, you can do something as simple as saying, “Here’s big idea number two coming your way,” right? Because it doesn’t have to be that complicated. So I think the idea is that our lectures just like textbooks are filled with more important content and less important content and when we’re talking about the more important content, the chapters bring attention to that with bold headings and things of that nature. What are we doing in our lecture to help them see? Where is the bold heading of our lecture? Do they get to see those subheadings? Can they figure that out or are we in a little rabbit hole of detail somewhere… that they don’t necessarily even need all of that information?

John: In one of our earlier podcasts, Alex Butler was using an example where there were certain key big ideas in his class and he used images on that. And he put those images on whenever there was an application of those big ideas. And that sounded like a really nice application continued over the whole semester, not just within a single lecture, even.

Rebecca: I couldn’t help but hear as you were talking, “accessibility, accessibility, accessibility.” because structured content is one of the biggest themes of accessibility — or one of the biggest principles to make things accessible digitally — but you’re talking about the same exact concept in a lecture. What is the skeleton or the outline of what we’re talking about so that people can kind of fill in the blanks? And sometimes you have to make that obvious to someone who doesn’t have the expertise that you have because it’s obvious to you what those are.

John: To develop the scaffolding that they need to make sense of it all to fit it all together.

Rebecca: And an outline is like a scaffolding.

Christine: That’s excellent. Yeah, I love it. That’s an excellent point. That’s terrific.

Rebecca: I was wondering, Christine, if you could talk us through one of your lectures, one of your classes. What does it look like? What does it feel like?

Christine: Sure. So I begin class with an activity called “dusting off the cobweb.” So the first thing that I do is, they know as soon as we walk into the class together, they got to put their books away, their notebooks away, and they need to just rely on their brains to engage in practice retrieval. And the question on the table is, What did we talk about last class? So they have a minute and a half to begin that exercise. And they’re talking with a partner and they’re trying to remember what they recall from last class. After about a minute and a half or so then I have them open up their books and their notebooks and fill in any gaps. So they’re going to continue to talk about What did we talk about? And at this point you hear, “Oh, my God, I can’t believe I forgot about that” and I’m sitting out there thinking, “Yeah, I can’t believe you forgot about it, but I’m glad you’re remembering now.” So about three minutes or so passes, and then what I do is I’ll randomly call on students. I’m not a fan of randomly calling on students if you don’t give them processing time… talking about accessibility issues and students with disabilities and processing information in a different way. But after I gave you three minutes to do what I call a fairly low-stakes engagement, I think you’re all fair game. So I’m going to call on you and you’re going to remind us of what we talked about from last class. So now we spent about another five minutes or so recapping some of the highlights. If they don’t mention something that I think is particularly important for us to refresh on, especially given the new content that we’re going to do, then we do that. So I always begin with this because what that does is it activates their prior knowledge so that they’re ready to take in the new information and the new information will be easier to learn because it can stick onto that previous knowledge that they just learned from the last class. So that takes about, you know, five or seven minutes or so. The next thing I usually do in my lectures, I shift to a reading assignment so they usually have some kind of reading assignment — might be a journal article, might be the chapter — and they have questions that they need to answer. This is another activate their prior knowledge kind of activity and holding them accountable for the reading and learning outside of class so that I’m not having to spoon feed everything to them. So now we spend another 10 or 15 minutes where they go over those questions, and this will be with a different partner, and I actually go around the classroom and do spot checking of their assignment while they’re engaged in those activities and I grade them — they’re, you know, low stakes kind of grades, but I grade them nonetheless. So it keeps them on track and it really keeps them focused and interesting that you mentioned that the visual image before because sometimes I’m asking them to do their reading assignments in picture format, like I want them to use either SmartArt or graphics or images because otherwise I find that they’re trying to like copy content that they don’t get. So I’m trying to get them to digest the content as best they can when we’re working together. So then when we’re done with that, now as I spot check, I can see which topics seem to be the ones that they got pretty well. And I still will go over them, but in a much briefer way versus the topics that I could see that they might have been struggling with and they had questions like, sometimes I’ll be walking around, it will be like, “Dr. Harrington, I really didn’t understand this question” and that might be a theme. So then that’s the one that I’m spending a little bit more time on. It’s kind of a very modified version of the intro teaching. It’s not quite exactly following that model but I’m using that reading activity to kind of guide the lecturing and then during the lecture, I will identify what my three big ideas are and I usually do some kind of gesture. So I’ll introduce it with like a gesture about whatever the topic is, get them really excited. I get a little loud and excited, “It’s big idea number one time!” and then I go into the content and then I always have a practice opportunity after it. So after every big idea — and that varies, it might be as simple as a turn and talk or one-minute summary. Sometimes I put them into smaller groups to do like a case study or to develop and answer Socratic questions related to the content. So that piece will vary depending on the nature of what it is that I’m talking about — and I’ll repeat that through the next two big ideas. And then at the end of class I usually will do a very quick — it might be only five minutes — like a preview of the next chapter to make it a little bit easier for them to read. So I’ll highlight stuff, I know that a content’s coming up, let’s say it’s the learning chapter in psychology. I know classical conditioning is often challenging for them to wrap their hands around, I might give them an example and expose them at least to some of the vocabulary and the language that’s going to be in the chapter so that they’re all set and ready to roll with the reading assignment for the next time. That’s kind of what a typical lecture would look like for me.

Rebecca: Thanks. I think a lot of times we often hear some best practices but don’t really take time to think about how that actually plays out throughout an entire class period.

Christine: Mmm-hmm. In the back of the book, you’ll see that there are lots of forms that I have created for faculty and one of them is the sequencing. So, on one of the forms, I’m asking them to plan by identifying: what are the big ideas? How are they going to draw attention to those? What examples are you going to give? What kind of active learning break are you going to give? And then there’s another document that helps them sequence the activity: to always begin with some kind of introductory activity, and then going through those three big ideas and at the end, some kind of concluding activity to get them set for the next learning adventure.

John: Those worksheets and forms that you provide at the end of the book are superb, and that alone is a good reason to buy the book, in addition to all the other wonderful content included in it.

Christine: Thank you.

John: For those people who use multimedia in presentations… who use PowerPoint or visual imagery or perhaps videos, do you have any recommendations on how multimedia could be used or how presentations can be designed to more effectively maintain student attention?

Christine: Absolutely. I think that this is another area that’s gotten the baby thrown out with the bathwater. You see all these sessions, Death by PowerPoint… that PowerPoints are overdone, and sometimes people are rolling their eyes if you’re going to use a PowerPoint. Well again, the PowerPoint is an incredibly effective tool, if it’s done well. If it’s done poorly, it’s an incredibly ineffective tool. We need to make sure we’re using evidence based practices for creating the multimedia, whether it’s slides, videos, whatever tool it is that you’re using. And this is an area that has a robust amount of literature. Mayer has done, I think hundreds of experimental studies on what works best with multimedia presentations and has really found that adding visual images really enhances learning. In psychology, we have a concept called the picture superiority effect, where our memory for pictures is stronger than our memory for word. Something I didn’t mention before in terms of my presentation, my lecture, I always have the PowerPoint as my visual backdrop. It’s not our textbook on slides, it’s really a visual story. It’s kind of like the picture to the storybook. That’s what I view the multimedia presentations to be. So if you look at his research, basically you should have one big giant image and maybe a couple of words associated with that image, and that would be the best PowerPoint slide. And I joke with a lot of my faculty colleagues and I’m like, “Look, I know that I’ve had those slides with so many bullets.” In fact, I’ve heard professional say you’re only to have so many bullets and so many words. Well again, I can’t find any research on any of that. No, stop the bullets, stop the words, go with one big image and just a couple of words. But I said to my faculty colleagues, “I know I too have had slides that had too many words on them. And I’ll tell you exactly why that was the case. It was because I was just starting to teach that class and I didn’t want to forget something. So it was a tool for me, it was not a tool for them. What I learned to do is to have two separate tools, I could still have my additional notes if I didn’t want to forget something. But it is not a visual aid to put it there because it’s not helping them, it’s hurting them.” So to create a powerful presentation really means you need to think clearly about what images are best going to communicate your content, and then to put that up on the board as a visual backdrop, and then take any notes that you need to put it aside and it can be helpful to share those notes. Again, going back to the accessibility issue, if you have notes, why not share them with students? It’s a good idea to share them, but they’re not visual aid for your lecture. Because students can’t do what Mayer called, the redundancy principle. But I actually like to call it the be quiet or shut up principle. We can’t listen and read at the same time. So if you have a slide that has a lot of words, you have students saying, “Hmm, should I be reading this? or should I be listening to what that person is saying? I can’t do both at them same time.” And what usually happens is nothing, so you don’t get anything out of it. If you do have to use a lot of words on the slide, which I think would be a very rare occasion, then you should shut up and let them read it or read it together — I don’t think it matters one way or the other. I’m not familiar with any research that points you in one direction — and then describe it, but don’t talk over your slides. That really is problematic for learning. It’s not even that it’s not helpful, it’s actually harmful to learning.

John: In presentations, instructors will often use some technology to get feedback from the students. What are some effective ways of getting feedback from all of your students or for many of your students, during a presentation?

Christine: Well, I think that the technology tools available today really allow us to engage our students in a new way. So whether you use something like a Poll Everywhere, or a Kahoot! tool or clickers, or asking them to engage via Twitter, there’s so many tools out there that can get students engaged. And I think especially with large classrooms, if you’re trying to lecture, sometimes you don’t know whether they understood the concept that you described. And even asking a quiz question about it, and having them answer it via technology can show you and them whether or not everybody’s kind of on the right track. Now, of course, whenever you introduce technology, you also introduce the possibility of increased distraction. So you have to be mindful. I think that you want to be careful about it. And I think students often respond well to faculty when they believe that you really care about them and are interested in them being successful. So by sharing the rationale and structuring it before you begin to use those tools — and you might even make a joke about it and say, “Look, I’m going to be having you pull out your cell phones but you’re going to also have to put them back, you know, so we’re going to be like on cue here, it’s out, in, you know, like bring them into the class, put them out of the class.” — So I think it is important for us to recognize that the temptation to be distracted is going to be high once we start using their cell phones, because there could be a message on there that they all of a sudden pulls them into a different direction. But the value of getting everyone engaged is powerful and some faculty will even count those to increase their accountability throughout the class and to keep them more focused on the questions if they do count in some way. But I think you have to be careful about that because some students may need more time on task to learn that content. So if I didn’t get it in five minutes, I don’t think I should be penalized. So, it is a tricky process and I think you just need to know your students and what works best in your classroom. But there’s so many great tools out there that really allow faculty to engage students throughout, and engage them in the practice retrieval actions as well.

Rebecca: So one of the things that faculty always want to know is whether or not they’re doing a good job. So how would you recommend faculty get useful feedback about the quality of their lectures and maybe tips for improvement?

Christine: Yeah, I think this is a really important point because I have heard about some faculty actually getting poor evaluations just because they use the lecture. So I think that we have to do a variety of things. First of all, we need to, A: educate our peers, and obviously our chairs and deans and whatnot, about the value of the lecture. And then we need to then figure out well, how can we best evaluate whether or not a faculty member is doing an effective lecture or not? In the back of my book, I also do have a chapter on evaluating lectures and I think that the listeners would probably find it valuable to go through that to see. Because what I basically have done is, taken all of the research-based practices and turn them into kind of self-evaluation or peer-evaluation questions. You can also engage in self assessment as well as the peer assessments, I think both are critical to have you really think about. I think it’s essential that we talk to the person who’s observing us ahead of time so they have context for what’s happening. So to have kind of a pre-meeting that really describes what’s happened and transpired in the class before this isolated lecture that you’re coming into, so that they know any of the story about why it is you might be doing what you’re doing. And also to ask them specifically: “You know, I’m trying out this new brief active learning break, we haven’t done this one before. Maybe we’re going to try asking one another Socratic questions, could you help me be another pair of eyes to see, were people engaged in this? Were they struggling? Were the instructions clear? Were they taking the ball and running with it right away or where they fumbling and not really knowing what to do immediately and needed more guidance?” I think that there are so many strategies, it’s not like there’s a wrong or a right way to do it exactly. I think that the key is, are you integrating the research based practices in a way that supports the learning goals of your class? So really keeping hyper focused on the learning outcome that you’re trying to accomplish that day.

John: So we always end with a question: What’s next?

Christine: Well, I mentioned to you before that I just took on a new position, so I’m now at New Jersey City University. I am a faculty member and also co-coordinator in our brand new (Ed. D.) Community College Leadership program. So as part of this, I mean my primary focus is going to be obviously the launching of this new program, some of this includes curriculum development, and marketing, and recruitment. And I’m really excited to get this program off the ground because we really need to build the leadership capacity in the community college sector, at all levels. So very excited about that. But I do have a new book that I’m going to be writing, I actually just got the contract last week, it’s going to be about the guided pathways movement, which is all focused on increasing student success completion rates. And it’s primarily a community college initiative, although, many of the four year colleges also have been getting into the space, and the book is going to be focused on engaging both full- and part-time faculty in Guided Pathways Leadership. So to get them engaged in this movement, and to really see themselves as leaders in the student success reform efforts… so really excited about that and I have some other potential things in the mix to that hopefully will pan out as well. But lots of great stuff happening for me, so I really appreciate this opportunity and look forward to staying connected with both of you.

John: We really appreciate you joining us.

Rebecca: Yeah, we always have a lot of fun and walk away thinking about a lot of things we can start doing to improve our own classrooms.

John: And you can find Christine’s books at Stylus Publishing, and I believe there’s a discount code available for our listeners.

Christine: Yes, if they put in ETS — which stands for excellent teaching series — 20, they will get a 20% discount on the Dynamic Lecturing book but also the Designing a Motivational Syllabus, and I believe it’s going to work for all the books in the series that will eventually come out. So it will be a 10 book series once all of the books are out and published, but ETS20 would be that discount code.

Rebecca: Well, thanks for joining us. We always have a great time.

Christine: Well, thank you. Always my pleasure, I appreciate it.

John: Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, and Jacob Alverson.

66. Just-in-Time Textbook

Posted in higher education, OER, open pedagogy, pedagogy, peer instruction, reflective practice, scholarly teaching, teaching, writing
January 30, 2019

What would you do if you are scheduled to teach a class of 75 students and discover that several very expensive textbooks would be required to address the full range of course topics?  In this episode, Dr. Jessica Kruger rejoins us to discuss how she responded to this challenge by working with her students to  create their own textbook. 

Jessica is a Clinical Assistant Professor in the Department of Community Health and Health Behavior at the University at Buffalo.

Show Notes

Transcript

John: What would you do if you are scheduled to teach a class of 75 students and discover that several very expensive textbooks would be required to address the full range of course topics? In this episode, we talk with someone who responded to this challenge by having her students write their own textbook as they progressed through the course.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Jessica Kruger, a Clinical Assistant Professor in the Department of Community health and health behavior at the University at Buffalo. Welcome back, Jessica.

Jessica: Thank you. Happy to be back.

Rebecca: Today our teas are:

Jessica: Not today, but maybe a little bit later to relax, thinking about all of this stuff I need to do before the start of the semester. [LAUGHTER]

John: …and I am drinking Rose Garden Black Tea that you brought back from…

Rebecca: …Epcot. So I’m drinking the same thing. We’re having a tea trial this afternoon.

John: It’s one of the blends that they only sell at the Twining store in Epcot I believe…

Rebecca: …Yeah, and it’s a nice counterbalance to the lovely weather we have outside.

John: We have invited you here to discuss the open pedagogy project that you ran last semester in a class with 75 students. Once again, scaling things to a higher level than people normally do it. Could you tell us a little bit about that class?

Jessica: This class is a 300 level public health course. It’s titled Methods and Mechanisms in Public Health. It covers three main topics, so we start out with environmental health and then we move into information about health theories and health behavior theories and then we end with disparities. And so with this class, there was not one single book that would encapsulate all the topics. Instead of having the students buy three or more different books, I started to think, well, what else could I do?

John: The natural thing is to have them write their own book…

Jessica: Of course, why not?! I think the cost of textbooks are continuing to rise, and especially with this sort of course that specialized with these three different areas. I don’t think I would ever find a publisher that would make something quite like that. So why not write your own?

Rebecca: How did you pick the topics that were included and how did you get going on this project?

Jessica: So I actually heard about open pedagogy and writing a textbook with your students at the CIT conference that happened last year, and I was really inspired by Robin de Rosa and what she had done. And so immediately after hearing from her, I thought, “I bet I could do this.” At that time, I don’t know if I was deranged because it was the end of the semester, or maybe it was a stroke of brilliance. Probably a mix of the two. But I thought, let’s figure this out let’s see how to do this. And so, as I was looking over the syllabus and the topics, I started flipping through their textbooks, I started looking at other resources that were available and began to put the topics together and I broke it down by the weeks of the syllabus. So my students were actually writing the textbook before they learned the content. Which for them, was very scary and for me, a little bit scary too. But the great part is they actually had to go out they had to find resources, they had to put it together. And as I was building this, I created Google documents and made skeleton outlines for the chapters and that’s how they kind of got started.

John: So you created the skeletal outlines and then they fleshed it out?

Jessica: Yes, literally. The outlines were a title of the chapter, some objectives, and headings, different headings of sections. I attach some information in each Google Doc, some resources that were out by the CDC or other peer reviewed sources that I thought could be helpful. And of course, I invited them to come meet with me, especially if they have no clue what I was talking about with this topic.

John: …And within each week, did you have a subset of students work on it? Or did you divide it up among all the students? How did you arrange that?

Jessica: So in this class, my very small class of 75, which is actually my smallest class this past semester, I broke them up into groups of four to five students. They didn’t get to choose their groups, but they did get to choose their topics. As they went through and looked at it, we broke it down and said, “Okay, this group decide what is your top picks,” and they could choose. And they knew the order of the chapters, so if they wanted to get it done out of the way, they could do it at the beginning of the semester or wait till the end. And so each group worked together to make a contract, divide up the work and choose how they’re going to execute this.

Rebecca: So what worked well about that method and what didn’t work well about that method?

Jessica: Oh, students love group work. We all know that they love group work, right? If I could figure out the secret to making it go smoothly all the time, I guess I would probably be a millionaire. But nevertheless, I think there was some strategies that did work well. And the fact was, I had the maker group contract. Barbara Oakley actually has a article called “How to work with a couch potato,” and in that article it talks about how to deal with someone who’s not pulling their weight and how to create a group contract that’s actually useful. And so the students worked together with their group and talked about how they’re going to evaluate each other. I didn’t set a peer evaluation, they did, and they also broke up what they’re going to do. So as they’re creating this chapter they would all right in a different colored text. So one student was green, one student was orange, and so they can see visually what each person did. And my caveat was, if someone drops the ball, they drop the ball, you don’t have to make up their work, as long as it’s stated in the group contract, that is fine. If they don’t do the summary they don’t do this summary, your chapter doesn’t have a summary. That’s okay.

John: How did that work? Did everything get completed?

Jessica: Actually, we only had one chapter I believe that doesn’t have a summary and most everyone did their part. There’s always some squabbles back and forth. There were some groups who did really good strategizing and had someone go back through and create one voice for it, other groups didn’t. But overall it worked out quite well breaking it up that way, but also giving them that out, realizing that they didn’t have to make up something that someone else didn’t do.

Rebecca: So how did you handle chapters that weren’t fully complete and you needed the other students to read those chapters?

Jessica: I had a wonderful TA last semester, and the chapters would be due about a week before they needed to be put on to UBLearns. She would alert me to anything that was going on or anything that should be changed and I would look over the chapters. And then I would bring in some other content or modify it. But overall they actually did a really great job of putting this together and finding sources…

John: …you mentioned UBLearns…

Jessica: …UBLearns is actually our Blackboard learning management system. So they would offer it in Google Docs, and then I would take that and create it so that they could comment on it, but not actually edit it. And this allowed them to get feedback from their peers, which we plan to preserve the feedback for the next time is classes taught, so this book is a living continuing iteration.

Rebecca: So as the students were reading the chapters as assigned reading, is that when they were providing the comments and the feedback?

Jessica:Yes, they would go through and we had some very astute grammar students that would go and pick commas, and also asked for more explanation, which was excellent. I could actually use that in my teaching to talk further about areas where I could tell that students needed more help.

John: Did the original writers go back and fill in some of the gaps at that point?

Jessica: Sometimes they did, but for the most part, we left it as is. We used version one and the plan is for the students who take this class next to take those comments, continue adding, continue changing, and revamp the book. So there’ll be multiple versions for each semester that it’s taught.

Rebecca: Can you talk a little bit about how you handled copyright and giving credit where credit is due for each of these chapters and how you might handle that in future iterations?

Jessica: Overall the class work together to figure out their creative commons license, and what they chose was a CC BY-NC-SA. Which means, people must attribute and give credit, it’s non-commercial and it’s share alike. We talked about this as a group, we learned about copyright and they all actually signed a contract as if they were publishers, as they are. We discussed how to attribute content, some chapters did it one way some chapters did it another way. So you’ll see in tech citations in some areas and and others you’ll see, “this was modified from the CDC website at this location.” And so some of it was actually openly sourced information that was reused. Now, this is their first time writing a textbook, so can I stand behind that everything is completely cited perfectly? Absolutely not. But they did a good faith effort to make sure that their information was cited properly.

John: …and there is a little note at the bottom of each page listing sources and saying that, to the best of our knowledge, this is not subject to copyright. If you find anything that appears to violate it, please notify us. So there is a procedure for addressing that stated on each page, I believe?

Jessica: Correct.

Rebecca: Can you talk a little bit about how the groups decided to evaluate each other?

Jessica: Yeah, I think group evaluation and teamwork is such a challenge. I myself, despised group work as a student, and as a faculty member sometimes I’m like, “Ah, I know they need to work with others, but…” But really having them create their own contracts and having them evaluate each other on their own terms, so some groups decided that “we will do everything perfect and we will come to every meeting.” No, that’s not a group contract. That’s not something that you can actually achieve. But what you can achieve is open communication, and so most of the groups actually used GroupMe or other tech tools to stay in contact, they would set up meetings sometimes before after class. They would also work on the Google Doc in tandem, you can actually see when someone else is working on a Google doc and point out different sources to each other and discuss how they’re going to put it together. And so that allowed them to evaluate each other and most students gave each other pretty high marks in peer review. And I don’t know if that’s because they all like each other and they’ve all been in multiple classes together, or if it’s actually how they feel that this came together. But overall, it happened, it got done and so I do think they work together pretty well. Some groups obviously better than others depending on their strategy, but having them create the contract I think is the important aspect that I found

John: And you were able to monitor that by seeing the colors of the contributions?

Jessica: Yes. So I had them color code their contributions and if they didn’t color code because someone was an editor or whatnot, they would note that so they knew what each person was doing. And then before I would send it out to the class, I would take out all those colors and do some type editing. But overall, each of them were contributing and most of the students in the class are juniors and seniors, so they’re upper level students.

Rebecca: Did you have students evaluate each other at the end of their contribution, or at the end of the semester?

Jessica: It was at the end of their contribution, and it’s fascinating because I asked them during that process, not only to evaluate themselves, but to evaluate this project. And it’s fascinating because as I’m going through it, I’m looking at it and asking them, “Do you think other classes should do something like this?” And about 50% said no, at that time. I was like, “Oh my goodness. Is this going to fail? What’s happening? Why do they despise this so much.?” But, at the end when we did an overall evaluation, it was actually overwhelmingly positive, it was all positive about the whole project. So it just goes to show you that as you’re in the thick of something, you may see it as challenging, overwhelming, but in the end, when you see that final product, when you see that 19th chapter for 200 page book that you’ve created with your fellow students, that’s powerful.

Rebecca: You ended up getting your book printed and copies distributed to each of your contributing students. How did you pull that off during the semester you were writing it?

Jessica: Very carefully. I worked very closely with my TA and I spent countless hours editing and making sure pagination was correct. I think in the end I probably put in a whole work week just getting that together and working with the university printing service. I sent it in, it was excellent. I’ve never printed a book before not knowing how to do anything like that, and thankfully our university had some funding to allow me to print books for all of the students. And to see their faces when they got that book, was just outstanding. I actually created a celebration and so I invited the Director of the program, the Dean, the Chairs, and even people from OER Services at SUNY. And the students walked into a cake and people clapping for them, and then I reveal the printed book.

Rebecca: So you had to have had the book finished multiple weeks before the end of the semester to make that happen?

Jessica: Well, I was hoping to have it finished sooner. But you know, life happens, and so they actually got it on the day of their final and their first question is after it’s revealed is, “Do we still have to take a final?” In fact, they did, but it turned out alright. So it was the day of their final which was a week after the end of the semester. But it all turned out fine, everyone was happy, and they got to eat cake.

John: What proportion of their grade was based on this collaborative work?

Jessica: It was actually about 10% of their grade… [LAUGHTER]

John: [LAUGHTER]… low stakes assignment, relatively, for writing a book.

Jessica: It was very low stakes, which, as you choke on your tea, I couldn’t believe that they would do it for 10% of their grade. But in fact, when you take all of these small pieces of writing and put them together, it actually wasn’t a huge whole. When you have 75 people writing a little bit at a time, they got into it when they had to do it but when they were done, they were done. So it was actually just a small percentage of what they had to do. I like to use non-traditional teaching techniques and experiential learning, and so this class also took a field trip and did a lot of other exciting teaching techniques. So this book was something that was a small percentage of their grade.

John: [LAUGHTER] That’s impressive.

Rebecca: How much time do you think each student actually spent on the writing that they contributed?

Jessica: I would speculate a few hours. They were writing in chapters and I didn’t give them page limits, which was interesting because most groups wrote about six pages on average. But then you had some groups write more and they were allowed to put pictures and videos and diagrams because some of these are models. So some of the chapters were far longer and some students are more lengthy writers, some are more succinct. And so it just depended on the group and what topic it was.

Rebecca: Did they also have to present their chapter?

Jessica: They don’t have to present their chapter but they did have to read the chapters. The whole test was based on the book and the presentations that I gave during class on the content.

Rebecca: What advice do you have for other people who might want to take on such an adventure?

Jessica: [LAUGHTER] Ah man… [LAUGHTER] I think you should do what sparks your interest and you should really follow your passion and if this is what you want to do, proceed with caution, but dive in. That seems to be my approach to a lot of these quote unquote weird pedagogical techniques that I like to use. But this was really a joy. It was a lot of work so much work, not only to set it up and get it ready, and then convince students to do it for only 10% of their grade. If you would have seen their faces but, I said, “Hey, guys, guess what? We’re going to write a textbook.” They all looked at me like I was insane. And so did my colleagues. [LAUGHTER]

John: [LAUGHTER] That was gonna be another question. Has anyone there considered following up with this and doing something similar on their own?

Jessica: I have not had anyone take me up on that. I’ve had a few people asked me, “Do you think others should do this?” And my answer is usually, “You should think about it.” It’s a lot of work, but it’s very rewarding the students get so much out of it. For my evaluations, the students not only said that this helps them learn the content better than a traditional textbook, because it was written by other students, but it also helped build their confidence. They were writing a book, and now they can put that on their vitae, and all the students were so excited because they’re going to take it home to their parents and show them what they’ve done in class. They also talked about just that it was new and novel and how that makes them excited about it. But it depends how much time you have, how patient you are, and what the topic area really is.

Rebecca: When you go to do a revision next time you teach the class, do you expect your time costs to be the same, or do you expect it to be a bit different now that you have a structure?

Jessica: I think now that I have a process and understand how much time it’s going to take versus saying, “This will be fine. It’ll work out well,” I think it will take a little bit less time. But I think with that you also have to get students buy-in, because it’s not completely new, it’s something else is someone has created. And sometimes that’s more challenging to add to it, to modify it to make it better. But I think it’s a good exercise in teaching students the importance of revision and adding to something and building it.

John: Did any students object to having the work being posted publicly, or were they all happy with that?

Jessica: They were actually really excited. When we first were doing that hey said, “So where is this going to go?” I said, “Well, it’s going to be public. Everyone’s going to be able to read this.” And they looked at me, and at that time I don’t think they really understood what open source was, completely being on a website in the Lumen platform, being able to see this content. But once it was printed in a book, they said, “Well, where does it go next? Are we going to print more of these is in the library?” I said, “Well you just wait.” And then I was able to send them the URL, thanks to the SUNY OER Services who put it together, so that the students can now show it to their friends and post it digitally and share it.

Rebecca: [LAUGHTER] What kind of kool-aid did you hand out the day you’re talking about this assignment? [LAUGHTER]

Jessica: [LAUGHTER] Well, I should mention that all of these students were actually students that I’ve had previously and I think having that level of trust and understanding that, “you never know what’s going to happen and Dr cougars class, just be ready to roll with it.” I don’t think I could have done that with students that I haven’t had for multiple semesters. So that trust and rapport was really important and saying, “You can do this, I believe in you. Let’s do this together.” I think without that, this would have been more challenging and students would have said, “Who cares about that 10%, I’m out.” But in this case, they understood and they followed along and they were happy to do it. It was something of a challenge. I think some of the comments in which students told me about this, were just amazing. One of the students said that they generally felt this was a great experience and a wonderful opportunity and that experiences like going on field trips and writing a textbook was exciting, and made me feel like a kid again in elementary school. It makes me more motivated and looking forward to learning experiences. So I think having novel experiences and having something that’s new but also exciting and exhilarating and gosh, a little challenging, is good for the students.

John: …and they’re actually creating something themselves, which by itself should be a little bit more motivating than passively consuming a textbook that someone else created.

Jessica: Exactly.

Rebecca: I think the report that you mentioned earlier is important too. I think that’s an interesting component to this particular project that some people might not realize how important that that can be. But having that little bit of trust to go on a bigger adventure, then maybe they’d be willing to otherwise, I think is key, but something that we all can be thinking about.

Jessica: Oh yeah, I think it’s so important to make connections with your students and people do it in many different ways. I’m usually known as being (taryn?? 22:00), that shameful word in academia. But in fact, I think it’s so important and that’s how I can get so much buy-in from students and get them to join me in these learning adventures that we tend to go on.

John: We do have a note to ask you again, to come back at some point and talk about the field trip, things that you do with these large groups of students as well. But I think maybe we should leave that for a future podcast, if you’re willing?

Jessica: Excellent. Sounds great. I love being on the show.

Rebecca: I don’t know if I dare ask our final question. But we always wrap up by asking what’s next? [LAUGHTER]

Jessica: [LAUGHTER] What’s next? Well, this semester I vowed to focus and more self care and I’m actually teaching a new course called Stress and Population Health. And so with that course, I’m trying to take my own advice, which is sometimes the most difficult, and only doing a few crazy activities during the semester. So my students will go on more field trips, they will do some experiential learning, but they’re also going to be focusing on stress within the college campus, and performing some stress reduction tabling around public health and also learning a little bit more about meditation and how overall in the US we’re a little bit too stressed. So with that, I think “what’s next” is we should all take a little bit more care for ourselves, to be around for students and to give a little bit more. So, that’s where I’m putting “what’s next.”

John: That sounds like a good strategy and perhaps chance to relax a little bit and I believe that when I hear about it later, at the end of the semester. [LAUGHTER]

Rebecca: [LAUGHTER] You have to remember that it has to be in comparison to, you know, someone’s lack of stress is really dependent upon how much stress they generally pile upon themselves.

Jessica: [LAUGHTER] Exactly. So, I’m only doing half of what I typically do in my crazy teaching. But still, I think it’ll be fun, exciting, and I’m looking forward to another great adventure and semester.

John: …and you’re also doing a COIL course at some point, aren’t you?

Jessica: I have currently already done one COIL course and I have actually just created a another COIL connection in Jamaica and have plans to create additional COIL connections so that we can actually compare components of health cross culturally and cross nationally.

Rebecca: Sounds really cool.

John: …and we should note for listeners outside of New York that COIL courses are Cooperative Online International Learning courses where classes pair up with classes from other countries.

Rebecca:Thank you so much for joining us again, Jessica and sharing with us how you rolled this project out and giving us all a little bit of inspiration and a little motivation to do some of this work ourselves.

John: Yes, thank you. I’m still amazed by the 10% but I looked through much of the work that your students have done, and it’s a really impressive work.

Jessica: Thank you. They’re very impressive students. I’m honored to have work with such amazing people. It couldn’t have been done without them believing in themselves and believing in what we were doing was important.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer and Jacob Alverson.

[MUSIC]

60. Inclusive Teaching

Posted in active learning, activist pedagogy, commons, equity, evidence-based practices, higher education, learning, open pedagogy, pedagogy, peer instruction, teaching, writing
December 19, 2018

Are your class conversations dominated by a small number of voices? In this episode, Dr. Danica Savonick joins us to discuss a variety of class activities that support an inclusive learning environment and promote equity in participation while increasing student learning. Danica is an Assistant Professor of Multi-Ethnic Literature at SUNY Cortland, and a recipient of the K. Patricia Cross Future Leaders award, a Woodrow Wilson Fellowship in Women’s Studies, and a Mellon/ACLS Dissertation Completion Fellowship.

Show Notes

Studies of bias in the classroom:

Transcript

John: Are your class conversations dominated by a small number of voices? In this episode, we explore a variety of class activities that support an inclusive learning environment and promote equity in participation while increasing student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Danica Savonick, an Assistant Professor of Multi-Ethnic Literature at SUNY Cortland. Danica is the recipient of the K. Patricia Cross Future Leaders award, a Woodrow Wilson Fellowship in Women’s Studies and a Mellon/ACLS Dissertation Completion Fellowship. Welcome, Danica.

John: Welcome.

Danica: Thank you. Thanks so much for having me. I’m really excited to be here.

John: Our tea’s today are…

Danica: I’m drinking a coconut lime seltzer.

Rebecca: That sounds pretty good.

Danica: It is.

Rebecca: It’s a good alternative to tea, I suppose.

Danica: I think I’m pretending that I’m on a tropical island or something.

Rebecca: Yeah, the weather around here would make me want to do that, so perhaps it’s the same there.

Danica: How far away are we from from each other? I’m here in Cortland, you’re…

John: About an hour and 45 minutes, I think, by car.

Danica: Okay.

Rebecca: Very rainy today.

Danica: Yeah, and I hear we have some snow coming up in the next 24 hours or so, so should be interesting.

Rebecca: I have the Prince of Wales tea.

John: …and I have a holiday tea from Twinings that I picked up in the tropics in Orlando at the Online Learning Consortium a few weeks back.

Danica: Sounds yummy.

John: It is good.

You’ve written quite a bit on creating a supportive environment for discussing issues of race, class, gender and sexuality. Let’s first talk a little bit about the context in which you address these issues. What courses do you normally teach?

Danica: I generally teach American literature courses. Sometimes those are general education courses, sometimes they are within the English major. I’ve also taught a number of writing classes that are a little bit more interdisciplinary in nature, and regardless of whichever course I’m teaching I like to give them a theme or put my own little twist on them. For instance, if I’m teaching a writing course, this semester the topic is the purpose of education and so we’re drawing from a wide different disciplines… people who’ve been writing about different learning methods and then when I teach English courses, some of the topics I like to do are the arts of dissent and we’ll look at the theme of dissent in American literature. This semester I’m currently teaching Intro to Multicultural Literature, which has been super fun and then next semester I’ll be teaching a graduate course on feminist world-making, which I’m really excited about.

Rebecca: Well that sounds really exciting.

John: What are some of the challenges you face in discussing some of these issues in your classroom and trying to have productive conversations?

Danica: Well, some of the problems that I’ve noticed are consistent regardless of what classroom or what school I’ve been teaching in, but some of them vary according to the student population. But one of the most common problems that I see is just a lack of student participation, or if there is participation it’ll be the same two or three students who dominate the conversation… and actually just this weekend when I was home for the holidays I was talking to my family about this—my aunt is auditing a course at SUNY Purchase—and she was saying that the same one or two students speak every single class period and she’s curious about what the other students have to say and what they’re thinking… and even my grandmother who was at Brooklyn College in the 1950s… she said she remembers feeling too scared to talk in most of her classes… and so it was only one or two of the… I guess… the brightest and most vocal students who would talk in the classes. And then, of course, as I started teaching I started to notice this as well and I think it’s every new instructor’s nightmare probably that “What if nobody talks? What am I gonna do? What am I gonna do with all that silence?” And so, I guess the main problems I’ve been trying to address are not having the same one or two students dominate the conversation but having really every voice be heard in the classroom… and the more I’ve thought about it and the more I’ve come to study classroom dynamics the more I’ve realized it’s not entirely the fault of the students in those situations, and actually quite often it is the shared responsibility of both the professor and the students to create a kind of environment where everyone feels like their voice matters and that they have something that they can say… that they won’t get shot down by the professor… that they’re not intimidated by their peers and whatnot. So a lot of my work has been trying to increase participation in classrooms and also because my focus is often on race and class and gender and sexuality in literature, we have to figure out how to have productive conversations around those really difficult issues. And for a lot of students, it’s their first time talking about these issues and so we’ve had to establish ways that we feel comfortable talking about those important questions and issues.

Rebecca: I was actually just gonna follow up to what you were saying… really curious about the emphasis on the first time students have talked about some of these things, and I think that that’s really important. We’ve been having a reading group on our campus with a book called Race Talk and that’s something that we’ve mentioned pretty frequently: that a lot of these students have never been in a context to have a conversation about race… a lot of the faculty have never been in a situation to have a good conversation about race… So, when it’s someone’s first time, how do you help that be productive and feel safe? Because you have to be vulnerable to be in those situations.

Danica: Definitely. One of the most effective things that I think I’ve done is tell students that we’re inevitably going to mess up in these conversations because our educations have not provided us with the language and the grammar and the vocabulary for talking about conditions of structural inequality… and so I make that the baseline or the premise. We know we’re gonna say the wrong thing and we are likely going to accidentally offend someone and so as a class what we do is establish protocols or ways that we want to collectively address how to handle those situations and we come up with a set of community guidelines and principles and ideas that we agree upon for how to behave when we realize that, “Oh no, I could have said that better. I wish I hadn’t said that…” or if one student feels offended by something and so I think that has really helped, especially for students who are having these conversations for the first time; they know that it’s okay to say the wrong thing and we have an established procedure in place for how to deal with those moments.

Rebecca: Can you talk a little bit about how you set those guidelines up and how students participate in that process?

Danica: Yeah, definitely. This is one of my favorite things to do every semester is have students co-author a set of community guidelines in order to foster inclusive discussions of difference. Because I want every student to understand that their voice matters and I know one of the reactions that you can get is students can start feeling alienated if they say the wrong thing. They can disengage. They can start thinking that I don’t have a place in this conversation. …and so one of the ways that we create that environment is we’ll co-author this set of community guidelines. Rather than having students write them from scratch… I think that can be really difficult… so, instead what I’ll do—it takes about I would say half of a class period to maybe half an hour, could be 40 minutes, it depends on the size of the class—I’ll print out some really basic guidelines, four or five things that I think might work well in the class as principles that we might want to agree to abide by. So things like we won’t make assumptions about anyone in the class’s race, gender, ethnicity, things like that, and we read over them as a class. I usually project them at the front of the class and they also have them in front of them on a piece of paper and we’ll read through them as a class. They can ask questions… they can ask me to define a word they don’t understand, and then I give them about ten minutes to read through them quietly on their own with pen and paper and cross off and edit and add and remove anything that they don’t like about the guidelines… to add additional guidelines… to change the wording of certain guidelines… and then rather than calling on students individually and having to put them on the spot, I have them work in pairs of two to go over some of the amendments and edits and adjustments that they would like to make and then after five minutes or so we go around the class and each pair presents one or two amendments that they would like to make and so it really ranges from adding different adjectives and verbs to adding whole new amendments or saying that they didn’t like one of the ones that I put up, which is totally fine with me. The idea behind not having to ask them to do it from scratch is just that they have something to work with—it’s not that I’m wedded to those particular principles, I just wanted to give them some kind of language and some kind of grammar for how they might formulate the different community guidelines.

Rebecca: It seems like the pair scenario would help to mitigate any issues that might arise from a dominant group dominating the rules.

Danica: Yeah.

Rebecca: That was gonna be my question but then I realized as you were talking that that might actually be how you solve some of that issue.

Danica: Yeah, and it’s pretty egalitarian. We go around the room and each group says something, even if it’s by the time we get to the end sometimes the groups are like, “Well, everyone already said what we were gonna say, so we just wanted to agree that we really liked the amendment that this other group made.” And so that way each pair gets two or three minutes to add something, to say something, and then we move on, and so it’s not like one pair gets to really dominate. The other thing I forgot to mention is students go home, they have at least one or two evenings to think about the guidelines that we came up with. They have access to them. They can open them up at home and it’s not until the beginning of the following class that we ratify them, and often when we come back together at the beginning of the next class they’ll have thought of one or two things that they want to adjust and once we make the final edits and adjustments then we as a class decide that we agree to abide by them.

John: Another nice thing about doing it in pairs is when people are speaking it’s a little safer because they’re representing their group; they don’t have to take a stand and it makes it a little more comfortable perhaps for those who might have been reticent.

Danica: Exactly.

Rebecca: I like that you have the ability to review over a couple of days as well because that also gives students who don’t want to speak up the opportunity to email you or communicate with you separately too, right?

Danica: Yeah, definitely.

John: How well have the guidelines worked? Have students responded well? Do you get more buy-in to the guidelines since they created them?

Danica: Yes, absolutely. I was really surprised the first time I did this. I was like, “This is one of those wacky pedagogical experiments; I might fall flat on my face, they might think that I’m an alien from another planet.” But, they were so enthusiastic and I’ve actually had students from former classes say “That was one of the most meaningful things that we did that semester. I think about that a lot. I wish more teachers did that…” and so I’ve gotten really positive feedback on it and it’s also fun. It’s always one of the best conversations that we have throughout the semester. And you know it turns out that they have a lot to say about the issue. Actually, I often do this assignment when we’re teaching a work of literature called Citizen by author Claudia Rankine, which talks a lot about microaggressions… and students have witnessed and they’ve experienced these microaggressions in the classroom and so they’re eager to have a chance to participate in crafting a classroom that isn’t going to have these kinds of uncomfortable and awkward moments. I also should say that when we do this I share with students beforehand several of the studies that have been done recently on classroom participation and who feels most empowered to speak in the classroom. So, there’s been a lot of studies done on gender and the experiences of students of color and what a lot of these studies have found is that those voices that are most empowered to speak in mainstream media and culture are also the students who feel empowered to take up time in the classroom. And so I share this with students before we begin the community guidelines activity and they’re always really interested. I have the sense that some of them have witnessed or experienced or might have some sense that these things go on, but to actually see the research and to see the findings and to see these massive studies that have been done, they’re just interested in it, and especially because my classes are about race and class and gender and structural inequality, I think it’s fascinating for them to see the way that what we often think of as huge systemic issues can come to influence who speaks and who participates in the classroom as well.

Rebecca: Maybe we could share those citations in our show notes?

Danica: Certainly.

Danica: You’ve used something called Commons in a Box. Could you tell us a little bit about what that is for people who are not familiar with that?

Danica: Sure, it’s a free open-source learning and writing platform. It came out of the CUNY Graduate Center. It’s a combination of WordPress and BuddyPress, and so it’s this easy to install package that allows you to create digital learning spaces, and so different universities have taken it up to do different things. Often I’ll see institutions using it as a space for their professors to host course websites. They might want to have some kind of blog that features student writing. They could use it for digital humanities projects… and it’s free and it’s open source and so all you really need is server space. As often as possible, I’ve tried to host my courses on either Commons in a Box, or currently I’m using an installation of wordpress.org as an alternative to using Blackboard or Canvas and I could talk a little bit about why if you’re interested.

John: Yeah, could you tell us a little bit about what the advantages of this is compared to say one of the common course management systems?

Danica: Sure. I see the primary benefit of these platforms as they help students to develop transferable skills that are going to aid them in the world beyond the classroom, and so I’ll talk a little bit about what I mean by that. WordPress is one of the most common platforms on which the websites in the world are built .The latest statistic that I saw was something like 30% of the world’s websites are built on the WordPress content management system, and so I like to organize my courses on WordPress so that I can familiarize students with how websites are put together… how you can build them… how they think… how they organize information… and so what I try to do throughout the semester is scaffold students’ interaction with the platform. At the beginning it’s pretty user friendly: they create an account, they are able to log in to our site and then gradually they start going into the backend (which WordPress calls the dashboard) and they start creating their own content. So, they get to experience the process of going back and forth between the backend and then the front-end and seeing what that process is like and how information is organized on the WordPress platform. So, they start creating blogs and then what I like to do towards the end of the semester is deconstruct our class website and take it apart and break it and redesign it with students so that they can see, first of all, how easy it is to build a website. A lot of my students are new to this. They’re not necessarily computer science majors. They haven’t taken computer science courses, and so they’ve interacted with a lot of websites but they haven’t really gone in and thought about how they might build their own and so I show them how our course website is built and we redesign it we do all kinds of things and then often for students’ final projects they will have the option of designing a website related to something that we have done in the course and they often choose that option. They like it… they like getting to experiment with WordPress. For most of them it’s their first taste of the platform and several of them have said that they’ve gone on to learn more about WordPress because they’ve become really interested in it and I see this as a really great opportunity for students first to think a little bit more critically about how the internet works and how these pages that we’re constantly interacting with… how they’re constructed… and also to develop a transferable skill that could become a really valuable part of their resume and the skills that they will bring to the work world. Being able to build websites on WordPress is huge and so I find that starting that process early can be really helpful. It also creates an opportunity for us to have conversations like why is our course built on WordPress when all of your other courses are on Blackboard and we get to talk a little bit about what Blackboard is and the different ways that content management systems, especially in higher education, work to structure certain kinds of relationships of teaching and learning.

John: Does the institution host WordPress, or are you hosting your own instance of it?

Danica: Ideally, the university will host it. When I was at CUNY they have a really strong culture around open educational resources and free writing platforms and there’s a big community around that. It might exist at my new institution—I have to do a little bit more work to find it. As far as I know there’s a lot of people that are using Blackboard at my institution. Ideally… best-case scenario… the university would provide server space and then you could have an installation of WordPress or Commons in a Box, but currently I’m using Reclaim Hosting and Domain of One’s Own in order to have a classroom commons installation that I’m using across the three different classes I’m teaching.

John: Do you use the open aspect of that? Is the students’ work public or are you keeping it closed to the classroom, or is that something decided on a case-by-case or class-by-class basis?

Danica: Yeah, that’s a great point, thanks for bringing that up. It varies. Parts of the class web sites are public, parts of it are private… and another benefit of working in a quasi-public, quasi-private space is that it allows us to have conversations about what information students are putting on the Internet and what they want visible. What do they want to become part of their professional digital identity? What do they want to show up in search results versus what do they not want to show up in search results? So, we have a lot of these conversations early on in the semester when they’re establishing their accounts. We talk about the risks and the repercussions versus the benefits of using their real name to do the blogging that they’ll be doing on the site, and then for their final projects… often, but not always, I would encourage them to use their real name because they put a lot of time and effort there carefully revising these projects and they are deliberately constructing them with the idea that they’re going to be writing for a public audience. But, of course, in this climate of anti-immigration that we’re living in, you have to be super careful about what you’re encouraging students to put their name on and so I always have conversations around that. There’s always an option never to use your real name. You can always use a pseudonym for the blogging and for the final projects. You can always submit solely to me instead of publishing to a public audience, because I understand there are severe risks and in some cases they will outweigh the benefits of creating something publicly.

John: And we should note that you have an article in the describing your work here and we’ll include a link to that in the show notes as well.

Rebecca: You mentioned student blogging. Can you talk a little bit about how the student blogging is used in your classes and how that augments student learning and how that might facilitate some of these conversations that might be tricky to have?

Danica: Sure. I love student blogging—I don’t know how I would teach these courses without it. My courses are structured around the blog—it’s one of their major assignments, and so for every single class two or three students are assigned to blog about the assigned reading—I think the requirement is something like 800 words or so—and that they have to do a small close reading… so an analysis of the excerpt of whatever literary text we’re reading and it has to end with two discussion questions, and for every student who isn’t blogging. So, the majority of the class they have to leave a comment on those blogs before our class period starts, and so the blogs are due at noon the day before class and then students have from noon until our class period to leave their comments and then the way the course is structured the same day that those three students are blogging… so they’re each writing a blog… they are also facilitating a class discussion… a ten-minute activity… or it can be a presentation… it can just be more of a conversation. They have ten minutes at the beginning of the next class to do whatever they want, and I encourage them to make it the best lesson plan that they have ever seen or the way that they want their ideal course to be structured, and so I encourage them to try things like think-pair-share or to do interactive activities and it’s really exciting to see, first of all, the things that students choose to blog about, because with the readings that we’re discussing there’s so much that you could talk about. I have certain things I want to talk about but those might not necessarily align with what students are interested in within the text, and so having these open-ended blogs allows students to identify what it is they’re most interested in; it allows them to get feedback on their writing from their peers prior to our class session. One way that I’ve come to think about the blog… that I talk to students about it… is as a rough draft for a paper. They’re putting out a thesis… they’re putting out an interpretation… they’re providing some evidence from the text to support it… and then they have this tremendous opportunity to get feedback from all of their peers… and so in the comments the other students will be like, “I really like this point…” “I have another example that can help you support your point…” They might raise objections; they might raise counter points: “Well, have you thought of this other thing?” And so it’s a really great way for them to increase the quality of their writing and their ideas by getting feedback from their peers. Actually, this happened just in our previous class, a student was using a term “devaluing” to talk about sexuality in one of the books that we were reading and a lot of his fellow classmates were saying that word wasn’t working the way that he thought that it was working. So, in his facilitation he kind of talked through the feedback that he got and as a class we came up with a better word that would more precisely name the kind of relationship that he saw developing in the literary text, and so with the class facilitations it provides students with an opportunity to practice their public speaking and to practice standing up in front of a classroom. A lot of the students say that they’re really nervous at first, but that they’re glad in the end that they did it and they always get through it and we always manage… and so this kind of pairing of the blog with the in-class facilitation really teaches students that they are active knowledge producers and that they have something to contribute to the class and that their voice matters. They know that they’re not allowed to just disappear and sink into the background—they’re actually the ones up there in front of the class leading the lesson and it’s interesting to see actually the ways that it increases their performance once they’re back in the chair of the student, because they know what it feels like to be up at the front and so they’ll put out a question and they then get to experience what it’s like to have no one raise their hand and so they become much better as students and much more engaged once they return to their seats and resume that more traditional role of being a student. I never know what students are gonna do for their facilitation. They don’t have to run it by beforehand, so it’s always exciting. I don’t know what they’re gonna do in class today and it’s really made my role as an educator different and I’ve had to learn to listen really carefully to the things that students are saying when they’re up there presenting and my job becomes connecting what they’re saying to the main ideas and the main skills and the main topics of the class. So, for instance, if a student is giving a presentation I might interject and say that’s a thesis statement,… what you just said… you just made a thesis statement and then they start to recognize learning how to make an argument, how to make a thesis statement is one of the skills of the course, but it takes a long time or they’re not quite sure what I mean by that, but when they’re talking they’ll just do it naturally and so my job becomes pointing out to them that they are already doing the things that we’re learning about and just helping them recognize better the ways that their facilitations are connecting to the themes and the skills of the course.

Rebecca: I wanted to circle back to the leaving comments for other students—so they do the close reading, they post about that and then students comment on it. When they’re commenting, how do you help students learn what a good comment is?

Danica: Yeah, that actually becomes a topic of discussion early on in the semester. They’re given a few guidelines: it should be, I don’t know, a hundred and fifty words or so; it needs to make a contribution to the post; it can’t just be “I liked your post” or “I didn’t like your post,” and then what I’ve tried this semester is we implemented—kind of halfway through the semester—this rule that each comment needs to provide a quote from the text so that the commenter is either supporting providing further evidence that will support the author of the blog’s claim, or providing a counter example. One of my students last class, he said “Conversation makes the best interpretation” and I really loved that because they’re starting to learn through the commenting the ways that all academic writing is a conversation among various viewpoints and that when they’re writing a scholarly paper… when they’re writing a research essay… they are inserting their voices into larger conversations; they’re in dialogue with people. It’s not like you write a paper in a vacuum; it’s actually a synthesis of all these different viewpoints and ideas and so I see the commenting as kind of a rehearsal for class discussion. So when we show up in class, say, for instance, students aren’t being particularly talkative, I can say, well, you said this in your comment, and so I know that they’ve already engaged with the ideas and it allows often our class conversation to reach a higher level because they already know what several of their peers’ interpretations of the text are; they’ve already thought about them; they’ve already thought about the pros and the cons and how we might need to complicate some of these analyses; and so it just takes our class discussions to the next step.

John: Do you do anything to ensure that everyone responds to a certain number of posts to make sure that you don’t see everyone replying just to one other post to make sure you get some balance there? Do you have a mechanism for doing that?

Danica: Yeah, that’s a great question. So, for every class students have to comment on at least one post; they’re welcome to comment on more than one, but the requirement is one comment prior to class. I don’t have a mechanism for ensuring that. The class is 25 students, if we have three bloggers, one blog might get ten comments and the others might get four or five, and one way to kind of address that in the classes is encouraging students to think about their blogs, think about the title of the blog, think about the content of the blog, think about how they’re competing for the attention of their peers. I encourage them to say, “Okay, your peers have three blogs to choose from, how are you gonna get them to read yours?” It’s a way of getting them to think a little bit about audience and what is the function of a title. What is the work that a title can do? …and from that introductory paragraph how can they give their reader a sense of what their blogs gonna be about? How can they convince their reader that there’s gonna be a good payoff that their blog is worth reading? And so it’s interesting to see the different ways that they try to attract the attention of their peers. Because they do want those comments and I find they get excited about the different feedback—they’re not required to respond to the comments, but they do often… which, yeah, it’s always exciting to witness. I try to linger, I lurk a little bit on the blogs and I’m often not interjecting in the conversations, but just kind of reading through them and that’s actually the other benefit is that it ends up serving as a mode of formative assessment because I can see what they have understood from the readings and what might be missing… what might be the things that I need to address in the time that I have—what’s not quite getting through to them, either in terms of aspects of the reading that they overlooked or in terms of the skills. So, if I tell them that your blog needs to have a main point; it needs to have a thesis, and I’m seeing that they’re not quite doing that I can then adjust my lesson plans so that that becomes the focus of the next class and I can use their blogs, their own words as an example to say, “Okay, how could we give this blog a stronger thesis?” …and so it’s quite common that we’ll end up editing or revising some of the blog posts. They get projected up on the screen and students, because we’ve created a culture that they’re constantly giving feedback on each other’s ideas, students feel a lot less embarrassed or they understand that we’re all trying to become better writers and so they’re okay with it if I project their blog post and we talk through “What are some of the pros? What are some of the cons? How could we strengthen this?”

John: And it’s a much more authentic learning experience having them focus on audience and trying to build a strong thesis statement.

Rebecca: It seems like the blog post assignment really primes students well for the final projects that you had mentioned earlier that have a public audience because they’re already practicing writing for a specific audience and it’s another writing for a more general audience, I would assume. Can you talk us through that a little bit?

Danica: The final projects for my class often vary, but they’re usually collaborative… they’re usually digital… they’re usually public… they’re usually some kind of creative student-driven element. It’s usually students identifying the topic and then running with it, whether that’s a research blog or whether that’s currently my students in Intro to Multicultural Literature are co-authoring a glossary of key terms for literary studies—I have never done this before. It is a total experiment—I don’t know if they know that this is my first time doing this, so it’ll be interesting. I don’t know if they’ll hear in this podcast, but whatever, it’s fine. So I’ve done different versions of these collaborative public final projects. They vary sometimes based on the content of the course, students’ level of preparation, what are the aims and objectives of the different courses. It’s a little bit different for a basic writing composition course versus a more advanced literature course… and so one of the format’s I’ve done is have students co-author scholarly articles that they would submit to an actual journal, and so I did this in one of my freshman writing classes. We spent the entire semester talking about contemporary issues in education… so related to technology in the classroom… active learning versus lecturing… conditions of educational equality in segregated schools… and about halfway through the semester they were put into groups and they had to identify a research question. They did an annotated bibliography, they developed a whole research project, and then they made it into an article, a short article that they submitted to the scholarly academic peer-reviewed journal Hybrid Pedagogy to see if they could get it published or not, and I had been in contact with the journal’s editors since the summer before the class, so they knew this was coming. It would not have been possible if I hadn’t been working with them because they knew there was gonna be a really quick turnaround time where the students needed to know if they got revise and resubmit, if they got rejected, or if they got accepted… and I knew that this was a wildly fanciful or an unrealistic expectation to ask students to get a scholarly journal article published—these are basic writing students at Queens College—a lot of them are first-generation students… they work jobs… they are English language learners… and so in addition to reviewing of the conventions of English grammar and how to write a paragraph… how to write an academic paper… all things that were new or needed to be reviewed… they were also trying to get their writing published in a major publication… and so what ended up happening with that is that several students got revise and resubmit. But by the time they did it was the end of the semester and finals were happening and so I tailored the assignment a little bit towards the end. I tweaked it, because all semester I’d been telling them “these blogs are important, these things that you’re writing your research, everything, all of this matters because people are actually going to be reading this and you want them to take it seriously and you want them to listen to you. You don’t want to lose their attention halfway through.” … and so we needed to come up with a way that they would still get published even if they chose not to endure the editorial feedback loop of revise and resubmit, or the accept with minor revisions, and so what we had them do is they took the feedback that they got from the editors and several of the groups chose to post to HASTAC.org, which is a tremendous resource. It is an academic scholarly network of 15,000 plus members of scholars and students and academics and artists and activists and so there’s a special group within HASTAC that showcases and features and highlights the writing of undergraduates. So many of my students ended up submitting their final blogs there, but one group did continue—they kept revising their submission and going through the queries that they were getting from the editors and then the copy editors and just all of these stages of the writing process that were very new to them. This is a required writing course… no one showed up at that course eager to do all these drafts and revision and the skills that we teach in a basic writing course… but they continued in that editorial feedback loop for about a year after our class ended and then in August of 2017 their article was published in Hybrid Pedagogy, which was very exciting and so that is now something that they can put on their resumes and I was just so impressed with them for sticking through it because we know everything that goes into writing a journal article but for them they didn’t even know at the beginning of semester what a peer-reviewed journal article was… and so it was like a huge learning process. So, that’s one of the formats of these collaborative public final projects: submitting something to an established publication, which required a lot of willingness on the part of the journal editors to work within a really quick timeframe and the managing editor Skyped into my class several times and talked to students about the journal, helped them with their submissions, they got to pitch their ideas to him, it was great. Some of the other formats I’ve used that have also been good—I’ve had students write explicitly for HASTAC and that’s an opportunity for them to tailor their writing for a very specific community. So that’s something that we did this semester in the writing class that I’m teaching. We read so much about HASTAC… we read about its history. There was an article in Inside Higher Ed calling it the ethical social network and talking about their commitment to protecting their users’ data; we learned about who is a member of HASTAC, who are the different people who are reading it; how does the website organize information—by topic, by tags, by categories and so they were reading and analyzing the site itself before they even started writing their research blogs. So it was a similar process where they identified a research question and they authored blogs that were specifically going to be then tailored for the HASTAC audience, and so one of the big aims of that assignment was this skill of kairos and figuring out how to tailor your writing for a specific community of readers and figuring out what are the conventions? what are the affordances of this specific writing space? and how can I best get my point across to this very specific audience? So, that was useful in helping us have conversations about audience awareness and tone and how you make an argument; how do you convince someone that your point is right without alienating them. So that’s one example of having students write for HASTAC. …and the nice thing about HASTAC is that it comes with a built-in user community. You have sixteen thousand people who are visiting the site and reading things. The other kind of format for these public projects is what we’re doing now with the keywords. Students are co-authoring these individual keywords, they’ve identified specific words that have emerged that they’re interested in throughout all the readings that we’ve done this semester and through our discussions and through the blogs and the final product for that will be something that is hosted on our course WordPress site… so it’ll be a page or an offshoot—I’m hoping to write some kind of table of contents that will link to each of the student’s posts—hopefully there will be media, and so this will be something that they can then share. They can decide that they want to make it part of their professional identity, part of their portfolio, or they can decide not to—it’s really up to them. …and so one of the things that I’m constantly thinking about in developing these assignments is like how to actually connect students to audiences of readers and people who could actually benefit from a keyword entry on memoir or on ghosts… that’s another keyword, apparently we talk a lot about ghosts in my class… and this is coming out of the research that I’ve done on activist pedagogy and really thinking about the role of the teachers connecting students to audiences and people that could potentially benefit from the writing that they’ve been doing. So, thinking about these projects as both a benefit to the students in the class and also to larger publics and communities.

Rebecca: How have students responded to this sort of work? You know, you mentioned that, you know, some of the classes are required courses—students are not necessarily marching in excited to do these sorts of things… so it sounds like you’ve hooked them a little bit. What is their final response to these?

Danica: In general it’s been really positive. The jury’s still out for the semester. We’re gonna do course evaluations I think next week, so I’ll learn more. But in the past, I do a lot of framing around what we’re doing in part because I was always a very willful student and I did not like being told what to do. But if I understood why I was being told to do that thing then I would get really into it and really excited and so with these student-centered assignments and activities I’m always super explicit: this is why we’re doing this; these are what I see as the benefits of this; this is why we’re authoring a set of Community Guidelines; this is why you’re doing a presentation; you’re doing a presentation because public speaking is one of the most valuable skills that employers look for and so when I’m writing your recommendation letters I want to be able to tell them what a great public speaker you are and that’s why I’m asking you to stand in front of the classroom and facilitate this. Also, I should mention this semester and at SUNY Cortland a lot of the students are going on to become teachers and so it’s important to have these experiences at the front of the classroom. I think that being explicit really helps students, and the other thing I’ll say is that I often am explicit about how frustrating student-centered learning can be, and we talked about how it can feel difficult and how sometimes we just wish that the teacher would give us the answers rather than making us figure it out ourselves or making us work in small groups and so I try to create spaces for students to express those kind of emotions and reactions to things. I also try really hard in designing these student-centered assignments… to design… to create the conditions where for instance, we’re doing a collaborative writing project… I try to give them an assignment that actually requires multiple minds and that if they had tried to do that exact same assignment on their own the final product would not be as good as if they were doing it as a group. So I put a lot of thought into kind of carefully constructing these in a way that they will be oriented to succeed in them. Recently I wrote a blog on collaborative close reading, which is a really, really difficult skill to teach—it takes years, you know, for most of us to learn how to do close reading, but I’ve tried to create this assignment that had students work on it in groups, and so rather than having to notice all of a million different things that are going on in a passage of literature, they had a bunch of different minds put to the task and they were all looking at the same paragraph for 20 minutes and dissecting it and they were all contributing their different insights and so rather than having to go at it alone they were able to learn from the different perspectives that the other students brought to the text, and so I think just being really explicit about why behind everything has helped to ensure that the reactions have generally been positive.

John: How have other faculty responded? Have other people started working on building more productive conversations? Have other people in your department started working more on open pedagogy projects?

Danica: Well, I would hesitate to say anything explicitly about my department because I’m so brand-new. It’s my first semester in the department. But one thing that’s been super exciting for me has been to see people… especially with this recent blog that I wrote on collaborative close reading… it went viral on academic Twitter and people have been reporting back, because that’s one of the things that I asked them to do is let me know how it works… let me know if, you know, you have any suggestions for how to make it better… and almost every day I’m getting tweets from people at universities across the country saying, “I tried collaborative close reading and this is what my students did…” and they’ll post pictures of the passages that their students highlighted and so that makes me feel like I’m part of a community that is bigger than my own institution. So, when I’m running these, of course I hope that they will be helpful to my colleagues, but I’m also… I really feel like I’m part of a bigger academic community and part of that is because I post these blogs to HASTAC, and so it really is, people are a community of 16,000, however many users, but then it can get tweeted out. So, even if you don’t have a HASTAC account you can still read the blogs and there’s so many ideas about scholarship being really isolating, but things like that… and getting to talk to people and discuss pedagogy with people at different institutions makes it feel a lot less isolating. But in terms of your question about reactions of colleagues, I have been super lucky both at CUNY and now at SUNY in terms of support for the kinds of things that I’m interested in doing. These are schools with very strong commitments to education, where people are already interested in and talking about student-centered methods and curious and wanting to learn more. The other day I have my students write found poems—which is the genre of poetry where you take some kind of existing document—often it’s a bureaucratic document—and you make it into a poem by cutting it up and whiting it out and mangling it and turning it into poetry and my students created these awesome, awesome found poems; they were beautiful. So, we spent a day in class, I gave them whiteout… I gave them scissors… I gave them tape… They started as banal documents; they made them into stunning poems. They would bring in their tuition bills or song lyrics with offensive stereotypes in them—one of them brought in the transcript from the Brett Kavanaugh hearing and they took these documents and as a way of thinking about language and power they made them into these gorgeous found poems and so I went to the chair of my department and I said, ”Hey, you know, my students created these poems and I would love to have someplace to display them ‘cause I think they’re really awesome” …and not only did she give me permission to tear down what was on the bulletin board, she helped me do it. We tore down these old flyers that had been up there for decades and we put my students’ found poems on and so now we have this beautiful display in our hallway of student work and several of my colleagues have reported seeing students stop and read the poems and take pictures of it and so they’re excited to see their work has become part of this gallery. In general I’ve just been really lucky and fortunate to work with colleagues who are similarly invested in helping students. I did think of a few ideas for people who might not be so fortunate on ways that they could start doing student centered things. The first would be, and I’ve already mentioned this, creating a free profile on HASTAC.org because there are so many people out there doing really creative and exciting things in their classrooms… and connecting their classrooms to larger movements for social justice… and thinking about how do you engage students in really important discussions about contemporary social issues… and so HASTAC has been a phenomenal place for me to connect with other people who are doing that kind of work. I try to start small, and so something like think-pair-share is so easy… it’s taking an index card, giving students 90 seconds to respond to some kind of open-ended prompt, then they turn to the student sitting next to them, they share their responses and then we go around the classroom and I transcribe each group’s answer to the question on the board… and so their ideas become the material that I then get to teach the course through, and we crowdsource responses to some question related to whatever the topic of discussion is that day and something like that is so easy, it’s so simple—we have gone through… I would say we’re in the thousands of index cards in terms of my courses this semester. Because the students like it and they recognize that that changes the classroom dynamic. They recognize that suddenly it’s not just one or two students dominating the conversations. So, when they get up to the front of the class and they get to facilitate something I would say, I don’t know, 65, 70 percent of them choose to do think-pair-share because they recognize that it really lowers the barriers of anxiety about participating in class—everyone has 90 seconds to scribble something on their index card and it’s only an index card,— it’s tiny—there’s not any kind of pressure to write something beautiful and then that becomes just such an easy way to really transform the dynamics of the classroom… low cost… low time investment… and even when I’m thinking at the bigger scale of assignments and rethinking the research paper so that it’s not just being submitted to the professor, but it’s for a public audience, I try not to overhaul everything at once. So it’s like each semester I’ll try one new assignment. Not throwing everything away and starting from scratch each time, because it takes a lot of energy to do these things. And so thinking about how we can make small changes and experiments but not overwhelm ourselves or our students… and the other thing that I would suggest if somebody finds themself in a situation where they want to start trying these things but might not have the kinds of tremendous support that I’ve been lucky to have is… there’s just so much research out there on the effectiveness of student-centered pedagogy. We’ve read a lot of it in my course this semester on writing and education. We read a lot of the studies that have shown how positive of an impact it can have on students to discover ideas for themselves and to work in small groups and solve problems and arrive at answers rather than sitting and listening to a lecture. Just kind of having some of those studies in my back pocket—I’ve always felt that if I was called upon… you know, “Why did you do that? I can’t believe you let the students help assess each other’s papers.” I would have some things that I can cite, that I could go back to to say “Well, actually, it’s been shown that asking students to metacognitively reflect on the implications of their writing is a great strategy.” So that kind of thing has been really helpful for me in terms of thinking about relationships to colleagues and different reactions to this kind of pedagogy.

Rebecca: We normally wrap up by asking, what next?

Danica: Well, at the small scale, I guess, we have this digital glossary of keywords that will be coming out from my multicultural literature students… going to learn all about ghosts and power and assimilation and why these words are important for how we think about and analyze literature… so really excited to see what they do with that… and then I guess at the bigger scale I’m working on a book on the activist pedagogy of teacher poets from the 1960s and 1970s, and I’m hoping that some of that work will help us really understand the ways that a lot of contemporary student-centered practices… things that we’ve talked about today… a lot of them emerged in the 60s and 70s and especially in relation to the critiques of power emerging from the social movements of that era from the women’s movement and the civil rights movement and protests against the Vietnam War… and so I’ve been thinking a lot about how those critiques of power necessitated new relationships of teaching and learning and this was especially happening in the work of poets who I’m interested in and so that book is also considering the ways that interactions with students shaped American literature in ways that we rarely consider and also the tremendous role that poets and authors and especially feminist poets have played in creating a lot of the contemporary student-centered pedagogy that we know today to be so effective.

Rebecca: That sounds really exciting.

Danica: Yeah, it’s really fun to work on.

John: In some of your posts you’ve listed a large variety of techniques that people can try and we’ll include links to those in the show notes as well. Thank you, this has been a fascinating discussion and we look forward to hearing more about what you’re doing.

Danica: Thank you so much for having me, this has been really fun.

Rebecca: Yeah, thanks for joining us. [Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

55. Open pedagogy

Posted in accessibility, commons, higher education, interdisciplinary projects, learning, OER, open pedagogy, pedagogy, peer instruction, teaching
November 14, 2018

Imagine an academy that values a public knowledge commons and supports and recognizes the academic labor required to develop, maintain, build and evolve that commons. Imagine your students actively contributing to that commons. In this episode, Robin DeRosa joins us to discuss open pedagogy, free textbooks, and the building of such  a commons.

Show Notes

Transcript

John: Imagine an academy that values a public knowledge commons and supports and recognizes the academic labor required to develop, maintain, build, and evolve that commons. Imagine your students actively contributing to that commons. In this episode, we discuss open pedagogy, free textbooks, and the building of such a commons.

[MUSIC]

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Robin DeRosa…

Robin: That’s me.

John: …Professor of Interdisciplinary Studies and Director of the Interdisciplinary Studies Program at Plymouth State University. Robin is an editor of Hybrid Pedagogy and is a co-founder of the Open Pedagogy Notebook. She has also published on a wide variety of topics, including the Salem witch trials. Welcome, Robin.

Robin: Thank you.

Rebecca: Today our teas are…

Robin: Oh, I thought we were talking about teasing people for a second and I was like, I don’t have a tease. [LAUGHING] What am I teasing?

Robin: No, I actually have two cups of tea in front of me, which is how I like it. One is a ginger tea and one is a sunny orange because I have to stay away from the caffeine at a certain hour of the day, so I’m all herbal.

Rebecca: Sounds like a nice combination.

Robin: I know, I’m just taking one and then the other; it’s perfect.

John: And I have a Tea Forte Black Currant Tea.

Rebecca: I have a Jasmine Earl Grey, that wasn’t there before, so I gave it a try.

John: It’s been there for a year.

Rebecca: Wow, it’s been hiding under the big pile of tea that we have.

John: In exactly the same tray…

Rebecca: No…

John: I just refilled it today.

Rebecca: Oh, you know what, it was the box sitting on top that you didn’t take back after we refilled the tray.

John: Okay. [LAUGHTER]

Rebecca: That’s where I got it from.

Robin: But your listeners are probably like, how could you not see a box, but if they saw this table, my jaw was on the ground; it is a really quite an impressive tea table that you’ve assembled here. You should be very proud.

Rebecca: We don’t mess around.

Robin: You do not mess around. I would use even other words but I know… public… this is some serious hardcore tea happening here. [LAUGHTER]

John: We invited you here to talk about your work with Open Pedagogy. For those that are unfamiliar, can you explain what is meant by Open Pedagogy?

Robin: Sure, which is such a funny question really because if there is a thing—and I could just say it—because there’s a lot of productive disagreement in the community about what Open Ped is; it’s one of the reasons that my colleague Rajiv Jhangiani and I started the Open Pedagogy Notebook because it’s more of a collage approach to defining Open Ped by people doing and practicing in different ways and then sharing that, but if I had to boil it down I would say it’s really about access, both to knowledge and to knowledge creation… so, the idea that we remove barriers to sharing resources and helping people access conversations and find pathways into education, but then we also try to find ways to amplify student voices to make them contributors to the Knowledge Commons and not just consumers, and I think it’s pretty salient right now as students are really in some ways maybe being pushed into these kind of training and competency models that are really about kind of downloading information and instead Open Ped suggests that we really want students to interact with knowledge and shape the world that they’re going to graduate into, not just train for it.

John: So they’re more actively engaged in the academic conversation?

Robin: That’s the idea. Right. In some ways there’s a lot of kinship, I think, with connected learning and with the idea of involving students in their academic and professional networks. Right from the beginning, because even as people who are new to our particular field, they have so much to offer and as an interdisciplinarian, we talk about that all the time that the outsider’s perspective is beneficial—it’s part of the reason you assemble an interdisciplinary team to tackle a problem and newcomers to a field ask sometimes questions that really can illuminate the challenges that a field is facing in new ways, so I have found that even the most beginning introductory students in a particular area have something to contribute both to the field itself and also in terms of helping their peers in terms of, for example, making educational materials. Students are really well positioned to make great educational materials ‘cause they understand better than anybody what’s hard to understand about a certain area.

John: They’re not subject to the curse of knowledge…

Robin: The curse of knowledge. [LAUGHTER] I have that curse, John.

John: We all do to some extent.

Robin: But it is true like when especially when I was teaching Early American Lit and you just finished your PhD and you start teaching and then you teach something for 15 years and no matter how gifted of a teacher you are, sometimes you’re like, “I don’t understand how they don’t understand this,” like “what’s hard about this?” and of course they really understand what’s hard about it, so when we did student projects where students were working on a textbook that we were crafting together, they really made some great materials for each other that I think were far better than some of the lectures I would have prepared or had prepared over the years.

John: So tell us a little bit about how you got involved in that first project you had?

Robin: I was at, ya know, one of those faculty development events that you guys might be aware of and they had brought in, ya know, a keynote speaker, and I don’t want to say I wasn’t prepared to be impressed, but it was a technology oriented conference and I was definitely one of those curmudgeons that was highly skeptical about how useful… or actually more skeptical about the ways technology was being deployed, so I was prepared to be mad, that was how I came in the room, and I actually still think that’s generally my positionality with technology is like, I’m pretty prepared to be skeptical at all times. But the keynote speaker happened to be Cable Green from Creative Commons and I had this really just pivotal “aha” moment when he was talking about the Creative Commons licenses where I realized that my students were paying 90 bucks every semester, each student, for access to public domain Early American Literature and my heart just sort of fell on the floor. It’s like why are we paying commercial publishers when all of this stuff is out of copyright. So, some students and I spent that summer before the next fall’s class rebuilding the Heath Anthology of American Literature out of public domain texts that we found online and we did not build a hearty replacement for the Heath, but we built a sufficient replacement and when we got into the class the students were super psyched that I had saved them 90 dollars, which is totally real money to my students and makes a big difference. They were grateful but they did not like the book, because it had nothing except public domain literature, so there were no introductions, no maps, no footnotes, no glossaries, no “Don’t worry, I know this doesn’t make sense to you, but let me walk you through it” kind of ancillary scaffolds. So, of course, it seems obvious now but at the time we thought we were rock star smart when we figured out like, hey, the students can make this stuff for the book, and so the students worked ahead, different pods of students would work ahead a week or two and build wrap around materials for that chapter and we got there the rest of the students would use it and, of course, it just ended up being 500 times better than the Heath Anthology of American Literature, partially because they were excited that their classmates were reading their work instead of putting it in Moodle or Canvas or Blackboard, where things go to die. So, they got excited about doing what David Wiley and others have called these non-disposable assignments and then they start getting creative, they start making little videos. Ee’d drop those in… two-minute intro to the Haitian Revolution or whatever. I put a little app in the sidebar called Hypothesis where students can annotate and so they liked that and then at the end of the semester people are like, “The best part of this class was the textbook,” which…

Rebecca: Which they made.

Robin: … they made… and they never said about the Heath anthology. So, that transformed my pedagogy partially because I was excited about making all sorts of access oriented changes in our program; we opened a food pantry at the same time as we did this, so we were thinking about lots of things in terms of accessibility of resources for students. But, in terms of thinking about not dumping my student’s work down the digital toilet every semester, it gave me stomach cramps when I first thought about what I had been doing. Every time I taught the course it was the same course, the student’s contributions weren’t transforming anything—it was no wonder that some of it felt dry to them. There was a lot of hoop jumping, and I still think I was a good teacher, that wasn’t like I was bad, but this idea of really empowering students to really, truly engage with the fields and the materials and shaping how the course runs has really changed everything for me.

Rebecca: There’s something that you’re talking about… the idea of building the textbook together, but then the course material is sort of the same from semester to semester and the materials are created by the students, so how does that continue to transform semester to semester so it doesn’t feel like it’s a one-off project… that it continues to evolve and it continues to be a value and that students continue to learn new things?

Robin: Such a great question and I get it a lot because people—I think in some ways mistakenly assume—that that first build was the exciting build, but totally that first build was the annoying build, right, because we got to do a lot of legwork tracking down these texts. We had to do a lot of copyright discovery, like “Can we use this version of Thomas Jefferson or not” and it wasn’t all that exciting, and that’s still ongoing. So Rebus community, who’s working on that textbook now, they took our version and they’re building it out; they’re still doing that kind of discovery in that initial work, and to be frank, that will never end. Even with Early American Literature you’re constantly discovering, changing excerpts, building things out, but to me what’s even more exciting is, for example, there’s a whole unit at the beginning of Native American Oral Tradition and asking critical questions about what’s the genesis of American literature. So, at the same time as I was teaching that that semester we had the Dakota Pipeline stuff happening and lots of stuff about water protectors and about native history so you can relate things to current events that way, but also think about when I finished the course, my colleague Abby Goode, who teaches Early American Literature, but very much from an environmental and sustainability perspective, she kind of remixed the whole book so that now it was about the environment in Early American Lit; they chose some different texts, they reframed the introductions. Her book, in my opinion, is quite a bit more coherent than ours was, which was more of just a collage of things. So, there’s all sorts of opportunities for how you shape and reshape, and, of course, what students are learning there, which is really the point of those survey classes in English is that there is no American literature; there is only the canon that you intentionally assemble. There’s a lot of politics and a lot of editorial choices and each semester it could look different and that’s an important lesson because the Heath anthology was not a neutral objective collection either and I think that’s been a helpful way for them to encounter the building of canon.

Rebecca: I think it’s really easy for people to imagine how this works in the humanities. Can you provide some examples or ways of thinking about being open in the same way in STEM or in business or other disciplines that might not latch on quite as quickly?

Robin: Yeah, and my colleague at Keene State, which is one of our sister institutions, Karen Cangialosi, published a wonderful article recently called “You Can’t Do That in a STEM Class,” which is basically the answer to your questions. We should just stop answering questions and you should go read that article now. But really, the open dcience movement is a huge movement, in some ways dwarfs open education and I think climate change is a really good example of this, but also just open access publishing, the idea that in order to have scientific breakthroughs we need to have the public sharing of scientific knowledge and collaboration in science and so bringing our students into that early as opposed to saying, “Here when you’re a student you’ll be confined to this one class and stuck in this one book, but then when you become a scientist we assume you’ll just understand how to become part of this larger, more public scientific community.” That makes really little sense. So, what you’ll see in classes by people who work this way in STEM, and Karen’s a good example, is that their students are working on issues that are of critical importance and they’re putting their research and ideas into the commons and asking mentor scientists to engage with them. So, we understand that our students are not always going to be doing top-level research; the next breakthrough in diabetes research is not necessarily gonna come from a sophomore. Although occasionally you hear those stories, right, but really what they’re gonna do is they’re gonna ask for guidance, they’re gonna ask for help, they’re going to amplify other scientists’ work and translate them for their communities so that a new generation of scientists can get access to the issues and that’s how we’re going to assure that our scientists are working for what I might call the public goods. So, we’re seeing lots of people using blogs for this purpose instead of just doing their labs in a vacuum or whatever, sharing some of that work and creating sites together, or working in experiential ways through internships. This is why I sometimes bristle about OER being kind of like a cheap or free textbook movement. It’s really very much about a public knowledge commons and how we bring students authentically into that, so when my students are out in the field… and I teach interdisciplinary studies now, so my students are pretty much not in the humanities, to be honest. I have lots of business students, lots of students working in marketing, lots of students in allied health, physical therapy; they’re all in my program, and the work that they do in our open textbook and with OERs is one thing and we do create all of our own materials for the program, but beyond that they’re also creating capstone projects that are generally online, often openly licensed; of course they hold the autonomy to make those decisions for themselves. At that point they’re pretty educated about how open they want to be and then they’re also working on applied projects out in their fields and I see all that stuff as part of the same way that we engage our students in the public world of knowledge.

John: And science is often taught, or at least it’s perceived by students, as this body of knowledge they have to memorize rather than this ongoing dialogue and a series of active research projects, and when they are more engaged in the process of making those connections it’s likely they’ll learn it better and they’re likely to become much more interested in the subject, because one of the main problems in the STEM fields is that students give up early on. But if they can see the relevance, I would think they’d be much more likely to continue onward.

Robin: Yeah, I think it’s a great point. I’m also the mom of a teenager right now who—is this gonna make my whole family sound bizarre? I’m not sure… but she’s got an obsession with taxidermy, so if you go into her room it’s all—I’m not joking; we’ve got boars, we’ve got bison, we’ve got deer, we’ve got every pelt you could imagine, and then her bookshelves are filled with skulls and bones…loves it. She’s out there digging for bones from the time she was little, researching which skull is this, what skull is that; she doesn’t like science, though; that’s what she tells me: she doesn’t like science. I’m like, you do like science, you nutty kid.

John: You’re kind of doing it; this is where a lot of science started.

Robin: That’s right, and so I’ve been waiting for her and she’s had great teachers here and there, but she really did finally have a biology teacher last year who helped her understand that she does love science, but before that she thought, I don’t really like these worksheets and I don’t really like memorizing these tables, and she’s an interactive person. So, I think there’s a lot of compatibility between open and active learning and experiential learning and high-impact practice and all these buzzwords. People call me an advocate for open, which I am, but really I’m an advocate for learners, like paying attention to the kinds of things they are constantly telling us that they need in order to be successful. While we’re over here shopping for some kind of software program, they’re sitting right here telling us, I’m hungry or I can’t afford my materials or I don’t feel like my voice matters or I don’t know enough to be useful here, so you just tell me what I need to know.

Rebecca: Or I’m not represented.

Robin: I’m not represented. That’s a huge one because when you transfer to this mode of learning, it’s a little bit the sort of Wikipedia model, although Wikipedia is a horrible example because of representation in Wikipedia and the stats we have on that, but the idea that you can pay attention to all the voices in your community but the open movement is really wrestling with this right now to figure out how much is about open and licenses and sharing and how much is about creating an ecosystem of inclusivity, access… the kinds of things that truly do shape a commons, which we mostly don’t have in education, so the commitment, I think, is for me is less to the technicalities of open and more towards the long-game vision, which is really about how do we bring more voices to the table to engage in the community for whatever the community’s needs are.

John: I wanted to go back to a point you made earlier; it reminded me a little bit about some behavioral economic studies and I haven’t thought about this before, but I think it’s relevant. There was some interesting experiments done by Dan Ariely a while back. Dan Ariely calls this the IKEA Effect and he notes that when Duncan Hines first started selling cake mix they sold horribly and the reason was you just added water, you stirred and you baked and people didn’t feel that they had created something, so they changed the mix so you had to add an egg to it and stirred and mixed in and baked it, but by the simple act of breaking an egg and mixing it in, it felt like they had created something in a way it wasn’t where they just added water. One of the experiments he did was he had people create these origami by following directions… of paper cranes and he asked them to evaluate how much they thought their creations were worth and then he swapped them and he asked them to evaluate someone else’s creation and people valued their own at roughly twice as high as the others across the board, and then he changed it in another iteration of this and he took some of the directions out… so it was really hard to replicate and objectively, when people evaluated the other people’s that time they rated the value of them much lower in terms of how much they were willing to pay, but because they put more work into building these things themselves, they rated their own creations much higher, and the simple act of creating something gives you this feeling of ownership and value that I think would be a useful part of this in terms of getting students much more engaged with the process and more engaged with their own learning.

Robin: Yeah, so I would say two things about that super fascinating set of stories that I’m totally going to use all the time, maybe tomorrow, when I speak with your faculty. So the first is that we run a customized major program where students create their own majors and the cake mix effect is enormous in our program; we have almost a hundred percent retention in our program, which I think is so very much attached to this idea that if you create it yourself that sense of ownership is huge. So, the one way I would revise your stories is the Duncan Hines model is kind of a slight of hand, right, you know, we could of just put the egg in there, and so that’s the part that I’m always wrestling with is this no hoops and mean it, so it’s actually kind of a big leap because in education we’ve known for a long time that we want to build student ownership, but we do a lot of tricks to kind of say, doesn’t this look like ownership, so how do we authentically do that instead of just sort of fake removing the egg and that’s where I think you see a lot of institutions push back—they are happy to make their students feel empowered, but they are not happy to empower their students.

John: That’s scary.

Robin: Yeah, it’s very scary and we talked about student-centered learning; when I started doing student-centered learning I was like, “Chumps, you are not doing no student-centered learning because once you center your students the whole course changes, falls apart a little bit.” It’s also the magic of tenure; it’s very risky to do a lot of the things that I’ve been blessed enough to be able to experiment with it, which have paid off, I think, hugely for our students, but there’s a lot of pushback sometimes from students and oftentimes just from institutional structures that can’t really accommodate learning that looks like this very easily, so that egg is kind of the whole thing there, right, and I love your metaphors; I’m gonna borrow them.

Rebecca: When you want to be authentic that also means that you have to be ready to completely change any plans; it’s like, oh, now we’re going down this rabbit hole that I didn’t know we were gonna go down, but I guess we’re going there and we’re all gonna go together and be open to that.

Robin: Some people are like wired for that, like my husband is a sculptor and that’s kind of the studio ethos, but so not how I came to teaching. I mean, I didn’t have my lectures written out but just super organized and when I would come to an epiphany it was always an epiphany that I had planned for many weeks… I think, “Oh my gosh, this reminds me of this thing, can you believe it?” And of course I knew all along that we were gonna be arriving at that epiphany, so when students would move in a different direction, even if I could tell at the time it was brilliant, you would have to pull them back to the place that you were going, but I have definitely changed my mode of operation because the content, really whatever you teach, it doesn’t exist in the world in 14-week packages, so the idea that you can’t do such-and-such because you’re going to miss this key fundamental thing is just bizarre when you think of the scope of knowledge, so I understand people wrestle with accreditation and we wrestle with standards and all these things are realities. But, for the most part, I think really radically meaning a lot of those buzzwords that we use is revolutionary. If you read your mission statement for your university and then you actually do some of that stuff, it’s gonna be crazy; nobody’s actually doing the things they say that they do, in my opinion.

Rebecca: A couple of weeks ago we had an episode about metaliteracy, which expands the idea of information literacy to include the idea of creation, so the idea of becoming more literate in the making of things as part of that information literacy process, which is clearly very connected to the idea of being open, especially when your students are creating this content and creating knowledge. The question that I have is one that I’m wrestling with currently as an educator who’s really about access as well, but I’ve been focusing a lot on access for people with disabilities in thinking about accessibility in that way, digital accessibility and learning those skill sets and where those come in and how do we make sure that things are visually organized and consistent so that an experience through these things that students are making is a good one for everyone who comes afterwards as well?

Robin: I am so happy that you asked that question because this has been my last three weeks; I have barely slept because I’ve started getting so excited. So, for probably the last year or so my own personal challenge has been to think about accessibility in terms of making our materials more accessible, so I’ve been learning about how screen readers actually work in order to fix my own syllabus to redo a lot of annoying things because I didn’t realize you had to use the headings to make things easier. So, I’ve just been learning that basic stuff and that’s been just a long, slow and interesting process. One of the last things that I really hadn’t learned about at all or hadn’t even really thought about was in giving presentations, which I give a lot, I had to think about slides… and so at Open Ed ‘18 in Niagara Falls… I wasn’t there, which is actually an important part of the story because one of the keynoters was Jess Mitchell, who is kind of a mentor of mine in terms of accessibility stuff—she’d be a great guest—and Jess gave a really moving and powerful keynote focusing mostly on inclusion in open and she is very much an accessibility advocate and what was amazing to me as someone who was not there and didn’t see a recording was when I looked at her slides afterwards I was able to experience really the whole keynote because they were designed to be accessible to folks who were in the room, text was organized in a certain ways and things were very clear and I came away really grateful for how she had set up these slides, which was interesting because they were really different than the kinds of slides I make. I had always prided myself on like “Robin made some fancy slides,” you know, they’re like just pretty and like visual impact and bold images, but because they were, I think, graphically designed in a lovely way, I mistakenly thought that that meant that they were actually accessible because they were clear in certain kinds of ways, but they weren’t, they weren’t set up well, so what happened was when I learned and saw in action some of the techniques that she was using, I started to look at this keynote that I was giving the other day—I had like two days left—and now the keynote was ready to go, all the slides were made; I looked at the slides and I was like, oh, crap, you know, no… So, I thought I’ll just redo these slides real quick, but what I ended up doing was really learning about the accessibility changed everything about how I approached the making, which actually ended up changing all of the ideas in the keynote in this dramatically productive way. So accessibility for me, of course, is not really just about like, oh, you have low vision or whatever; it’s very much part of this access broadly-writ idea… that openness… But beyond that because it’s built into how we build; it’s really about how we’re gonna design infrastructure and that is actually my passion right now; it’s less about making these materials—okay, so great, here’s an accessible material, great, they should be—but beyond that it’s about let’s just design an ecosystem now with access at the heart. So, in terms of accessibility, none of that sort of retrofitting one-by-one whatever, but also just what would happen with everything if access for the broadest array of learners was key. I was recently in Providence, Rhode Island at College Unbound, which is just a very cool program for adult learners completing college—they have to have least nine credits to start—but many of them have many more… and there are mostly students of color, mostly poorer students and mostly, I’d say, they seem like over 30 in age and they start their seminars with a hot meal and then after the hot meal they go into their different cohorted seminar rooms and tons of those students have their kids with them and the kids are just a normal part of the learning environment there and the whole place is designed around what kinds of access people needed, what times of day and what services in order to come here to learn and I just feel like everything about the content that we’ll produce and the ways we’ll set up schools and just everything will change if that’s how we build—we build around what I might call human beings, right, which is like the most innovative idea of all, right, it was not technology, it was humans.

John: Audience matters, as Rebecca is fond of mentioning on this podcast.

Rebecca: I almost did it earlier but I…

John: Well, I did it for you this time.

Rebecca: …I contained myself. [LAUGHTER]

John: But, It is important.

Robin: And it’s exciting, I mean, honestly, it’s just exciting because you do realize when you start thinking this way that it is again gonna change everything, right, you’re not just gonna put a caption on your video, it’s gonna be like every single thing is gonna change and that’s why it’s also important to say like, “Here’s how I still suck,” because you can’t just decide to do this and then be done. I’m just learning every single day, I’m messing up every single day and I think it’s better to kind of own that and think of it as a process, which is really invigorating.

Rebecca: To speaking about the process, how would someone get started? What advice would you give someone who is inspired to be more open in their process and the way that they teach and what they put out in the world? What’s the first step?

Robin: Well, the first thing I might encourage people to think about is what excites or interests you here? I think starting with a thing is not really the way to start. So, for example, a lot of times people will come down into our teaching and learning center (where my office is co-located—in the teaching and learning center). So, people will come down—“I need to start a blog with my students”—“Oh, okay, we can help you with that, why do you want to do a blog? “I don’t know; everybody’s blogging.” “Okay, we got to blog.” You really don’t have to blog; you could blog, we could help you, but I think having a sense of the goal: do you want to connect your students out to their communities? Do you feel like that would be valuable for your students? Would you like to lower some access barriers for your students? For me, there’s a lot of excitement that happens when I think about the hardships that we face in public education and trying to make a case for working in more public ways and what public work looks like, so I tried to start with what might excite faculty. So, you can do that on a one-on-one level or when I talk to large groups of faculty I usually start by helping them understand some of the implications of the high cost of textbooks, so if you just say to a faculty member, “That textbook costs a lot,” it’s too abstract. Usually they’ll just say, okay, this was 200 dollars and this one is $180; I’ve picked the 180 dollar textbook, I’m a good person… and they are… but showing them some of the data on what happens to students who can’t afford textbooks, and we have that data collected now and you can reach out to your librarians to access that data really quickly, talking about that with faculty and helping them see this as a social justice issue that impacts whether their students will pass classes, take credits, graduate from college, that I have found is persuasive, but then also talking about engaging their students in the world, really helping them to contribute rather than just consume, become better critical thinkers, all of those things are persuasive. Saying faculty don’t care about cost… I think first of all is not super true, but it’s also like we’re told all the time as faculty, cut costs. Cutting costs does not do wonderful things for learning most of the time. The things we’re asked to cut, especially in our public institutions right now. The age of austerity is decimating to innovation, in my opinion.

John: And the cost of textbooks has been rising at three to four times the rate of inflation for the last several decades?

Robin: Yeah, If you graph it out, I think the thing that I found most shocking was there’s the Consumer Price Index, you know, down below and then there’s the spiky line of the textbook cost and then if you map healthcare—it’s actually in between—it hasn’t been rising as fast as textbook costs, so I think people sometimes find that alarming.

John: Shocking, because that was also rising much faster than the inflation rate.

Robin: Exactly.

John: Going back to the issue of access, the students who have the most trouble affording textbooks often come from households where the parents have less education. Because there’s less early human capital development in those households, those students are already often starting at a bit of a disadvantage and many of them will choose either not to buy the book or wait as long as they can before buying the book. So, they’re far behind when they’re starting their classes and that would be a major factor in their retention on campus.

Robin: Yeah, actually some of this data that you’re talking about comes from the Florida Textbook Study in 2016, which is very persuasive for faculty, I think, but there’s some really new data—Eddie Watson out of Georgia, I believe, that just came out that shows that the benefits of switching to OER in terms of things like course throughput rates, grades and passing and…

John: the drop, fail, withdrawal rate, yeah.

Robin: …that the benefits are especially pronounced for students of color and for Pell eligible students. Some of our most vulnerable learners stand to make the biggest gains when they have access right from day one, and faculty recognize this when you talk to them about it because they are very used to having the small number of students in this side of the room saying, “My check isn’t in yet; I need to wait two weeks until I get paid,” or “I ordered a cheaper version that’s gonna be here in six to eight weeks” or whatever, so nobody’s surprised by it, but to realize that you are actually empowered to solve a problem in higher education is surprising to people and OER actually solves a pretty concrete problem and pretty quickly and the data shows us it solves it pretty well.

Rebecca: So what you’re saying is that OER is the gateway to open pedagogy?

Robin: Well, it’s so funny… [LAUGHTER] I have actually become maybe more famous in the community for saying the opposite because that is actually the party line: catch them with the OER and then show them the pedagogy, but as you’ll see in the faculty development talk that I’ll do here at SUNY Oswego tomorrow, I do that a little bit but definitely I think people are kinda like, “Okay, I’m in, yeah, sure,” and then you start talking about the teaching and learning and that’s when people really kind of come alive and then they shrink back again because they say, “Well, that’s you, because you’re techie and you’ve been doing this forever” because it looks overwhelming and I just want to tell them, first of all, I’m an early Americanist; there is nobody less oriented to this work than I was when I started, but I only heard about Creative Commons maybe like four years ago, like that was the first time I heard of it and now every single thing I do is related to this stuff. The learning curve is overwhelming at the very, very beginning, but the tools that you use and the ability to make these kinds of changes, especially if you do them incrementally. It is really within anybody’s ability and people should trust me when I say that because my husband is a sculptor—he’s a studio sculptor, teaches welding and that kind of stuff and he’s doing all of this now. So, he does OER, but he’s also doing lots of connected learning and his students have their own domains and he is somebody who for the most part does not really even enjoy email, so anybody can engage and I think we need good librarians and good instructional designers and we need to keep funding teaching and learning centers because paying big money to fancy software programs and outside contractors, these are sort of Hail Mary passes to save education. But, in my opinion, teaching and learning and instruction shows real benefits, but we don’t invest in it and we therefore can’t expect to get the full rewards that we could get if we were really focused on working with our faculty.

John: And a lot of the really powerful tools used in these courses are free, like Hypothesis, as you mentioned before. Do you recommend, for example, the use of Pressbooks for OER materials?

Robin: Yes, I’ve been very inspired by the Critical Digital Pedagogy folks out of Hybrid Ped and one of the things they talk about is analyzing your tools and I’ve been really trying, along with my work in accessibility—the other kind of learning curve for me right now has been trying to go through my own tools and gravitate towards not just free but open tools and that’s challenging in some ways; in other ways we’re all ready to go. I favor nonprofit companies like Hypothesis and Pressbooks is Open-source software; I use it through Rebus Community, which is a non-profit OER publishing community that’s developing now under the direction of Hugh McGuire, who was previously with Pressbooks and developed Pressbooks. So, I think the tools should not stress anybody out because the tools will be different next week, right? So, it’s not worth getting too worried if you’re like, this tool it makes no sense, okay, well wait till next week; they’ll be another tool, but it’s good to ask critical questions about if we’re really trying to not just save some cash but to maybe transform into more of a learning ecosystem that focuses on the public good, then we need to build infrastructure that has similar commitments to the kinds of content we might look at or the kinds of processes we might use in our pedagogy. That’s my goal now, is to transfer whatever I’m using into tools that have the same sort of investments that I do.

Rebecca: Speaking of infrastructure… We have infrastructure for students in teaching and learning the classroom kind of side of things, but we also need infrastructure to support faculty who want to be open and do open publishing and do this public good or public discourse methods in general. So what recommendations do you have for helping us move in that direction for public scholarship?

Robin: Yeah, but there’s a lot of myth-busting that needs to happen around open access publishing. Mostly faculty do have some pretty good autonomy, so the promotion and tenure processes that faculty will tell you, “I can’t publish in this journal because it doesn’t meet the impact factor regulations for my field.” Well, those are mostly coming from, like that old joke, “It’s coming from inside the house,” right? Really what this is is about faculty education to help faculty understand that it’s not in the best interest of faculty or knowledge to have the commercial publishing industry stranglehold on academic publishing, but of course faculty are concerned that there are quality issues, they think open access publishing sometimes is like, “I self-published this on Amazon” or whatever, so helping them understand that there are definitely low quality, predatory open access presses just like there are low quality, predatory commercial presses and helping people understand that what you’re really talking about is not whether it’s open or closed but what’s the peer review and what are you looking for in peer review. I think we’re seeing lots of institutions move towards open access policies that give faculty lots of autonomy in how they control their materials, but we need to do a better job educating ourselves about what’s wrong and broken in academic publishing right now.

John: SUNY has just introduced an open-access policy for the whole SUNY system very recently, and (at least at our institution) the upper administration, including the President, the Provost, and the Deans, have generally been very supportive, but it doesn’t always make it down to the departmental chairs and personnel committees and that’s a barrier that, as you said, we’re imposing on ourselves and it’s tough to get through, especially if you’re a junior faculty member coming up for tenure.

Robin: That’s right, and usually I tell administrators that I work with, it’s great that you’re supportive; please don’t tell anyone, you know, because we don’t want these to be top-down initiatives, they have to grow from the faculty and I don’t mean that again in the kind of Duncan Hines egg sense—like a fake way; it’s important that faculty steward the new era of academic publishing—that matters; that should not come from administration, should not come from state legislators. The state legislators are only too excited really to say, everyone must use OER. I went to our board of trustees and they were really happy to give us money for open and they said, we’re gonna pass a resolution that everybody needs to consider an open textbook—I said, thank you, I don’t want your resolution; I’m very grateful… Because it really is important that we do the education at the source which really is for the most part with faculty and actually with students, I think, is where it matters and we’ll grow it that way and the reason I have hope is that I’ve never talked with anyone for any length of time and had them say at the end, well that’s horrible and stupid. There’s lots of nitty-gritty problems to iron out and the open access community does not yet know exactly what the best path is for funding open access presses or all sorts of issues, but it’s very hard to find someone to say to you what you’re saying is horrible, so I think that we will see huge transformation in both OER and open access publishing in the next five to ten years, but we need to grow it with our people.

John: And some of the STEM fields have led the way there; the National Institute of Health and all their grants require

Robin: NASA, the White House….

John: …that things be publicly available and in public access.

Robin: Yeah, and of course those publishing models are a little bit different because there’s so much grant funding in science and the public has a right and that might be a little bit different than, say, a monograph by a historian. When we talk about open we always want to talk contextually, I think, and specifically about what makes sense for, I think, two groups: the public and the knowledge—thinking about both knowledge and users for every different example.

John: Where do you see open pedagogy as going in the future? It’s a relatively recent area and you’ve been very actively involved in this, but where do you see things going in terms of new and interesting directions?

Robin: I cannot answer that question because it boggles my mind… the question you’ve asked and I don’t think I can answer it and it wouldn’t help me to go away and think about it. I think what I could answer is where I hope things are going, and I feel very strongly that there needs to be a robust connection between open education and public education, and I feel like we are in a very dark time where our public education channels are being insidiously co-opted for private profits, and even in some of our public institutions you’re seeing the language of public just dissolve, so we’re seeing college presidents saying, “Yeah, that’s over, that era is over; we’re not going to get any more public dollars; it’s dried up; we need to get corporate money, we need to do partnerships, we’re gonna fund ourselves in these new private ways.” I think this is our chance to intercede in what I see is a very downward trend and I think open education has some really incredible possibilities for helping us articulate what public practice looks like and if we can articulate what public practice looks like, the fact of the matter is, and I do not think this is an argument, I think it’s a fact;—it’s true—I really think it’s true that it is in the public’s benefit for the public to fund the kinds of public work that we’re talking about here for both students and researchers and if we could help explain why by being a little more coherent for ourselves with what it means to work in and at public education and research, I think we’d have a better chance at making that case for the public. So that’s where I’m hoping to take all of this is to say really what we’re talking about here is a resurgence for public education at the K-12 level, resistance to the charter takeover and higher ed to say it’s time to reclaim a public mission for our public universities and fund them appropriately and realize that innovation comes from people and not from private, gated… Right now, the idea is that all things innovative thrive in the market and I think that’s because we’ve been intentionally starving and strangling our publics. Wow, that was like really radical.

Rebecca: That was good, yeah.

John: And it’s an important message.

Robin: I think it’s coherent and I think it’s persuasive; I feel people come alive when I talk about it, but we need also some national leadership on this both from inside education and inside government and I don’t just mean nationally,—the United States is in a squalid mess right now, which it is—but just even in the Obama years and whatever like who are our champions for public, where is that coming from? I’d like to see more mentors and like to see our college presidents use this kind of language—if you’d like to hire me to be your college president… [LAUGHTER] Call me. [LAUGHTER]

Rebecca: Maybe that’s a “what’s next.” [LAUGHTER]

John: So, we usually end these podcasts by asking, what are you doing next?

Robin: Tomorrow, I will be here at SUNY Oswego. You know, the question of next is a really hard one; My own personal life has been changed so radically by this. I never saw myself leaving the English department, I never saw myself having a whole in some ways second career. I used to be asked to be department chair because it was your turn, not because I was anything special, and I would go under the table, you can’t make me do it. I declined everything. I really think, though, people with a grassroots passion for doing this work need support at higher levels in higher education. In terms of me personally, I started thinking about trying to step into some of those roles and I can’t say I feel sort of super personally excited about some of the aspects of that work, but I know that even though I see this as a grassroots movement,—and I do use that word—it’s really hard to change institutions, and in order to do it we’re gonna need to get people at every level to care about these kinds of things and so I’m inspired by people like Tressie McMillan Cottom and Sara Goldrick-Rab and they’re faculty, but they step out to set a national example, and I’d like to maybe think about trying to move this stuff a little bit more institutionally, as opposed to just inside of programs or with particular faculty development events. I’d like to see some institutions really step out and lead. SUNY is doing a great job. You guys have about 48 of your 64 institutions, I think, actively engaged and you are careening towards some system-wide impacts, partnerships with CUNY, statewide conversations; this is where I think things really get exciting to me.

John: The community college and SUNY have really been leading and they’ve been very active in doing this. The four-year colleges have been moving, but not quite as quickly and the university centers have a bit more inertia. So, SUNY has been making some really great efforts in providing incentives and doing a lot of encouragement and the workshops they’ve been funding have helped to try to get more grassroots movement, but it’s not as quick as many of us would like, but it’s much faster than it was a few years ago.

Rebecca: Incremental change is still change?

John: It is.

Robin: It absolutely is. Someone was telling me… Is this an economics thing about the parable of the ant, that ants are going up a hill? Okay, somebody on Twitter, you just sent me this,—I’m losing my brain now—but anyway, an ant is going up a hill and when because of the position of the ants eyes they can’t assess the whole hill, so all they do is at every point they could assess, I want to get to the top of the hill, and all they can assess is, okay, this is the next step that I take, so then the ant gets to the next step and it assesses again and that’s the kind of way incrementally the ant will get up. In that sense the ant doesn’t really even have to know where it’s going; it is just able to constantly resurvey and take one more step and I found that really reassuring when someone sent that over to me today. That’s kind of a metaphor for how you can keep going when you don’t always know exactly where you’re going, and also to your other point, community colleges are clearly the national leaders in this work and I find that really good for education because community colleges are actually really good at teaching and learning and it’s important to look at what they’re doing with open to learn our lessons, but also we can just learn a lot from partnering with our community colleges more effectively.

John: And they’re often the first point of access for first-generation students who may find it difficult to go directly into a four-year college, and they have many of the students who most need that sort of access.

Rebecca: Well, thank you so much for spending time with us and engaging us in this really great conversation; I hope that incremental change becomes much bigger increments as we hear more people and more people get on board.

John: And if you get one person in department doing it, it’s a whole lot easier to convince others to try.

Robin: There’s no secret trick or no secret sauce, it’s just people, so every time somebody as a human gets invested you actually get a lot closer to where you’re going, I think. It’s exciting, it’s exciting, and thank you guys for having me because this like fancy stuff and I feel very listened to and I’m gonna put all sorts of pictures on Twitter of myself in front of these microphones. [LAUGHTER]

John: Okay, well thank you.

Rebecca: Yeah, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.

[MUSIC]

49. Closing the performance gap

Posted in active learning, assessment, collaboration, evidence-based practices, growth mindset, higher education, inclusion, instructional communication, learning, pedagogy, peer instruction, scholarship of teaching and learning, teaching
October 3, 2018

Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, Dr. Angela Bauer, the chair of the Biology Department at High Point University, joins us to discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

Show Notes

  • “Success for all Students: TOSS workshops” – Inside UW-Green Bay News (This includes a short video clip in which Dr. Bauer describes TOSS workshops)
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc.
  • Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative learning techniques: A handbook for college faculty. John Wiley & Sons.
  • Life Sciences Education
  • Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.
  • Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6), 613.
  • The Teaching Lab Podcast – Angela Bauer’s new podcast series. (Coming soon to iTunes and other podcast services)

Transcript

Rebecca: Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, we’ll discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Angela Bower, the chair of the biology department at High Point University. Welcome Angela!

Angela: Thank you John! Hello Rebecca

Rebecca: Hi, welcome!

Angela: Thank you

Rebecca: Today’s teas are…

Angela: I have a piping hot mug of Earl Grey

Rebecca: Yes! Someone drinks tea again! [LAUGHTER]

John: I have ginger peach green tea.

Rebecca: And John get ready for this, Dragon Oolong tea

John: That’s actually very good.

Rebecca: We have a joke that I have my standbys and this was not one of them.

John: We’ve invited you here to talk about your work on reducing the performance gap for underrepresented students in STEM disciplines. Before we discuss this could you tell us a little about the performance gap?

Angela: You bet! Well, I can tell you about my experiences at two different institutions, although I can say that my experiences at these two institutions with respect to the performance gap in foundation STEM courses is not unique to these institutions. It’s a natural phenomenon that if we look at foundation courses that are taught with a traditional lecture format typically, which still is often the case with STEM courses. We see a gap in the performance between majority students and then students from underrepresented groups and this was the case when I taught biology courses at the University of Wisconsin in Green Bay. If we went back and looked at the data in a foundation course in Introduction to Human biology course that we had taught for decades, we could see prior to a change in pedagogy in terms of what was happening in that course, we could see for decades prior a gap in the performance between majority and underrepresented students at that institution. Likewise the same is true at High Point University with the way that we used to teach our introductory biology courses. And this performance gap is really troubling because often times what it means is that the students in those courses that are not performing well, we tend to lose them and in the end we end up having a much lower percentage of students who are members of underrepresented groups choosing to major in the sciences and then of course choosing to pursue professions related to the sciences as well.

Rebecca: Where does this performance gap originate?

Angela: Well that’s a really interesting question and there are a lot of misconceptions about where that performance gap originates. I’ll go back to when I first started teaching. What I assumed which was entirely wrong was that the academic performance gap was an indication of a difference in the academic preparedness of students from underrepresented groups. So I like many other faculty in my field assumed that students were coming to our institutions not having a high school education that prepared them to perform well in these introductory science and math courses. I later came to find out that this was not at all the case. We did a study at University of Wisconsin in Green Bay where we controlled for let’s say SAT scores of these students coming in; so if we looked at maybe white students and black students who came in within the same range of SAT scores specifically their math scores which is a strong indicator of how well they will perform in the course and if we looked at how they performed in the course even when we controlled for SAT Math score, we saw a performance gap. So that suggested to us (and this was the big lightbulb moment for me) this doesn’t have anything to do really with academic preparedness at least with respect to our student population. What it has to do is something that we’re doing in the classroom. We really were teaching in a way that favored the academic performance of students of a particular group or did not enhance the academic performance of our underrepresented minority students and so that’s when we decided to make some changes.

Rebecca: How did you start to figure out what to pinpoint?

Angela: Well interesting how we stumbled upon an approach that worked at Green Bay. We had a variety of instructors teaching this introductory course, so we had multiple sections going on at a given time and as you know is often the case to change what’s happening in the classroom to get instructors to adhere to a different pedagogy; who are often swamped and busy and maybe have taught the course for a certain way for a number of years and are not comfortable with moving to for example a more student-centered format with respect to their courses (which by the way is pretty daunting when you have lecture sections of 150 students). So what we initially decided to do just to see if it would have any impact was we started to do things outside of the classrooms in a way that was very student-led and student centered. Students who were enrolled in this introduction to human biology course in Green Bay were provided with the opportunity to attend workshops outside of the classroom that were totally optional and we call them toss workshops TOSS, which was an acronym that stood for targeted opportunities for success in science. And we chose students to serve as mentors for these TOSS workshops who were science majors who had at least received grades of B or better in these introductory courses, but who had really strong social skills. Then I worked with those student mentors to train them in student-centered pedagogies and also a culturally inclusive classroom approaches and then they would offer these tops workshops at a variety of times during the week when we were sure that we would get a number of students who could attend. In other words we tried to make sure that those TOSS workshop opportunities didn’t interfere with their academic schedule and these weren’t required and we went about recruiting students for these workshops in a really interesting way. We conducted this in collaboration with a man named Dr. Sean Robinson who is now on the Whitewater campus in Wisconsin, but he at the time was working within our Multicultural Center and he worked really hard to recruit underrepresented students who were his advisees in the Multicultural Center and spread the word. Likewise he helped us recruit a fabulous mentor for our students Mr. Junie Lee who was a grad of our program at Wisconsin, who was also very good at networking with these students and we made it very widely known and it became a popular fun thing to do. Because the students who led the toss workshops were very socially engaging they planned fun activities to review the week’s content and it certainly became a destination for these students and it created a really strong sense of community. And over the four years that we ran this TOSS programming what we found from the very first semester that we offered these opportunities outside of the classroom was that we closed the gap and that happened every single semester for eight semesters across those four years. Multiple sections even with different instructors in different pedagogies in these courses having that opportunity to participate in TOSS workshops totally changed the culture and we closed the gap. And so what we learned from that was that a sense of community was I think probably key here. We know this because (well we assumed this) of an interesting analysis that Dr. W Furlong did, who had worked with our office of institutional research in Green Bay. We looked at the student mentors that led the TOSS workshops and we looked at the activities that they had planned for their TOSS workshops in a given semester and I also ranked the students then in terms of the level of difficulty of the exercises that they performed. In other words how academically rigorous were those workshops versus how much social stuff is going on and we actually found an inverse correlation with the academic rigor in the workshops and the impact of the workshops themselves. So in other words when those mentors were just more social and creating a strong sense of community and spending more time just engaging students generally not necessarily reviewing academic content, that’s when we saw the most profound effect. So this is what led us then to really now focus our efforts with respect to closing the performance gap on really the more emotional components of learning, the affective domain of learning. And yeah of course I’m not at all saying the cognitive part isn’t incredibly important because absolutely it is, it would be silly to say that it wasn’t but generally what we found in Wisconsin and now also what we’re finding at High Point University is that we can close that gap by paying really close attention to that effective emotional component of learning and taking really intentional steps to just create a very inclusive community for students and to engage them in very intentional ways.

John: Have there been changes in classrooms as well to try to build the same type of inclusive environment within the classroom setting?

Angela: Yes. So once faculty learned that the impact of TOSS workshops in Green Bay, more and more of them started to pay more attention to what inclusive classroom approaches looked like. and then at High Point University now where we have also done some interventions that have been really successful in closing that performance gap. We now are employing a model where all faculty who teach our introductory biology courses are on board with employing these techniques that intentionally address the affective domain of learning.

John: What techniques did you use at High Point to address the affective domain?

Angela: When I came to High Point University five years ago, of course we pulled up some of the institutional data, the historic data regarding performance of students in our introductory biology courses and of course saw that performance gap, a long-standing one. And so we really tried to build a model that built upon the knowledge I gained in Wisconsin about the importance of building community and addressing the affective domain of learning, we then attempted to employ that uniformly at High Point University. This time in the actual courses themselves and not with these optional outside of the class activities, so at High Point now we’ve got all instructors on board employing approaches in the classroom where they then administer to students our weekly messages that address the affective domain of learning and it involves growth mindset strategies, which is based on the work of the Stanford psychologist Dr. Carol Dweck who wrote a book on growth mindset strategies. And what we are doing at High Point University is providing students with weekly growth mindset messages maybe 5 or 10 minutes at the start of the course on Monday or Tuesday. I prepare maybe two or three PowerPoint slides with a growth mindset message that I send to all instructors before the start of the week and then they show that message to students and they reinforce the message throughout the week. And just to review what it means to employ growth mindset strategies; if a person has a growth mindset they believe that with hard work and persistence they can improve at whatever it is that they are doing. If a person has a fixed mindset they believe that essentially we are how we are and maybe you’re born a science person or maybe you’re born not a science person, maybe you’re a math person, maybe you’re not a math person and so students that have a fixed mindset will very much back away from things that challenge them because if they engage in an activity that challenges them and maybe they don’t perform so well with respect to that academic opportunity, they view that as a negative commentary on their intellect and who wants to publicly out themselves as being not smart or not good at something, so they will shy away from that and of course that has a very negative impact on their performance in the classroom, they don’t seek out the help that they need, if they vomit test they don’t go in and figure out what went wrong in that test. By sending them weekly growth mindset messages we’re telling them we believe you hang in there with hard work and with really smart use of resources that are available to you and figuring out what an effective study strategy is for you, you’re going to be successful in this course, we think you can do it now let’s get down to work. And long story short those growth mindset strategies have been very powerful in helping us close the performance gap, also now at High Point University in our introductory biology courses.

John: What would be some examples of those messages?

Angela: I would send out a variety of messages the ending on the week, some of them were science related and some weren’t. Maybe there was an interesting study that I read over the weekend on neuroplasticity and so of course being biology majors they would have an interest in that you can literally change your brain with practice. There was one study I sent a couple of slides showing MRIs of the brain in people who had intentionally performed or intentionally practiced a video game, it was a race car game where they practice going around a particular track for maybe two hours worth of practice and these MRIs showed literally their brain was changing in response to those two hours of practice so that was one example. I chose things outside of the realm of science and maybe one week we had a couple of slides on NBA superstar Michael Jordan and the fact that he didn’t make his high school basketball team, he got cut from his basketball team but again with hard work and persistence and employing the right strategies he was arguably the greatest basketball player of all times. We have a really funny clip from a Nobel laureates Junger report card from back in his grade school days about his teacher saying you’re never going to be any good at science, you just need to give it up and later he went on to win a Nobel Prize for his work in cloning. These are just a few examples of growth mindset messages that we sent to students on a weekly basis.

Rebecca: Have you noticed a difference between the in-class strategies of growth mindset versus community-focused TOSS workshops that you had done at Green Bay?

Angela: Both have been effective in closing the gap. They’re two very different academic settings, so one thing that the TOSS workshops were really great for in Green Bay was really fostering a sense of community among a group of students, many of whom were commuter students. And so they’d go to classes and then they’d leave the campus, they vacate the premises and we would never see them hanging out and studying up by our offices or engaging in social activities that involve their peers in science and with the implementation of these TOSS workshops we saw much more of that, we saw students hanging around after hours talking more with their professors. So I saw a significant change in our sense of community particularly with our underrepresented minority students in Green Bay as a result of those TOSS workshops. It’s I would argue much easier to address the community issues in certain respects at High Point University because we’re a private institution, students are required to live on campus, there are all sorts of social activities to engage them all the time. Now whether or not they’re feeling a strong sense of community or developing a strong sense of trust with the faculty that teach their science courses or feeling a sense of belonging I think what we’ve done with growth mindset strategies is heighten all of that with our High Point University students. So they’re two very different models, I think they are very well suited for those very specific institutional contexts. I have to tell you that when I first got to High Point University I tried the TOSS workshop model and nobody came because it was optional, it was outside of class at a time where quite frankly there were lots of other social events happening on campus, extracurricular activities, students are involved in so many different clubs on campus so that just was a model that wasn’t going to fly here, we needed to do something specifically in the classroom to catch those students when we had our captive audience.

Rebecca: It’s really fascinating to hear those differences, but also recognize that you need to adjust your design based on context.

Angela: I think context is a huge deal.

John: Did you make any other changes at High Point besides adding the growth mindset prompts?

Angela: What we did within the classroom was more than simply take these intentional approaches to address the affective domain of learning through the use of growth mindset strategy. So we also very intentionally changed the way we taught our foundation biology courses by also making sure that we were employing best practices to address the cognitive domain of learning. Prior to when I had come to High Point University lecture was the primary mode of teaching in our introductory biology courses and there was a lot of turnover that happened when I arrived, there were curricular changes, the hiring of many new junior faculty just in response to the growth of our program and as we hired these new faculty we were very intentional in training them in approaches that would involve active learning in the classroom. And we worked very hard with the instructors that teach multiple sections of this introductory biology course to standardize our curriculum in a way to make sure that students were getting exposure to active learning, best practices for addressing the cognitive domain of learning in addition to those that address the affective domain of learning with the growth mindset strategies. And what we found one year when we ran simply the active learning approach where we covered at least 25% of the material in those introductory biology courses with active learning strategies, but didn’t employ the growth mindset approaches. We still, with some of our underrepresented groups, still saw a gap— a performance gap. So again, while some studies have shown that active learning can help to close that gap or narrow that gap, we weren’t very successful in doing that at High Point University. Active learning alone didn’t do it for us and I think that’s an important point to make for whatever reason. It could be our institutional context, it could be a lot of different things, but my point is when we combined active learning with approaches that also address the emotional or affective components of learning, that’s when we hit the sweet spot. We needed to do both of those to close the performance gap.

John: What specific active learning strategies did you introduce?

Angela: Active learning strategies that are chosen by the instructors— they are individual. All of them use student-centered discussions of the primary scientific literature so that’s one of the outcomes of that introductory biology course. For us to introduce our first-year students right away to the primary scientific literature, so that’s one approach I know. Instructors, we primarily leave it up to them to choose what they want to do when we have people join our program or begin teaching our introductory biology courses. We provide them with that Bible of students under learning techniques called Collaborative Learning Techniques written by Elizabeth Barkley and her co-authors. They get that and I know a lot of them use that as their go-to manual if they want to set up an active learning strategy to cover a particular topic on any given day. They’ll employ maybe just simple think-parent shares or something more elaborate than that. That’s really up to the individual instructors to do it but we just ask that they spend about a quarter of their time doing that. We also revamped our laboratory sections associated with that course so that they are also much more student-centered. Now someone argue that laboratories by nature are active learning— I would argue that they’re not in the traditional way that they’re taught. Traditional labs are very cookbook in nature and performed in a way I would argue that don’t really actively engage the students brain. You come in, here’s what you do, settle this up and here’s what you should expect to see— that’s not at all what we’re now doing in these introductory biology courses. We are infusing scientific inquiry into pretty much every aspect of that course. Yes, covering the technical skills that they would need to be proficient cellular and molecular biologists but they’re also expected to come up with hypotheses to test and then use those technical skills as they design experiments, test their hypotheses, analyze their data, etc. So the labs also are very student-centered and, by the way, much more than 25 percent active learning happens in those labs— the majority of what happens is active learning. So when all is said and done, with the implementation of those active learning strategies along with their growth mindset messages, we’ve had a lot of success in enhancing the performance of our underrepresented students in closing the gaps.

Rebecca: One of the things that you mentioned in passing was training your faculty when they come in in active learning strategies and I imagine also these affective strategies as well.

Angela: Yes.

Rebecca: How do you employ that? Is that through a teaching and learning center on campus? Is this through your department? How does that work?

Angela: That is a good question. So when we had the addition of so many new faculty to our program— so one year, we had three new faculty. The next year, we had five new faculty. The year after that, we added two more. That’s huge turnover and again, reflects the growth that was happening in our program at the time. One thing that I did was I established a journal club— typically science departments have journal clubs where people present their data that they’re creating in lab or maybe they talk about a scientific paper. I set up a teaching and learning journal club where every other week, we would get together and talk about our recent publication in the scientific teaching and learning literature. And the new faculty who were coming in were really eager to learn this stuff because they wanted to improve their student outcomes and they wanted to be on board and teach in a way that was equitable within their classroom. So that was very helpful in terms of sharing with new faculty the findings regarding the evidence-based literature about the impact of these teaching practices on their student outcomes. So that was a really effective way to do it. Then that was the first year or two when everyone was on board and then, as happens, once people get their research programs established and then they start getting on committees, it became less and less likely that we would have a good turnout with our teaching and learning journal clubs. So my latest trick now is I just started a podcast that’s really focused on teaching in STEM and I’m calling it The Teaching Lab and my goal is to interview an innovator in STEM education every week or two who can share something about a recent publication in the teaching and learning literature that I can then share with my faculty. So it’s sort of an on-the-go professional development opportunity for them when they’re out what can their dog in the morning, they want to listen to a podcast or driving home at night because it’s becoming harder and harder as they establish themselves and now have their research programs up and running and they’re wanted in a million other areas on campus, it’s less likely that we will get together for this teaching and learning in the sciences journal. We call it our TLS Journal Club. We still have lunch together at least a couple of times a week and just bat ideas around and it’s a really wonderful community of colleagues I have here and I’m so fortunate in really having that journal club earlier on set the tone but now I’ve got to come up with other tricks because of the changes that have happened.

Rebecca: Looks like we’re all looking in the same bag of tricks, right? [LAUGHTER]

Angela: Well, it makes total sense.

John: Yeah, the asynchronous nature of podcast makes it a whole lot easier for people to fit it in their schedules.

Angela: Yes, and it’s easier then to sort of pick and choose. If there’s one podcast that eh, not so much use to me or of interest to me. I guess that’s probably more socially acceptable to skip listening to a podcast then just skip coming to a journal club and feeling the guilt associated with that.

Rebecca: You’re probably right. [LAUGHTER] What resources might you suggest for other faculty who want to work on reducing the performance gap in their classes?

Angela: It’s a good question. Just keeping up with the teaching and learning literature in the sciences. Life Sciences Education is a great resource that publishes a lot of evidence based up with respect to inclusivity, closing the performance gap, etc. I recommend Carol Dweck’s book on growth mindset to everybody and just going back and reading the classic literature by Claude Steele, the Stanford psychologist who did the original work on stereotype threat. Going back to those classic studies and looking at how profound the impact can be of stereotype threat, if you activate the threat of stereotype in a student, maybe it’s pointing out that women typically don’t perform as well as men in a math class. If an instructor just makes a statement like that before handing out an exam, it can have a profound impact on the performance of students. Just going back to read that classic literature about stereotype threat published by Claude Steele and colleagues and really thinking about how things we do in the classroom, totally unintentionally, can just have such a profound impact and how we really need to think about our messaging, and what we tell students, and the importance of building trust, and telling them that we believe in them, and also telling them you gotta work hard too, you know? And here’s how we’re gonna do that, here’s how we’re gonna progress this semester together, and here are the resources we can provide to you. Those are all wonderful resources, I think, for informing our teaching and really keeping at the fork front of our mind how incredibly important it is to address those emotional components of learning.

Rebecca: What are some of the strategies that you’ve used personally in your own classes in response to the work that you’ve done to improve inclusivity?

Angela: So I make sure, always, to be sending growth mindset messages to my students. If they don’t do well on a lab practical or don’t do well on an exam, I make sure to let them know that I believe in them and suggest to them alternative strategies or approaches that can help them to be successful. I work really hard, just generally, to foster a strong sense of community in my classroom, to invite students to come to my office hours even when there’s nothing wrong. Just come and hang out with me and let me know how your life is going. Fostering a sense of community is really important to help students feel like they belong. I try really hard to include in my classroom examples in discussions that are really culturally inclusive and that would reach a variety of learners. Right off the top of my head, those are things that I try to do on a regular basis. I try really hard to bring in speakers from underrepresented groups— scientific speakers from underrepresented groups because often, still, it’s a national phenomenon. We still see underrepresentation of certain ethnic groups within the sciences for sure. It’s less the case at least at undergraduate institutions that women are underrepresented. We’re seeing more and more women now in STEM disciplines, especially teaching at undergraduate institutions, but still we respect to scientists of color— very hard for our underrepresented minority students to find mentors who’ve had the same experiences that they’ve had. So while we work hard at High Point University to recruit faculty from diverse backgrounds and we’ve had some success in that regard, we could do better but we’ve had some success. We also try really hard to bring to campus scientists from underrepresented groups to meet with our students, to serve as role models for our students. That was very much the mode of operation for me in Wisconsin. I got some grant funding to make that happen in my students, who were able to interact with those scientists and be mentored by them, they still speak years later about what a significant impact that was on them in their life. So one example was Tyrone Hayes, who’s a very famous researcher at Berkeley, who is the guy that discovered how the pesticide atrazine is causing malformations and sex reversal in frogs and so forth. He is an African-American scientist who we had come and meet with our students in Wisconsin, talk about his research of course, but also just spend a lot of time during his visit hanging out with students and sharing his experiences and that really had a profound impact on them that they still talked about to this day.

Rebecca: Sometimes it’s things that seem so little to us that are so big to our students.

Angela: Absolutely.

Rebecca: We had a lot of interesting and exciting things going on but we always ask: what’s coming next?

Angela: What’s coming next? [LAUGHTER] I have a lot of data that I’m analyzing right now about the impact or growth mindset strategies on men versus women. I want to see if there’s a difference in responsiveness to the messaging in our male versus female students so there’s that. We’ve also implemented some growth mindset messaging in our introductory chemistry courses and we’re looking to see if they have the same impact in the chemistry courses as well, which is enough for now truthfully.

Rebecca: It seems like plenty.

Angela: Beyond that, you know we now have adopted growth mindset approaches as our campus quality enhancement proposal. So I made a pitch to our faculty when it came time to choose a new QEP topic that growth mindset strategies would really be an effective approach for addressing some of the problems that faculty/staff had identified that they encountered with students that frustrated them that they would like to see some approach adopted at our institution to address. Like students shying away from challenges or not really digging in when they were bumping up against obstacles within the classroom. Growth mindset strategies are incredibly effective for changing the way that students approach problem solving in the way that they feel about failure or challenges. It really helps them to be more resilient. Messaging itself is incredibly effective, we knew that that was the case in the k-12 setting based on Carol Dweck and colleagues work but now we and others are finding that in higher ed classrooms, likewise, they have a profound effect. So the growth mindset strategies and the messaging and the impact on a variety of student learning outcomes are now being employed High Point University campus pretty much universally in a variety of different disciplines and even within student life.

John: Well thank you, you’ve offered some wonderful suggestions that I think would be useful not just in the STEM fields, but in all of our disciplines.

Rebecca: Definitely. Thank you so much for your time and great research that you’re doing. I look forward to hearing what else you find.

Angela: Thank you so much for having me Rebecca and John.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

36. Peer instruction

Posted in active learning, classroom management, evidence-based practices, higher education, large classes, learning, mental models, metacognition, pedagogy, peer instruction, teaching
July 4, 2018

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.