65. Retrieval Practice

Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, Dr. Michelle Miller joins us to discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Her academic background is in cognitive psychology and her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications.

Show Notes

  • Miller, M. (2014). Minds Online: Teaching Effectively With Technology. Cambridge, MA: Harvard University Press.
  • Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968.
  • Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 1199327.
  • Kahoot
  • Bray, Niki (2018). 43 to 0: How One University Instructor Eliminated Failure Using Gamified Learning. Blog post
  • Retrievalpractice.org
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Pennebaker, J. W., Gosling, S. D., & Ferrell, J. D. (2013). Daily online testing in large classes: Boosting college performance while reducing achievement gaps. PloS one, 8(11), e79774.
  • Pauk, W. (1984). The new SQ4R.
  • Thomas, E. L., & Robinson, H. A. (1972). Improving reading in every class. (a discussion of PQ4R)

Transcript

John: Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, we discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today we’re welcoming back Dr. Michelle Miller. Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications. Welcome back. Michelle.

Michelle: Thank you so much. It’s so great to be here today.

Rebecca: We’re so happy to have you again. Today’s teas are:

Michelle: Well, I’m drinking Coco Loco, which is a blend from a local tea shop here in Flagstaff, Arizona. Steep Leaf Tea. And Coco Loco is a lot like what it sounds like. It’s chocolate and banana. So tea snobs may scoff at my choice, but it’s wonderful.

John: IIt sounds good.

Rebecca: And I think I saw a nice silver teapot that was poured into a green and blue tea mug.

Michelle: Yup.

Rebecca: A nice tall one.

Michelle: That’s what I need.

John: And I’m drinking ginger, peach green tea.

Rebecca: I went with a Christmas tea today. So Michelle, we invited you here today to talk a little bit about retrieval practice. Can you first start with defining that for us and letting us know what it is?

Michelle: Right. So retrieval practice is essentially the act of pulling something out of memory. So that is, in memory research, what we would term retrieval. So something is stored in memory and we want to pull it out so we can actively use that information, have it in our conscious minds and so forth. And so we go through this usually very fast process called retrieval. So retrieval practice is specifically the act of doing it, and we contextualize that with learning. So, when I’m trying to learn something or I’m the process of learning something, to say, “Oh, what was that fact I remembered, or what can I say about this?” When we do that, it produces something that we can call the testing effect. So this is kind of the clearest example… not the only… but the clearest example of retrieval practice in action during learning is when we sit down to take a quiz, take a test or something like that. So all the excitement that’s happened around retrieval practice in higher education, and really in the rest of education today, is around this finding, which has been replicated many, many times: that tests are, as one person put it, not neutral events in learning. When we take a test on something, that has a very powerful effect on our ability to remember it in the future. So, really simplified down to its core, tests help us remember in the future; when we take a test it strengthens our memory. So that’s what retrieval practice is and it can, as we’ll maybe talk about today, take many, many forms in learning settings. And I did want to clarify too, this is something that I definitely don’t want to take any kind of credit for discovering this. This has been around and has been known about for a long, long time. Some of the big names who are associated with this: Jeffrey Karpicke, Robert Bjork. Roddy Roediger, there are quite a few really heavy hitting cognitive scientists and cognitive psychologists who have established this. But there are many, many of us who are out there trying to disseminate this to other teachers around the world so that we can all tap into the power of this. And I have done a little bit of work in this area with my colleague here at Northern Arizona University, Laurie Dixon, who’s another psychologist… and we teamed up some time ago to look at a very practical implementation of retrieval practice in an Introduction to Psychology course that we conducted some years ago and this is a course, you can imagine, where just trying to get students to perform even a little bit better is a big project. So we examined how even something kind of basic… it was very high tech at the time… but just basic web quizzes that came packaged with the textbook. We said, “Well, if we actually assigned students to do these as part of the course, and if they went through and treated these as opportunities to learn, not just assessments in and of themselves, would that have any systematic impact on course performance?” And we found that in fact, there was a significant improvement associated with that. So that’s kind of the landscape of what retrieval practice is, and why we’ve been so interested in discussing this in the psychology of teaching and learning.

John: In fact, I saw you present on that about 11 or 12 years ago in Orlando at one of the NCAT conferences and it convinced me to completely revise how I was giving my classes and it’s made a big difference and resulted in some significant improvements in student learning. Given that we know so much about retrieval practice. Why are faculty resistant to doing this?

Michelle: Wow, that’s a great question, and that is one that I have been really facing a lot these days in my own practice, talking to other faculty members, and disseminating this through some different activities I do in this area. It is easy for those of us who work in this area, cognitive psychologists in particular, but a lot of us who like you heard about this a long time ago. We’ve seen the power of it. We forget that to other faculty, this can be a very off-putting concept. And so it’s really great for us to think about why that is. And I always think that’s really good for me to kind of go back to that and say “Yeah, not everybody is sold on this idea. And there are good reasons for that.” So like with a lot of things that we talk about in teaching and learning, I think that these really break down into two neat categories: there’s the philosophical issues that people have with it, and then there’s more practical and logistical issues with it. So kind of tackling those one at a time. Philosophically, when people say, “Yeah, I understand about the research but this just goes against something that I believe as a teacher or how I want my classes to be. Here are some ways that that can play out. First off is this idea, that I’ve heard in one form or another quite a few times, and that is this notion of superficial learning. So “Okay, sure, there’s a study that showed that maybe people retain something a little bit better. But surely that’s not this deep learning, whatever that is.” That’s a concept that we all want. So do tests and exams… just testing… create just a superficial form of learning? And while, of course, I understand that, and I absolutely applaud faculty for really thinking deeply about that issue, and caring about it. I Here’s the thing… we got to define that. We social scientists, that’s what we do… we have to kind of break things down and say, “Okay, what does deep learning mean?” and I don’t know that anybody has kind of definitively done that. But when I look at that, I say, “Well, this is not just one research study that showed a little improvement in a lab test. There’s quite a few studies that do use realistic types of materials. It’s not all just contrived laboratory studies. Furthermore, there’s also studies that show that when students engage in more quizzing and testing on material that they actually are able to transfer that learning better. And that is a very, very big deal in teaching and learning as a lot of us know is not just getting students to be able to solve a problem in one context or work a concept in one context, but can they do it in the next circumstance? And that very difficult process is aided by quizzing… and to me, what could be deeper learning than learning that transfers? So that’s part of it. Some of it is perceptions around multiple choice quizzes and tests. There’s an assumption too that if we’re talking about quizzing we must be talking about multiple choice questions… and first off, sometimes in larger classes, those are the final assessments… in that Introduction to Psychology course that we studied years back, that’s what the assessments were… so, having students practice in that format I don’t think that we should necessarily dismiss that. And as we can talk about in a little bit, there’s lots of ways to induce retrieval practice that actually don’t involve multiple choice questions. So, there’s a bit of that as well. And something that I’ve talked about with some faculty recently, too, is this baggage around K through 12. And maybe that’s something that’s resonant with you all.

John: Yeah, that’s given the testing effect a somewhat bad name, because high-stakes testing is being used in a lot of what’s going on with K to 12. But that I don’t think is what retrieval practice as you’re suggesting is all about.

Michelle: Right, and I have to be cautious here. I really like how you laid that issue out in K through 12, that there is a reputation problem… and that has happened because of the high-stakes standardized testing policy in the United States. And I got to be careful because I don’t want to represent myself as an expert in K through 12 or in K through 12 education policy. But I don’t think you have to be an expert in that to know that there’s been a lot of same pretty well justified public pushback against over-testing in K through 12. And yeah, I think that we absolutely do have to be aware of that. Students come to us in higher education… that’s a system that many of them have been through and our faculty are very aware and very cognizant of that too. So, nobody’s a blank slate here, not our students, not our fellow faculty. We have assumptions and ideas and experiences about testing that happen. I think those can be addressed. But, yeah, that is definitely another very big barrier. We got to differentiate between high-stakes standardized testing for the reasons it’s done in K through 12, and low-stakes testing and quizzing for learning as proponents of retrieval practice would have it.

Rebecca: Some of the pushback I’ve heard from faculty fall into two categories related to this as well. One is that they assume that retrieval practice is best implemented in 100, 200 level introductory classes instead of upper level 300, 400, graduate-level classes. And then the other area is that paper and pencil tests don’t make sense and all disciplines. And so they assume that a test has to be in a paper/pencil format, which could be online testing, or it could be multiple choice or it could be essay questions. But I think that, from being someone in the arts, like there’s other ways to test beyond that, but we don’t think of those as tests.

Michelle: Right. That’s another great lens through which to look at this issue; that we do need to broaden the definition to draw more attention to this and to make it a more appealing concept. But yes, how can we make it broadly appeal across lots of disparate disciplines? Not only does it not have to be a multiple choice type of exam, maybe it’s not a pencil and paper exam at all. And we as faculty have to think about what makes sense there. You make a good point about the levels concept. I think, these days most of us have heard of, “Well, there’s one particular Bloom’s taxonomy…” which is a wonderful framework for getting us thinking about being systematic about what we’re asking students to do with the information that we’re bringing to a course and trying to do things like align the teaching we do with the assessments that we have. That’s wonderful. However, I think it does ingrain in us that idea that “Well, just knowing things is sort of at the bottom… You sort of get that out of the way, and then we go on to the good stuff.” And from a cognitive perspective, these relationships are much more fluid and much more interdependent, so that yes, absolutely, the higher thinking that is what we want. That is what we should want. Or if we’re in highly applied disciplines (if we’re in the arts, for example), we need students to be able to do things with that information. But they have to have that. So I think it challenges us to think of new ways with that concept as well.

John: One of the barriers I think some people have is they don’t like to grade tests and so forth. But one of the things you mentioned in your book is that the testing effects been known for a long time, but it was really difficult to implement in terms of low-stakes testing, particularly when you’re teaching at a larger scale. But, as you’ve done yourself, and as you suggest in your book, computer technology makes it easy to automate some of this… certainly more easily for multiple choice and free response and similar things. But it makes it a whole lot easier for both students to have multiple attempts at learning something using some type of mastery quizzing and it makes it a whole lot easier for faculty who don’t have to spend all their time grading.

Michelle: Right and that’s absolutely where the practical stuff comes in. So, we’ve worked through some the philosophical objections, that: “No, this does not turn your classroom into some terrible assembly-line concept of learning. It’s not going to create a bad relationship with your students. It’s not going to simply carry on a legacy of bad policy as people perceive it. It’s not going to do those things.” And then, we do get to: “Alright, but what is this going to do to my life as a faculty member?” …and this is important stuff. Those who know me know I’m a big fan of James Lang’s work in his books. One of his more recent books is called Small Teaching and there’s a lot of different takes on it in that book, but it really hits home with respect to “We do have to think about not everybody’s in a position to, nor should we even try sometimes, to just take everything down to the foundation and rebuild.” That we do really need to think about “Do I want to do this, if it’s going to create 800 more questions for me to grade?” Is this a sort of a situation where, “Well, I don’t want multiple choice, so I’m going to have to give these open-ended questions. I’m gonna have to give feedback and I have 200 students, what will happen? If I am going to go with multiple choice questions… well, how am I going to do this? to have to write all of this?” And yes, it’s one of the most powerful outcomes of the educational technology revolution that makes it workable, and scalable in a sense, even with large classes to do these types of things and to bring them in. So, that is definitely a message that I hope faculty think about if they’re on the borderline of wanting to bring in more retrieval practice into their classes.

Rebecca: I’m in a discipline where the multiple-choice questions are using things digitally doesn’t always work for testing and practicing some of these basic things, and there’s not a good way to automatically grade it. But one of the strategies that I’ve used is actually some self grading, which has actually worked pretty well. I just check and I have them write notes about anything that they got wrong. So, it demonstrates that they’ve tried to understand when we go over it, and I give credit based on how thorough those notes are, rather than whether or not they got the question right or wrong. And that’s made a difference in my classes. And had I not come up with that solution, I think I would have abandoned it because it would have been too much work. But it it actually is working pretty well.

John: When we had a reading group on Small Teaching last year, one of the things that was widely adopted by faculty was a very simple form of retrieval practice where they had students at the start of each class reflect back on what they had done in the previous class. And most of them have continued to do that in subsequent classes as well.

Rebecca: One of the other barriers that faculty might raise is the idea that students don’t take low-stakes things seriously, or that they don’t put the same kind of time into it that they might for something that’s high stakes. Can you talk a little bit about how we might help students find value in retrieval practice and subsequently also with the faculty then?

Michelle: Right, so that, ”Well, we can put it out there, but will they do it?” I’ve kind of crossed this philosophical and practical barrier for myself of giving some credit for pretty much anything that I am hoping that students will do. I put the work in to set it up and I do believe as a teacher that there’s reason to believe that will help their performance, that I need to work it into the syllabus somewhere. I don’t think it detracts from learning, necessarily, to say, “Yeah, there’s some points associated with this.” And especially with our students who, for many of us, are going in a million directions at once. They’re juggling jobs, multiple classes, sometimes their own families. So having an incentive in the form of points—having some kind of a payoff—I think, helps them make that decision that this is at least worth the time to do. I think the other thing that we probably can all do more of, and that I’ve done more over the years, is framing and honestly marketing this to students… communicating with them about why. And when students disengage from an activity like this, when they say, “Ah, why do I have to sit down and do this thing? This is just another test. Oh, no.” …really conveying the excitement and the goodwill that we have in setting those things up can go a very, very long way. Of course, a student, if they just look at it inside and they have no context for why this was put into place, they’re going to have them say, “Well, maybe I won’t do that.” But when we can market to students, we can say “There is a lot of research that shows that this is a very, very good use of your time. And hey, you’ve probably taken a lot of tests in your life that were really about measuring or sorting you and figuring out what you know and what you could do. This is a very different kind of test.” So that can go along way and get the C students nodding: “Alright, I get it. I get why she put this assessment right here.” I think a lot of us have hit on the practical strategy too, that the little Easter eggs or goodies that we plant in the form of questions that get re-used on the higher stakes assessment. So most of us will have tests for measurement at some point in our courses. And yes, students really do pick up on it when you use one, two or more of those items that were in, say, the gamified quiz that you ran in class or the reading quiz that they did beforehand. They can see those and say, “Oh, wow, I got feedback on that. I got an opportunity to practice…” and if it draws in a few more students who see it as almost a legitimate form of cheating, honestly…. like a fun and sanctioned form of getting an advance sneak peek at the exam, then great! Then that maybe is an opportunity for them to come in and see that. So there is that. Actually taking it seriously ourselves, not just in the form of saying, “Well, here’s the points I’m going to give you for this,” but spending class time on it. That’s a big bridge for a lot of us to cross, right? Because we as teachers tend to be very focused on “Oh my gosh, class is for covering material. We use all these sort of distance metaphors to talk about what we want to do with our class time. But if I say, “You know what, guys, I believe in this, and I believe in it enough to where we’re going to spend the entire class period before the final exam….” I did that twice last week myself, when running exams. Or we’re going to spend five or 10 minutes of the beginning of every class period doing this, as one project recently published about doing. If you show yourself doing that, and offer them that, I think that also goes a long way towards it. And I guess to just say, well, taking it seriously… here again, what does that look like? What does that mean to different people? And we can kind of a little tongue-in-cheek say, “Well, why do we have to take it so seriously?” Sometimes games and learning can happen when it is presented in a more fun context. So not everything has to be deadly serious or spending hours and hours and hours of stressful time on. There are occasions when a light-hearted approach can be perfectly good and can still get us involved in that really critical activity of retrieval

Rebecca: I can share an example of doing that in my classes. We’ve done design challenges and sometimes we challenge other classes that are happening at the same time. That reinforces some of the basic principles that we think that students should be doing and reminds them of it… and they might work in a team…and then we have a competition. And it’s fun and what have you. And students like those, it breaks up the day, it makes it more fun. And then I’ve also done things where I give class time to do little design challenges in class that might be individual and then they can level up to working with a partner to finish solving a problem or something… and students value that. They recognize that next time they’re trying to do a project on their own,that it’s easier because they’ve had that practice or that opportunity for the retrieval practice. And my students have actually ended up asking for more of those opportunities.

Michelle: Great.

John: Could you go back just a little bit and tell us what you did in those couple of classes last week before your final. We’re recording this, by the way, in early December during finals periods in both of our campuses, but we’ll be releasing it a few weeks later… to put that in context.

Michelle: Oh, okay. Well, I’d love to. …and I’ve been talking about retrieval practice as you pointed out for years, and I still discover new ways to infuse this into courses. And the context for this is my cognitive psychology undergraduate course. It’s a 200 level. So it’s a lower-division course. And it’s about 60 or 70 students. And as you can imagine, it is a bit of a tough sell. For many of the students it’s their first encounter with this side of psychology. It’s not as intuitive as some other areas of psychology. So there’s a lot to learn and a lot of motivation to be done. One of the ways we bring this and in this course is using a technology called Kahoot, that’s spelled K-A-H-O-O-T,, and it’s really very intuitive and functions very smoothly… relatively free of bugs. That’s good stuff. It’s a program for doing gamified quizzes of various kinds. What I did, and at different points in the semester, and then really amped up in the last week of the semester is running these gamified quizzes. And this is something, by the way, that I hit on and got the idea to try based on a colleague named Niki Bray, who’s from Tennessee, and has actually done some really systematic work in reformulating some of her courses around in class quizzing in just really ingenious way. So I saw some of her presentation and I said I’ve got to try this for myself. I went with multiple choice questions. Kahoot does have some parameters… questions do have to be short…very, very short. And to some people that may be off-putting, but you can put together quite a few of these. And so we would put this up on the projector and students have the option of dialing with either their laptop or their smartphone and weighing in on each question. The neat thing about it is it has an algorithm for giving points based on your speed as well as your accuracy and it’s got a little leaderboard so you can actually have a little in-class competition. Now some people who use this do require all students to do it and they actually issue points for performance. Now, I presented it very much as a practice activity… and made it very, very clear because of my philosophy, I’m not going to assume that all students have devices or have smartphones or laptops or that they want to do that. But I said, “Look, remember we’re talking about retrieval practice, guys. So the real meat and potatoes of this is not buzzing it on your phone. That’s fun. But the real benefit of this activity is what you’re doing sitting there in your seat. And you could be doing this with a piece of paper if you want.” And that is what a few students opt to do. They try to answer the questions, they know what they need to go back and review and so on. And it’s nice because it spits out at the end, a whole report that tells me right away… Okay, which questions do we need to revisit? Which ones did students have the hardest time with and so on. That’s one of the things that I just brought in. And yeah, it was a big deal. I sacrifice a chapter of “coverage” so that we would have more time at the end of the semester. But to me, I would rather have students going into the final knowing that they’ve had this retrieval practice and they have a better chance of performing really well. Earning a good grade on this material I care about than honestly cramming in a little bit more mileage in terms of the quantity.

Rebecca: Sounds like fun.

Michelle: You know what, it does really bring a fun factor. There’s been a lot of different variations on in-class polling, and I will admit to this, I actually purchased and am the proud owner of a physical buzz-in quiz device, complete with a whole spaghetti nest of wires and an incredibly abrasive, buzzer sound, and everything. So previous to this, my educational technology did include… I think I could have up to eight intrepid volunteers who would play a quiz game and then I would have to appoint a points keeper and all this… and props are always a lot of fun. So when I say I believe in bringing in retrieval practice, I really do walk that walk. But I will say doing it by a smartphone does allow for more participation, and I don’t have to worry as much about the minutiae of scorekeeping and stuff like that.

John: I played with Kahoot a little bit at my classes at Duke and students have loved it.

Rebecca: We’ve talked a little bit about framing things so that students take the practice seriously. What do we do about the students who just push back, it’s like “This is too much work. This is a lot of extra time…” or that sort of argument.

Michelle: I think that that’s another piece of that barrier to more faculty adopting this, not just the work involved, but realistically, student opinions. Student evaluations matter a lot to faculty life. And of course, we all want to have that wonderfully rewarding semester, not the semester where we feel like we’re at odds with our students. So I do think a little piece of this is we do anticipate sometimes worse and more pushback than what actually happens in the end. So I think we have a fair amount of sort of a dread factor when we go into something new like this. But that said, when students have an issue with more quizzing or more testing, here’s how those come out. I think, first of all, we do have to sometimes separate out the technology aspects of it from the quizzing or testing, per se. So, as John mentioned, this is one of the amazing things that educational technology does. But the flip side of that is, if you’ve ever used technology for education, in any shape or form, you know that it breaks down. And when it breaks down at a big class, that’s a headache for everybody, it’s a misery. So if you’re trying it, and things are not working out, you just got to figure out “Okay, how much of this is the assignment, the activity per se, and how much of it is that wonky quizzing thing that I got from the publisher and it fell apart, or students hanging up when they’re trying to dial in with the poll and fine tuning those. I mean, when I first used kahoot, I decided it would be a lovely idea to put four chapters worth of material into a 40-question quiz. And when you get used to as you know, that they took a long time, I mean, 12 questions can keep you going for a very long time, especially if you’re discussing… and some students got kicked out part way through and then they weren’t on the leaderboard and I could have set the whole thing aside. But really, that was more I needed to get the technology working. So that’s a big piece of it. I think that sometimes it is a perception issue with the timing. Now, those of us who are just all over this as a teaching technique, we like to do reading quizzes before we talk about that material in class. So chapter three is up on the syllabus and your chapter three reading quizzes due on Sunday night before we do that…, and that can provoke a fair amount of confusion and honestly griping with students. They say, “Why did I get tested on this when we didn’t do it… we didn’t cover it in class yet?” And what do you know, that’s another framing and communications issue. Once we know that that is why we’re doing that you get so much less on that side of things. But here too we do also ourselves have to follow along with what we say. If we say you’re doing this reading quiz so that we’re establishing a foundation. we don’t have to teach everything in the class itself. We could spend the time applying. Well, guess what, then you do have to do that. So if I assign you to do the reading quiz on chapter three, and then on that Monday morning I go, “Okay, we’re going to go over chapter three starting at the top. And I’m going to show you all the PowerPoints for all these things that you already read.” Well, yes, then student morale and student support will fall apart at that point.

Rebecca: Those are very good reminders.

John: Yeah, I’ve pretty much adopted this approach all through my class beginning when I first saw you present it. So I have students do the reading in advance, I have them take reading quizzes with repeated attempts allowed on those. And then in class, I have them working on clicker questions, and I and the TAs go around and help them when they get stuck on problems. But there is an adjustment and students, especially when I first started doing it, would generally say, “But you’re not teaching me” …and you do have to sell students on this a bit. One source of resistance is that when students take quizzes, they often get negative feedback. When they read something and they read it over again, it looks more and more familiar. They’ve got that whole fluency illusion thing going and they become comfortable with it, and they feel that they’re learning it… or similarly, if they hear someone give a really clear presentation on a topic, it feels comfortable. They feel like they’re learning it until they get some type of summative assessment where they get negative feedback, and then they feel the test was somehow tricky. But it’s a bit harder to convince them that actually working through retrieval practice, watching videos at their own time and pace, reading material as needed, and then spending class time working through those problems is as effective. I’m getting better at it. But it’s been a long time trying to convince students and I did take a bit of a hit in my evaluations, especially the first few times.

Rebecca: Do you mean learning’s not easy?

John: Students would like learning to be easy.

Michelle: You know, it’s funny, I think almost in a way… see what you think about this idea. But it’s almost a mirror image of our illusions as teachers, right? That I gave a wonderful clear lecture and I assigned wonderful readings and I saw students highlighting them. Therefore, students must have assimilated this knowledge and it must be in there. So I think it challenges our students but it also challenges us as well. It can be quite an eye-opening experience to running something like a Kahoot. And that brilliant point that I gave this great example for…. what do you know…. 7% of the students actually nailed it. And so we can all use a reality check… teachers and the students. And you mentioned re-reading, that’s another one where I’ve had some pretty intensive conversations with other faculty, I’ll kind of say, “Oh, well, and there’s this great research that shows that students tend to re-read when they’re studying and we know that from a memory standpoint, that is really, really ineffective.” And faculty will say, “Whoa, whoa, whoa.I want my students to be re-reading.” Of course, now when they say re-reading, they may be picturing deeply interrogating a text… annotating it… looking at it from a different perspective. And absolutely, that’s a wonderful part of scholarship. But that isn’t what we’re talking about. We’re talking about students re-reading as a study form and mistaking highlighting for deep interrogation. So, just like with the rest of knowledge bases versus higher-order skills. This is another words “both and” it cannot be “either or.” Yes, we want students re-reading in the right ways, but students or teachers cannot mistake that for learning sometimes.

Rebecca: So, let’s say you’ve just convinced all of our listeners that we need to be doing this, how do we bake it into our course designs?

Michelle: Well, I think that, really getting creative with this, and as I talked to faculty when I visit other schools or talk to faculty of my own institution, I just see all of these new ideas all the time about how to do this. Once you do get that critical epiphany of alright, a test doesn’t have to look like a test on the surface… it does not have to be the ritual of “Okay, you’ve got a number 2 pencil that’s breaking and I’m standing over you while you have a panic attack for an hour.” That’s not what it is—that it’s about the retrieval. Once you get that, all kinds of creativity opens up. So I’ve also started sometimes on the very first day with the syllabus quiz. So, especially if you have a smaller class, we all struggle with that “I spent all week writing the syllabus and it’s incredibly important, and then I’m getting questions about stuff that’s on it all the way through the semester.” So, really on the first day, what I do is I take a very light-hearted approach and I divide the students up into teams, just physical teams. Everybody’s got a copy of the syllabus, sometimes I fake them out and I say, “Okay, we’re going to go over the syllabus point by point…” and I say, “No, you can read the syllabus. We’re actually going to do this other thing which research shows will actually help you remember it.” So the task here is that each team formulates a set of questions… you can make just a few. like three… so a little bit of teamwork. So formulate three good questions off the syllabus. And you’d be amazed, they come up with questions that even I can’t answer sometimes without having to cheat. So they talk about it. And then of course, you go around the room in some arrangement and each team gets to ask the other team their questions, and if you really want you could keep score and everybody loves bragging rights for being the team that stumped the other teams and won the most points. So, it can literally start right then. With that idea that I don’t read stuff to you in this class, this is about you. And I’m also not piling a huge amount of work on myself either apart from being the moderator and the MC and having written the thing in the first place. I don’t have to write questions. They’re writing the questions and they’re answering them too. So those are some of the ways that we can do that. I think especially in our larger classes we do want to think about things like peer grading or peer review of open-ended question responses. We do want to take advantage of things like publishers’ test banks to set those up as reading quizzes. And you’d mentioned earlier about “Well, what about this not being suitable for upper-division classes?” I had an upper-division class that just wrap this semester where we had reading quizzes as well. It may have been an upper-division course but it also serve that purpose of “Hey, you take a basic quiz over the chapter on Sunday and that really sets the stage for us to have a more substantive discussion.” All of those things. are ways that we can do this. I think open-ended reflection as well… so tests that look nothing like a test but are still retrieval practice. You’d mentioned about reflecting on what you learned last class period. So this sometimes goes by the name “brain dump.” And I did a bit of this last semester as well. So, this by the way, I do want to credit the great website retrievalpractice.org. So retrieval practice is actually that high profile it has its own website. It’s an absolute treasure trove of ideas. So, with the brain dump the way that I did it, or similar to what it sounds like you did, every now and again we start off class with you writing down everything you remember from last time on a piece of paper. I had students then turn to their neighbor and compare notes and see what they came up with. And I didn’t grade those. This was not a heap of grading for me. But in the end what they did turn into me for accountability and a few points was a very short reflection, just on an index card. So, “What surprised you?” and I review those and they’re very eye-opening but again I’m not there to police or micromanage what they put on their cards. So, that’s retrieval practice too. There’s lots of different flavors that we can bring in.

John: I do something similar with asking students to reflect on the reading in each module we work through. But because I teach a fairly large class, I didn’t want to have to deal with all the index cards. So I just have them fill up a simple Google form. And then I can skim through it and assign grades much more easily than shuffling paper. But it’s the same basic idea and it’s worked quite well.

Michelle: Absolutely.

Rebecca: Do you have any other examples of the kinds of little tests that faculty can run that don’t look like tests?

Michelle: Right? So tests that don’t look like tests. Well, here’s another that was probably a little bit more practical for a small class, and this is how I ran this. It, like the brain-dump exercise the way I ran it, was also very cyclical and very student generated. So we started out each class at the beginning of each week with students generating a set of quiz questions based out of the assigned reading. So we didn’t actually, in this particular example, have pre-quizzes or something like that. But students came in knowing that they can bring their book if they wanted. But they would need to sit down and write for me three questions, whether short-answer or multiple choice. Then I can flip through those and I would really quickly put those together for a quiz that went out the subsequent day. So, we’re alternating between generating questions and then returning to those questions. And then I would have pass out. I told them treat this like a realistic test. Actually try to retrieve everything, but, you know, when times up, you’re going to get to go back to it. And then they would grade it themselves. So I didn’t actually do the grading either. And that is really great for spurring discussion. And “Oh, my gosh, I thought I knew the difference between reliability and validity, but now that I tried to answer it, I realized that I didn’t.” And then you can throw it back out to the student who now has bragging rights for having had their question selected and say, “Hey, what did you mean? What was the right answer? And why did you put that down there?” And at the end of the day, they kept that quiz too. So it was really very much in their own hands to do. So there’s that. Other creative ideas that I’ve run across over the last couple of semesters… There’s a great project out there, run by Bruce Kirchhoff at University of North Carolina at Greensboro. So I got the wonderful pleasure of talking to that group last year. And he and some colleagues working in the area of botany actually put together a freestanding, custom built mobile app that students could take with them that presented different kinds of quizzes over the sorts of things that botany students really need to know like the back of their hands like how to identify different plants and how to discriminate among different examples and they found some empirical evidence that this actually raised performance up quite a bit. I’ve heard too of another Professor put together some surveys in Qualtrics, the surveying software took advantage of its ability to actually text message people and send them the questions. So this was an opt in sort of activity and it’s one that they didn’t just have it run 24-7 because it was a little intrusive, but what it did is it sent students questions that they could answer at different intervals throughout the day, which also takes advantage of another principle from applied memory research which is spacing. So students are getting these unpredictable questions, they have the option to answer them, and they could be happening even when they’re not in class. So those are some other ways. Some of them look like tests, some of them don’t. But those are creative ways to get students engaged in that practice.

Rebecca: As a faculty member, we often advise students and mentor students who might be struggling in other classes, ones that we don’t have control over. Are there ways that we can help those students use this methodology to do well in those classes where it might not be embedded?

Michelle: Right. And that’s so great that you bring that in as well because ultimately that is what we need as teachers and that maybe circles back to yet one other piece of objection that I’ve seen… actually this time in a published article from a few years back that said, well are we doing too much for students? Are we scaffolding them too much so they’re going to grow really dependent on these kinds of aids like reading quizzes, and reading questions. But if we also have in our mind very intentionally, that what we want students to have at the end of the day is also something they can walk away with, I think that we do have to be very mindful of like, “Okay, let’s not create the impression that just because retrieval practice is so important that you have to sit there and wait for me to put together this specific kind of reading quiz for you.” So I think here, the really powerful message is once again that one students take this to heart… once they’ve not just been told this, that “Oh well, you should quiz yourself as a study strategy,” but they’ve seen that I believe in it to the point where I’m going to put time, energy, and work into it as part of my class. And maybe they’ve even seen the results… they’ve now had their own little before and after experience of what happens when I do this… that they can be more likely to take this forward. So having more faculty across the curriculum endorse this is a powerful idea that “Hey, this is how your mind works. This is how your brain works. Your brain doesn’t just soak up stuff that’s in front of it. You soak up stuff that you have to answer questions about.” That I think is going to be a powerful message. And there’s actually another article out there that I just absolutely love it was done by a psychologist named Pennebaker and some colleagues at University of Texas at Austin some time ago. They replaced high-stakes assessments in their Introduction to Psychology course with these mobile in-class quizzes that were done every day. And one of the things that they report in this article is not just that students did better in that class, but that certain subgroups of students actually showed improvement, specifically closing of achievement gaps, really, in classes that they took after the psychology course. And these were classes that we’re not even in psychology. So you think about that for a minute. How does that happen? Now, nobody knows exactly for sure, because this was kind of an unexpected finding in the study was my impression, but a real possibility is that when students have sat in this class and every single day they have seen the power of taking a quiz… of spacing out their learning… and attacking it in a very active learning approach… Once they’ve seen that happen, they’re more likely to go home and say, “You know what? I can do this in my biology course too. I don’t have to sit there with the teacher’s quiz. But, if I attack it in the same way I might get the same results.” So I think with those things, we can have students walking away with that enduring practice that we want them to have. And it is funny too, because it brings up one of the, I’ll just say it was a really heartwarming teacher moment that I had this last semester when I did bring in these Kahoot quizzes. In the run up to the first exam, I had done my fancy little in-class quiz and was kind of patting myself on the back of what a great leader I was. So I came in to, I think it was the class right before this exam, and I come plowing into the classroom and it’s dead quiet. And there is a student at the front of the classroom. He has commandeered the podium and the computer and what has he done? He’s accessed the Kahoots, which I gave them the links. You know, they’re out there for all the students to see. And he is running them and the rest of the students in the class are taking them just as seriously as when I was administering them. So they’re running through the questions again, giving it another shot. I didn’t tell them to do this at all. It was one of these moments where I just backed out and closed the door after me. And I came back in when they were finished. And so I think those are the kind of moments that we can set ourselves up for when we really do bring this into our classes and we get behind it and a really authentic way.

Rebecca: I think one thing Michelle, in the way that I asked the question, I had also asked it from the perspective of a student hadn’t had the experience of doing a retrieval practice in a class. So, if you’re working with a student, maybe outside of class or as an advisor, are there things that we could do to help those students adopt those practices, even if they haven’t seen it modeled for them?

Michelle: So how do you help students when they haven’t had some of these experiences on their own? And I think this is a part of a bigger package of goals that I think a lot of us should have in supporting our students to really put students in the driver’s seat. To say, “Yeah, you don’t have to wait for a certain style of teaching or certain subject material in order to succeed with it.” I see that isn’t really part of a larger package of growth mindset honestly. So what can students do to make themselves the masters of this? Now, I think that there are some old standard, very traditional, approaches that are worthwhile. And when you look at those approaches, sometimes retrieval practice is at the core. So it may not be a matter of trying to get a very unfamiliar set of terminology or anything for students before, but really getting them to look at some of these approaches in a new way. So things like you’ve ever heard the term SSQ4R or PQ4R, there’s a couple of those that have acronyms and the things they have in common are that they tell students when you sit down to study or read a text or prepare for a test, here’s what you do. You don’t just start reading from the top with no goal in mind other than “Oh, I want to get a good grade later.” What you do is first of all, you set yourself up with questions. Which is, after all, a lot like retrieval practice. Start with a question, say, “What do I want to answer? Can I answer that now?” And if not, why not? To read your text or go through your material very intentionally around this questions, and then there’s always this piece of recite or review. That’s what one of those Rs stand for in some of those traditional systems. And that means closing the book. That means closing the book and saying, “Okay, I’ve sat here with this material for this amount of time. What can I actually say about it at this point?” So, sometimes directing students back into some of those and saying, you need to adopt these strategies, which are completely teacher-independent, they’re fairly discipline-independent as well. That can be good. If you’re doing them for the right reasons and with the right approach in mind, that’s very good. Encouraging students to take advantage of the publishers’ companion sites… Now this is a little bit more of an uphill climb. That is where we run into “Well, if there’s no points for it, Why should I do it?” But encouraging students to say “Look, you already paid for this textbook. To be crass about it, you paid all this money, did you know there’s a website over here and if you interact with those materials in this particular way that’s really, really likely to pay off for you?” So, besides just ensuring that our students have what I consider to be these basic foundational pieces of knowledge about the mind and brain: that we remember through testing, we remember through retrieval and an active engagement. All students should have that, but those are some specific things that we can counsel students to use across all their studies that really should pay off.

John: Do you have any other suggestions for those faculty that are thinking about expanding their use of retrieval practice?

Michelle: You know, just to really encourage and support faculty who are starting this journey, as it sounds like that you all have, to really re-examine how we can bring in this incredibly powerful principle, and to really reassure each other that “Yeah, this is not about really just piling so much more work on yourself.” We can sometimes even just re-examine the assessments and assignments that are already in the course and so that’s kind of one last piece of practice that I think that a lot of us can really stand to bring in. And that was a big thing for me as well to say, “Well, if I’m going to administer these tests… we administer tests, we administer midterms anyway… what else can we do to increase their value as learning experiences… as learning events. So here’s another where I’ve brought in various forms of test discussion activities. Instead of standing in front of the class with that deadly the day-after-the-test class period where I say, “Let’s go over it” to realize, you know what? That probably will not work that well. But one of the amazing things that we have learned from the research on retrieval practice is that it’s not just in the moment in the taking of the test, where this advantage to memory happens. It actually creates a sort of a receptive window for learning and for review when we’ve just taken a test on something. So if you’ve got a midterm in your class already, well, hey, why not carve out the time after that test is up to say give it back to the students… like I photocopy the exams before I’ve graded them. They have no feedback on them whatsoever. And I hand them back to students as a group discussion exercise. I say “Alright, here’s a blank copy of the exam, your group can fill out this blank copy together, just knowing what you know, revisiting all those questions, having those good discussions about what you understood and what you didn’t. And I offer a little bit of extra credit for really good performance on that. And there’s other ways to work that out so it actually takes off as a small group exercise in class. But regardless of the specifics of how you make something like that work, the spirit of it is the same. That when you sat down to do this, to try to drag all this information out of memory, that is not an end unto itself. It should be part of a bigger picture of learning in the class. And it’s sort of an untapped vein of potential we have as teachers and that our students have as well and that we can access it regardless of what the discipline is or how the class is setup.

John: …and students see it’s very relevant. I started doing something very similar in my econometrics class last year following a suggestion from Doug McKee who had been doing that in his class. And it worked remarkably well… and turned what was normally a pretty unproductive class period where we’d go over the test… and the people who did well with just be really happy and pretty much ignore any discussion, and the people who did badly were just sitting there unhappy and not really being very receptive. But when they sit there, and they’re explaining it to each other, it seemed like a really ripe time for them to learn the material much more deeply.

Rebecca: I remember the first time that you implemented that. You sent me a text message with a photograph of his students taking a test and it looked very active. [LAUGHTER}

John: …and they were having fun.

Rebecca: Yeah.

John: One of the nice things that came out as I was wandering around listening to their conversations, and I was hearing people say, “Oh, yeah, now I see where that came from.” Or someone would say, “Well, when did we learn this?” And then someone else would say, “Well, remember when we worked on these problems?” …and it just helped them make connections… and the power of peer instruction is so remarkable. I’m going to do it in as many of my classes I can.

Michelle: That’s perfect. That’s what we want as teachers, and that’s what our students benefit from.

Rebecca: So as you know, we always wrap up our podcast by asking what’s next?

Michelle: Oh, wow. So I am looking forward to kind of rebooting a course that I have not taught in some time. My senior capstone course in technology, mind, and brain. And that is a fun one. But it’s one that is going to take a lot of revision, since it is technology and things change so rapidly, as we all know. So that is going to be a big part of next semester. I also have a crop of research projects in various angles on teaching, learning and educational technology that I’m really excited to be moving forward in the next calendar year. One of those… really foremost among them is a project on virtual reality for learning. We have an incredibly creative and dynamic team looking at virtual reality here at Northern Arizona University. They put together an amazing series of interactive exercises that are part of the organic chemistry course here and teach some of the challenging concepts in that extraordinarily challenging gateway course. And so we now have a whole set of data from students who went through and did this at varying points during the semester. We got their feedback, we’ve got all different kinds of psychometric measures that we gathered from them at the time as well. So, I cannot wait to be tackling that and looking at all kinds of angles on how this part of technology is impacting student learning in this course.

Rebecca: Sounds like some exciting adventures.

John: That sounds like a wonderful research project. …looking forward to seeing what you find.

Rebecca: It was a real pleasure to talk to you again, Michelle. Thanks for spending some time with us.

John: It’s wonderful talking to you. Thank you.

Michelle: Oh, likewise, always a pleasure to talk to your listeners.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

60. Inclusive Teaching

Are your class conversations dominated by a small number of voices? In this episode, Dr. Danica Savonick joins us to discuss a variety of class activities that support an inclusive learning environment and promote equity in participation while increasing student learning. Danica is an Assistant Professor of Multi-Ethnic Literature at SUNY Cortland, and a recipient of the K. Patricia Cross Future Leaders award, a Woodrow Wilson Fellowship in Women’s Studies, and a Mellon/ACLS Dissertation Completion Fellowship.

Show Notes

Studies of bias in the classroom:

Transcript

John: Are your class conversations dominated by a small number of voices? In this episode, we explore a variety of class activities that support an inclusive learning environment and promote equity in participation while increasing student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Danica Savonick, an Assistant Professor of Multi-Ethnic Literature at SUNY Cortland. Danica is the recipient of the K. Patricia Cross Future Leaders award, a Woodrow Wilson Fellowship in Women’s Studies and a Mellon/ACLS Dissertation Completion Fellowship. Welcome, Danica.

John: Welcome.

Danica: Thank you. Thanks so much for having me. I’m really excited to be here.

John: Our tea’s today are…

Danica: I’m drinking a coconut lime seltzer.

Rebecca: That sounds pretty good.

Danica: It is.

Rebecca: It’s a good alternative to tea, I suppose.

Danica: I think I’m pretending that I’m on a tropical island or something.

Rebecca: Yeah, the weather around here would make me want to do that, so perhaps it’s the same there.

Danica: How far away are we from from each other? I’m here in Cortland, you’re…

John: About an hour and 45 minutes, I think, by car.

Danica: Okay.

Rebecca: Very rainy today.

Danica: Yeah, and I hear we have some snow coming up in the next 24 hours or so, so should be interesting.

Rebecca: I have the Prince of Wales tea.

John: …and I have a holiday tea from Twinings that I picked up in the tropics in Orlando at the Online Learning Consortium a few weeks back.

Danica: Sounds yummy.

John: It is good.

You’ve written quite a bit on creating a supportive environment for discussing issues of race, class, gender and sexuality. Let’s first talk a little bit about the context in which you address these issues. What courses do you normally teach?

Danica: I generally teach American literature courses. Sometimes those are general education courses, sometimes they are within the English major. I’ve also taught a number of writing classes that are a little bit more interdisciplinary in nature, and regardless of whichever course I’m teaching I like to give them a theme or put my own little twist on them. For instance, if I’m teaching a writing course, this semester the topic is the purpose of education and so we’re drawing from a wide different disciplines… people who’ve been writing about different learning methods and then when I teach English courses, some of the topics I like to do are the arts of dissent and we’ll look at the theme of dissent in American literature. This semester I’m currently teaching Intro to Multicultural Literature, which has been super fun and then next semester I’ll be teaching a graduate course on feminist world-making, which I’m really excited about.

Rebecca: Well that sounds really exciting.

John: What are some of the challenges you face in discussing some of these issues in your classroom and trying to have productive conversations?

Danica: Well, some of the problems that I’ve noticed are consistent regardless of what classroom or what school I’ve been teaching in, but some of them vary according to the student population. But one of the most common problems that I see is just a lack of student participation, or if there is participation it’ll be the same two or three students who dominate the conversation… and actually just this weekend when I was home for the holidays I was talking to my family about this—my aunt is auditing a course at SUNY Purchase—and she was saying that the same one or two students speak every single class period and she’s curious about what the other students have to say and what they’re thinking… and even my grandmother who was at Brooklyn College in the 1950s… she said she remembers feeling too scared to talk in most of her classes… and so it was only one or two of the… I guess… the brightest and most vocal students who would talk in the classes. And then, of course, as I started teaching I started to notice this as well and I think it’s every new instructor’s nightmare probably that “What if nobody talks? What am I gonna do? What am I gonna do with all that silence?” And so, I guess the main problems I’ve been trying to address are not having the same one or two students dominate the conversation but having really every voice be heard in the classroom… and the more I’ve thought about it and the more I’ve come to study classroom dynamics the more I’ve realized it’s not entirely the fault of the students in those situations, and actually quite often it is the shared responsibility of both the professor and the students to create a kind of environment where everyone feels like their voice matters and that they have something that they can say… that they won’t get shot down by the professor… that they’re not intimidated by their peers and whatnot. So a lot of my work has been trying to increase participation in classrooms and also because my focus is often on race and class and gender and sexuality in literature, we have to figure out how to have productive conversations around those really difficult issues. And for a lot of students, it’s their first time talking about these issues and so we’ve had to establish ways that we feel comfortable talking about those important questions and issues.

Rebecca: I was actually just gonna follow up to what you were saying… really curious about the emphasis on the first time students have talked about some of these things, and I think that that’s really important. We’ve been having a reading group on our campus with a book called Race Talk and that’s something that we’ve mentioned pretty frequently: that a lot of these students have never been in a context to have a conversation about race… a lot of the faculty have never been in a situation to have a good conversation about race… So, when it’s someone’s first time, how do you help that be productive and feel safe? Because you have to be vulnerable to be in those situations.

Danica: Definitely. One of the most effective things that I think I’ve done is tell students that we’re inevitably going to mess up in these conversations because our educations have not provided us with the language and the grammar and the vocabulary for talking about conditions of structural inequality… and so I make that the baseline or the premise. We know we’re gonna say the wrong thing and we are likely going to accidentally offend someone and so as a class what we do is establish protocols or ways that we want to collectively address how to handle those situations and we come up with a set of community guidelines and principles and ideas that we agree upon for how to behave when we realize that, “Oh no, I could have said that better. I wish I hadn’t said that…” or if one student feels offended by something and so I think that has really helped, especially for students who are having these conversations for the first time; they know that it’s okay to say the wrong thing and we have an established procedure in place for how to deal with those moments.

Rebecca: Can you talk a little bit about how you set those guidelines up and how students participate in that process?

Danica: Yeah, definitely. This is one of my favorite things to do every semester is have students co-author a set of community guidelines in order to foster inclusive discussions of difference. Because I want every student to understand that their voice matters and I know one of the reactions that you can get is students can start feeling alienated if they say the wrong thing. They can disengage. They can start thinking that I don’t have a place in this conversation. …and so one of the ways that we create that environment is we’ll co-author this set of community guidelines. Rather than having students write them from scratch… I think that can be really difficult… so, instead what I’ll do—it takes about I would say half of a class period to maybe half an hour, could be 40 minutes, it depends on the size of the class—I’ll print out some really basic guidelines, four or five things that I think might work well in the class as principles that we might want to agree to abide by. So things like we won’t make assumptions about anyone in the class’s race, gender, ethnicity, things like that, and we read over them as a class. I usually project them at the front of the class and they also have them in front of them on a piece of paper and we’ll read through them as a class. They can ask questions… they can ask me to define a word they don’t understand, and then I give them about ten minutes to read through them quietly on their own with pen and paper and cross off and edit and add and remove anything that they don’t like about the guidelines… to add additional guidelines… to change the wording of certain guidelines… and then rather than calling on students individually and having to put them on the spot, I have them work in pairs of two to go over some of the amendments and edits and adjustments that they would like to make and then after five minutes or so we go around the class and each pair presents one or two amendments that they would like to make and so it really ranges from adding different adjectives and verbs to adding whole new amendments or saying that they didn’t like one of the ones that I put up, which is totally fine with me. The idea behind not having to ask them to do it from scratch is just that they have something to work with—it’s not that I’m wedded to those particular principles, I just wanted to give them some kind of language and some kind of grammar for how they might formulate the different community guidelines.

Rebecca: It seems like the pair scenario would help to mitigate any issues that might arise from a dominant group dominating the rules.

Danica: Yeah.

Rebecca: That was gonna be my question but then I realized as you were talking that that might actually be how you solve some of that issue.

Danica: Yeah, and it’s pretty egalitarian. We go around the room and each group says something, even if it’s by the time we get to the end sometimes the groups are like, “Well, everyone already said what we were gonna say, so we just wanted to agree that we really liked the amendment that this other group made.” And so that way each pair gets two or three minutes to add something, to say something, and then we move on, and so it’s not like one pair gets to really dominate. The other thing I forgot to mention is students go home, they have at least one or two evenings to think about the guidelines that we came up with. They have access to them. They can open them up at home and it’s not until the beginning of the following class that we ratify them, and often when we come back together at the beginning of the next class they’ll have thought of one or two things that they want to adjust and once we make the final edits and adjustments then we as a class decide that we agree to abide by them.

John: Another nice thing about doing it in pairs is when people are speaking it’s a little safer because they’re representing their group; they don’t have to take a stand and it makes it a little more comfortable perhaps for those who might have been reticent.

Danica: Exactly.

Rebecca: I like that you have the ability to review over a couple of days as well because that also gives students who don’t want to speak up the opportunity to email you or communicate with you separately too, right?

Danica: Yeah, definitely.

John: How well have the guidelines worked? Have students responded well? Do you get more buy-in to the guidelines since they created them?

Danica: Yes, absolutely. I was really surprised the first time I did this. I was like, “This is one of those wacky pedagogical experiments; I might fall flat on my face, they might think that I’m an alien from another planet.” But, they were so enthusiastic and I’ve actually had students from former classes say “That was one of the most meaningful things that we did that semester. I think about that a lot. I wish more teachers did that…” and so I’ve gotten really positive feedback on it and it’s also fun. It’s always one of the best conversations that we have throughout the semester. And you know it turns out that they have a lot to say about the issue. Actually, I often do this assignment when we’re teaching a work of literature called Citizen by author Claudia Rankine, which talks a lot about microaggressions… and students have witnessed and they’ve experienced these microaggressions in the classroom and so they’re eager to have a chance to participate in crafting a classroom that isn’t going to have these kinds of uncomfortable and awkward moments. I also should say that when we do this I share with students beforehand several of the studies that have been done recently on classroom participation and who feels most empowered to speak in the classroom. So, there’s been a lot of studies done on gender and the experiences of students of color and what a lot of these studies have found is that those voices that are most empowered to speak in mainstream media and culture are also the students who feel empowered to take up time in the classroom. And so I share this with students before we begin the community guidelines activity and they’re always really interested. I have the sense that some of them have witnessed or experienced or might have some sense that these things go on, but to actually see the research and to see the findings and to see these massive studies that have been done, they’re just interested in it, and especially because my classes are about race and class and gender and structural inequality, I think it’s fascinating for them to see the way that what we often think of as huge systemic issues can come to influence who speaks and who participates in the classroom as well.

Rebecca: Maybe we could share those citations in our show notes?

Danica: Certainly.

Danica: You’ve used something called Commons in a Box. Could you tell us a little bit about what that is for people who are not familiar with that?

Danica: Sure, it’s a free open-source learning and writing platform. It came out of the CUNY Graduate Center. It’s a combination of WordPress and BuddyPress, and so it’s this easy to install package that allows you to create digital learning spaces, and so different universities have taken it up to do different things. Often I’ll see institutions using it as a space for their professors to host course websites. They might want to have some kind of blog that features student writing. They could use it for digital humanities projects… and it’s free and it’s open source and so all you really need is server space. As often as possible, I’ve tried to host my courses on either Commons in a Box, or currently I’m using an installation of wordpress.org as an alternative to using Blackboard or Canvas and I could talk a little bit about why if you’re interested.

John: Yeah, could you tell us a little bit about what the advantages of this is compared to say one of the common course management systems?

Danica: Sure. I see the primary benefit of these platforms as they help students to develop transferable skills that are going to aid them in the world beyond the classroom, and so I’ll talk a little bit about what I mean by that. WordPress is one of the most common platforms on which the websites in the world are built .The latest statistic that I saw was something like 30% of the world’s websites are built on the WordPress content management system, and so I like to organize my courses on WordPress so that I can familiarize students with how websites are put together… how you can build them… how they think… how they organize information… and so what I try to do throughout the semester is scaffold students’ interaction with the platform. At the beginning it’s pretty user friendly: they create an account, they are able to log in to our site and then gradually they start going into the backend (which WordPress calls the dashboard) and they start creating their own content. So, they get to experience the process of going back and forth between the backend and then the front-end and seeing what that process is like and how information is organized on the WordPress platform. So, they start creating blogs and then what I like to do towards the end of the semester is deconstruct our class website and take it apart and break it and redesign it with students so that they can see, first of all, how easy it is to build a website. A lot of my students are new to this. They’re not necessarily computer science majors. They haven’t taken computer science courses, and so they’ve interacted with a lot of websites but they haven’t really gone in and thought about how they might build their own and so I show them how our course website is built and we redesign it we do all kinds of things and then often for students’ final projects they will have the option of designing a website related to something that we have done in the course and they often choose that option. They like it… they like getting to experiment with WordPress. For most of them it’s their first taste of the platform and several of them have said that they’ve gone on to learn more about WordPress because they’ve become really interested in it and I see this as a really great opportunity for students first to think a little bit more critically about how the internet works and how these pages that we’re constantly interacting with… how they’re constructed… and also to develop a transferable skill that could become a really valuable part of their resume and the skills that they will bring to the work world. Being able to build websites on WordPress is huge and so I find that starting that process early can be really helpful. It also creates an opportunity for us to have conversations like why is our course built on WordPress when all of your other courses are on Blackboard and we get to talk a little bit about what Blackboard is and the different ways that content management systems, especially in higher education, work to structure certain kinds of relationships of teaching and learning.

John: Does the institution host WordPress, or are you hosting your own instance of it?

Danica: Ideally, the university will host it. When I was at CUNY they have a really strong culture around open educational resources and free writing platforms and there’s a big community around that. It might exist at my new institution—I have to do a little bit more work to find it. As far as I know there’s a lot of people that are using Blackboard at my institution. Ideally… best-case scenario… the university would provide server space and then you could have an installation of WordPress or Commons in a Box, but currently I’m using Reclaim Hosting and Domain of One’s Own in order to have a classroom commons installation that I’m using across the three different classes I’m teaching.

John: Do you use the open aspect of that? Is the students’ work public or are you keeping it closed to the classroom, or is that something decided on a case-by-case or class-by-class basis?

Danica: Yeah, that’s a great point, thanks for bringing that up. It varies. Parts of the class web sites are public, parts of it are private… and another benefit of working in a quasi-public, quasi-private space is that it allows us to have conversations about what information students are putting on the Internet and what they want visible. What do they want to become part of their professional digital identity? What do they want to show up in search results versus what do they not want to show up in search results? So, we have a lot of these conversations early on in the semester when they’re establishing their accounts. We talk about the risks and the repercussions versus the benefits of using their real name to do the blogging that they’ll be doing on the site, and then for their final projects… often, but not always, I would encourage them to use their real name because they put a lot of time and effort there carefully revising these projects and they are deliberately constructing them with the idea that they’re going to be writing for a public audience. But, of course, in this climate of anti-immigration that we’re living in, you have to be super careful about what you’re encouraging students to put their name on and so I always have conversations around that. There’s always an option never to use your real name. You can always use a pseudonym for the blogging and for the final projects. You can always submit solely to me instead of publishing to a public audience, because I understand there are severe risks and in some cases they will outweigh the benefits of creating something publicly.

John: And we should note that you have an article in the describing your work here and we’ll include a link to that in the show notes as well.

Rebecca: You mentioned student blogging. Can you talk a little bit about how the student blogging is used in your classes and how that augments student learning and how that might facilitate some of these conversations that might be tricky to have?

Danica: Sure. I love student blogging—I don’t know how I would teach these courses without it. My courses are structured around the blog—it’s one of their major assignments, and so for every single class two or three students are assigned to blog about the assigned reading—I think the requirement is something like 800 words or so—and that they have to do a small close reading… so an analysis of the excerpt of whatever literary text we’re reading and it has to end with two discussion questions, and for every student who isn’t blogging. So, the majority of the class they have to leave a comment on those blogs before our class period starts, and so the blogs are due at noon the day before class and then students have from noon until our class period to leave their comments and then the way the course is structured the same day that those three students are blogging… so they’re each writing a blog… they are also facilitating a class discussion… a ten-minute activity… or it can be a presentation… it can just be more of a conversation. They have ten minutes at the beginning of the next class to do whatever they want, and I encourage them to make it the best lesson plan that they have ever seen or the way that they want their ideal course to be structured, and so I encourage them to try things like think-pair-share or to do interactive activities and it’s really exciting to see, first of all, the things that students choose to blog about, because with the readings that we’re discussing there’s so much that you could talk about. I have certain things I want to talk about but those might not necessarily align with what students are interested in within the text, and so having these open-ended blogs allows students to identify what it is they’re most interested in; it allows them to get feedback on their writing from their peers prior to our class session. One way that I’ve come to think about the blog… that I talk to students about it… is as a rough draft for a paper. They’re putting out a thesis… they’re putting out an interpretation… they’re providing some evidence from the text to support it… and then they have this tremendous opportunity to get feedback from all of their peers… and so in the comments the other students will be like, “I really like this point…” “I have another example that can help you support your point…” They might raise objections; they might raise counter points: “Well, have you thought of this other thing?” And so it’s a really great way for them to increase the quality of their writing and their ideas by getting feedback from their peers. Actually, this happened just in our previous class, a student was using a term “devaluing” to talk about sexuality in one of the books that we were reading and a lot of his fellow classmates were saying that word wasn’t working the way that he thought that it was working. So, in his facilitation he kind of talked through the feedback that he got and as a class we came up with a better word that would more precisely name the kind of relationship that he saw developing in the literary text, and so with the class facilitations it provides students with an opportunity to practice their public speaking and to practice standing up in front of a classroom. A lot of the students say that they’re really nervous at first, but that they’re glad in the end that they did it and they always get through it and we always manage… and so this kind of pairing of the blog with the in-class facilitation really teaches students that they are active knowledge producers and that they have something to contribute to the class and that their voice matters. They know that they’re not allowed to just disappear and sink into the background—they’re actually the ones up there in front of the class leading the lesson and it’s interesting to see actually the ways that it increases their performance once they’re back in the chair of the student, because they know what it feels like to be up at the front and so they’ll put out a question and they then get to experience what it’s like to have no one raise their hand and so they become much better as students and much more engaged once they return to their seats and resume that more traditional role of being a student. I never know what students are gonna do for their facilitation. They don’t have to run it by beforehand, so it’s always exciting. I don’t know what they’re gonna do in class today and it’s really made my role as an educator different and I’ve had to learn to listen really carefully to the things that students are saying when they’re up there presenting and my job becomes connecting what they’re saying to the main ideas and the main skills and the main topics of the class. So, for instance, if a student is giving a presentation I might interject and say that’s a thesis statement,… what you just said… you just made a thesis statement and then they start to recognize learning how to make an argument, how to make a thesis statement is one of the skills of the course, but it takes a long time or they’re not quite sure what I mean by that, but when they’re talking they’ll just do it naturally and so my job becomes pointing out to them that they are already doing the things that we’re learning about and just helping them recognize better the ways that their facilitations are connecting to the themes and the skills of the course.

Rebecca: I wanted to circle back to the leaving comments for other students—so they do the close reading, they post about that and then students comment on it. When they’re commenting, how do you help students learn what a good comment is?

Danica: Yeah, that actually becomes a topic of discussion early on in the semester. They’re given a few guidelines: it should be, I don’t know, a hundred and fifty words or so; it needs to make a contribution to the post; it can’t just be “I liked your post” or “I didn’t like your post,” and then what I’ve tried this semester is we implemented—kind of halfway through the semester—this rule that each comment needs to provide a quote from the text so that the commenter is either supporting providing further evidence that will support the author of the blog’s claim, or providing a counter example. One of my students last class, he said “Conversation makes the best interpretation” and I really loved that because they’re starting to learn through the commenting the ways that all academic writing is a conversation among various viewpoints and that when they’re writing a scholarly paper… when they’re writing a research essay… they are inserting their voices into larger conversations; they’re in dialogue with people. It’s not like you write a paper in a vacuum; it’s actually a synthesis of all these different viewpoints and ideas and so I see the commenting as kind of a rehearsal for class discussion. So when we show up in class, say, for instance, students aren’t being particularly talkative, I can say, well, you said this in your comment, and so I know that they’ve already engaged with the ideas and it allows often our class conversation to reach a higher level because they already know what several of their peers’ interpretations of the text are; they’ve already thought about them; they’ve already thought about the pros and the cons and how we might need to complicate some of these analyses; and so it just takes our class discussions to the next step.

John: Do you do anything to ensure that everyone responds to a certain number of posts to make sure that you don’t see everyone replying just to one other post to make sure you get some balance there? Do you have a mechanism for doing that?

Danica: Yeah, that’s a great question. So, for every class students have to comment on at least one post; they’re welcome to comment on more than one, but the requirement is one comment prior to class. I don’t have a mechanism for ensuring that. The class is 25 students, if we have three bloggers, one blog might get ten comments and the others might get four or five, and one way to kind of address that in the classes is encouraging students to think about their blogs, think about the title of the blog, think about the content of the blog, think about how they’re competing for the attention of their peers. I encourage them to say, “Okay, your peers have three blogs to choose from, how are you gonna get them to read yours?” It’s a way of getting them to think a little bit about audience and what is the function of a title. What is the work that a title can do? …and from that introductory paragraph how can they give their reader a sense of what their blogs gonna be about? How can they convince their reader that there’s gonna be a good payoff that their blog is worth reading? And so it’s interesting to see the different ways that they try to attract the attention of their peers. Because they do want those comments and I find they get excited about the different feedback—they’re not required to respond to the comments, but they do often… which, yeah, it’s always exciting to witness. I try to linger, I lurk a little bit on the blogs and I’m often not interjecting in the conversations, but just kind of reading through them and that’s actually the other benefit is that it ends up serving as a mode of formative assessment because I can see what they have understood from the readings and what might be missing… what might be the things that I need to address in the time that I have—what’s not quite getting through to them, either in terms of aspects of the reading that they overlooked or in terms of the skills. So, if I tell them that your blog needs to have a main point; it needs to have a thesis, and I’m seeing that they’re not quite doing that I can then adjust my lesson plans so that that becomes the focus of the next class and I can use their blogs, their own words as an example to say, “Okay, how could we give this blog a stronger thesis?” …and so it’s quite common that we’ll end up editing or revising some of the blog posts. They get projected up on the screen and students, because we’ve created a culture that they’re constantly giving feedback on each other’s ideas, students feel a lot less embarrassed or they understand that we’re all trying to become better writers and so they’re okay with it if I project their blog post and we talk through “What are some of the pros? What are some of the cons? How could we strengthen this?”

John: And it’s a much more authentic learning experience having them focus on audience and trying to build a strong thesis statement.

Rebecca: It seems like the blog post assignment really primes students well for the final projects that you had mentioned earlier that have a public audience because they’re already practicing writing for a specific audience and it’s another writing for a more general audience, I would assume. Can you talk us through that a little bit?

Danica: The final projects for my class often vary, but they’re usually collaborative… they’re usually digital… they’re usually public… they’re usually some kind of creative student-driven element. It’s usually students identifying the topic and then running with it, whether that’s a research blog or whether that’s currently my students in Intro to Multicultural Literature are co-authoring a glossary of key terms for literary studies—I have never done this before. It is a total experiment—I don’t know if they know that this is my first time doing this, so it’ll be interesting. I don’t know if they’ll hear in this podcast, but whatever, it’s fine. So I’ve done different versions of these collaborative public final projects. They vary sometimes based on the content of the course, students’ level of preparation, what are the aims and objectives of the different courses. It’s a little bit different for a basic writing composition course versus a more advanced literature course… and so one of the format’s I’ve done is have students co-author scholarly articles that they would submit to an actual journal, and so I did this in one of my freshman writing classes. We spent the entire semester talking about contemporary issues in education… so related to technology in the classroom… active learning versus lecturing… conditions of educational equality in segregated schools… and about halfway through the semester they were put into groups and they had to identify a research question. They did an annotated bibliography, they developed a whole research project, and then they made it into an article, a short article that they submitted to the scholarly academic peer-reviewed journal Hybrid Pedagogy to see if they could get it published or not, and I had been in contact with the journal’s editors since the summer before the class, so they knew this was coming. It would not have been possible if I hadn’t been working with them because they knew there was gonna be a really quick turnaround time where the students needed to know if they got revise and resubmit, if they got rejected, or if they got accepted… and I knew that this was a wildly fanciful or an unrealistic expectation to ask students to get a scholarly journal article published—these are basic writing students at Queens College—a lot of them are first-generation students… they work jobs… they are English language learners… and so in addition to reviewing of the conventions of English grammar and how to write a paragraph… how to write an academic paper… all things that were new or needed to be reviewed… they were also trying to get their writing published in a major publication… and so what ended up happening with that is that several students got revise and resubmit. But by the time they did it was the end of the semester and finals were happening and so I tailored the assignment a little bit towards the end. I tweaked it, because all semester I’d been telling them “these blogs are important, these things that you’re writing your research, everything, all of this matters because people are actually going to be reading this and you want them to take it seriously and you want them to listen to you. You don’t want to lose their attention halfway through.” … and so we needed to come up with a way that they would still get published even if they chose not to endure the editorial feedback loop of revise and resubmit, or the accept with minor revisions, and so what we had them do is they took the feedback that they got from the editors and several of the groups chose to post to HASTAC.org, which is a tremendous resource. It is an academic scholarly network of 15,000 plus members of scholars and students and academics and artists and activists and so there’s a special group within HASTAC that showcases and features and highlights the writing of undergraduates. So many of my students ended up submitting their final blogs there, but one group did continue—they kept revising their submission and going through the queries that they were getting from the editors and then the copy editors and just all of these stages of the writing process that were very new to them. This is a required writing course… no one showed up at that course eager to do all these drafts and revision and the skills that we teach in a basic writing course… but they continued in that editorial feedback loop for about a year after our class ended and then in August of 2017 their article was published in Hybrid Pedagogy, which was very exciting and so that is now something that they can put on their resumes and I was just so impressed with them for sticking through it because we know everything that goes into writing a journal article but for them they didn’t even know at the beginning of semester what a peer-reviewed journal article was… and so it was like a huge learning process. So, that’s one of the formats of these collaborative public final projects: submitting something to an established publication, which required a lot of willingness on the part of the journal editors to work within a really quick timeframe and the managing editor Skyped into my class several times and talked to students about the journal, helped them with their submissions, they got to pitch their ideas to him, it was great. Some of the other formats I’ve used that have also been good—I’ve had students write explicitly for HASTAC and that’s an opportunity for them to tailor their writing for a very specific community. So that’s something that we did this semester in the writing class that I’m teaching. We read so much about HASTAC… we read about its history. There was an article in Inside Higher Ed calling it the ethical social network and talking about their commitment to protecting their users’ data; we learned about who is a member of HASTAC, who are the different people who are reading it; how does the website organize information—by topic, by tags, by categories and so they were reading and analyzing the site itself before they even started writing their research blogs. So it was a similar process where they identified a research question and they authored blogs that were specifically going to be then tailored for the HASTAC audience, and so one of the big aims of that assignment was this skill of kairos and figuring out how to tailor your writing for a specific community of readers and figuring out what are the conventions? what are the affordances of this specific writing space? and how can I best get my point across to this very specific audience? So, that was useful in helping us have conversations about audience awareness and tone and how you make an argument; how do you convince someone that your point is right without alienating them. So that’s one example of having students write for HASTAC. …and the nice thing about HASTAC is that it comes with a built-in user community. You have sixteen thousand people who are visiting the site and reading things. The other kind of format for these public projects is what we’re doing now with the keywords. Students are co-authoring these individual keywords, they’ve identified specific words that have emerged that they’re interested in throughout all the readings that we’ve done this semester and through our discussions and through the blogs and the final product for that will be something that is hosted on our course WordPress site… so it’ll be a page or an offshoot—I’m hoping to write some kind of table of contents that will link to each of the student’s posts—hopefully there will be media, and so this will be something that they can then share. They can decide that they want to make it part of their professional identity, part of their portfolio, or they can decide not to—it’s really up to them. …and so one of the things that I’m constantly thinking about in developing these assignments is like how to actually connect students to audiences of readers and people who could actually benefit from a keyword entry on memoir or on ghosts… that’s another keyword, apparently we talk a lot about ghosts in my class… and this is coming out of the research that I’ve done on activist pedagogy and really thinking about the role of the teachers connecting students to audiences and people that could potentially benefit from the writing that they’ve been doing. So, thinking about these projects as both a benefit to the students in the class and also to larger publics and communities.

Rebecca: How have students responded to this sort of work? You know, you mentioned that, you know, some of the classes are required courses—students are not necessarily marching in excited to do these sorts of things… so it sounds like you’ve hooked them a little bit. What is their final response to these?

Danica: In general it’s been really positive. The jury’s still out for the semester. We’re gonna do course evaluations I think next week, so I’ll learn more. But in the past, I do a lot of framing around what we’re doing in part because I was always a very willful student and I did not like being told what to do. But if I understood why I was being told to do that thing then I would get really into it and really excited and so with these student-centered assignments and activities I’m always super explicit: this is why we’re doing this; these are what I see as the benefits of this; this is why we’re authoring a set of Community Guidelines; this is why you’re doing a presentation; you’re doing a presentation because public speaking is one of the most valuable skills that employers look for and so when I’m writing your recommendation letters I want to be able to tell them what a great public speaker you are and that’s why I’m asking you to stand in front of the classroom and facilitate this. Also, I should mention this semester and at SUNY Cortland a lot of the students are going on to become teachers and so it’s important to have these experiences at the front of the classroom. I think that being explicit really helps students, and the other thing I’ll say is that I often am explicit about how frustrating student-centered learning can be, and we talked about how it can feel difficult and how sometimes we just wish that the teacher would give us the answers rather than making us figure it out ourselves or making us work in small groups and so I try to create spaces for students to express those kind of emotions and reactions to things. I also try really hard in designing these student-centered assignments… to design… to create the conditions where for instance, we’re doing a collaborative writing project… I try to give them an assignment that actually requires multiple minds and that if they had tried to do that exact same assignment on their own the final product would not be as good as if they were doing it as a group. So I put a lot of thought into kind of carefully constructing these in a way that they will be oriented to succeed in them. Recently I wrote a blog on collaborative close reading, which is a really, really difficult skill to teach—it takes years, you know, for most of us to learn how to do close reading, but I’ve tried to create this assignment that had students work on it in groups, and so rather than having to notice all of a million different things that are going on in a passage of literature, they had a bunch of different minds put to the task and they were all looking at the same paragraph for 20 minutes and dissecting it and they were all contributing their different insights and so rather than having to go at it alone they were able to learn from the different perspectives that the other students brought to the text, and so I think just being really explicit about why behind everything has helped to ensure that the reactions have generally been positive.

John: How have other faculty responded? Have other people started working on building more productive conversations? Have other people in your department started working more on open pedagogy projects?

Danica: Well, I would hesitate to say anything explicitly about my department because I’m so brand-new. It’s my first semester in the department. But one thing that’s been super exciting for me has been to see people… especially with this recent blog that I wrote on collaborative close reading… it went viral on academic Twitter and people have been reporting back, because that’s one of the things that I asked them to do is let me know how it works… let me know if, you know, you have any suggestions for how to make it better… and almost every day I’m getting tweets from people at universities across the country saying, “I tried collaborative close reading and this is what my students did…” and they’ll post pictures of the passages that their students highlighted and so that makes me feel like I’m part of a community that is bigger than my own institution. So, when I’m running these, of course I hope that they will be helpful to my colleagues, but I’m also… I really feel like I’m part of a bigger academic community and part of that is because I post these blogs to HASTAC, and so it really is, people are a community of 16,000, however many users, but then it can get tweeted out. So, even if you don’t have a HASTAC account you can still read the blogs and there’s so many ideas about scholarship being really isolating, but things like that… and getting to talk to people and discuss pedagogy with people at different institutions makes it feel a lot less isolating. But in terms of your question about reactions of colleagues, I have been super lucky both at CUNY and now at SUNY in terms of support for the kinds of things that I’m interested in doing. These are schools with very strong commitments to education, where people are already interested in and talking about student-centered methods and curious and wanting to learn more. The other day I have my students write found poems—which is the genre of poetry where you take some kind of existing document—often it’s a bureaucratic document—and you make it into a poem by cutting it up and whiting it out and mangling it and turning it into poetry and my students created these awesome, awesome found poems; they were beautiful. So, we spent a day in class, I gave them whiteout… I gave them scissors… I gave them tape… They started as banal documents; they made them into stunning poems. They would bring in their tuition bills or song lyrics with offensive stereotypes in them—one of them brought in the transcript from the Brett Kavanaugh hearing and they took these documents and as a way of thinking about language and power they made them into these gorgeous found poems and so I went to the chair of my department and I said, ”Hey, you know, my students created these poems and I would love to have someplace to display them ‘cause I think they’re really awesome” …and not only did she give me permission to tear down what was on the bulletin board, she helped me do it. We tore down these old flyers that had been up there for decades and we put my students’ found poems on and so now we have this beautiful display in our hallway of student work and several of my colleagues have reported seeing students stop and read the poems and take pictures of it and so they’re excited to see their work has become part of this gallery. In general I’ve just been really lucky and fortunate to work with colleagues who are similarly invested in helping students. I did think of a few ideas for people who might not be so fortunate on ways that they could start doing student centered things. The first would be, and I’ve already mentioned this, creating a free profile on HASTAC.org because there are so many people out there doing really creative and exciting things in their classrooms… and connecting their classrooms to larger movements for social justice… and thinking about how do you engage students in really important discussions about contemporary social issues… and so HASTAC has been a phenomenal place for me to connect with other people who are doing that kind of work. I try to start small, and so something like think-pair-share is so easy… it’s taking an index card, giving students 90 seconds to respond to some kind of open-ended prompt, then they turn to the student sitting next to them, they share their responses and then we go around the classroom and I transcribe each group’s answer to the question on the board… and so their ideas become the material that I then get to teach the course through, and we crowdsource responses to some question related to whatever the topic of discussion is that day and something like that is so easy, it’s so simple—we have gone through… I would say we’re in the thousands of index cards in terms of my courses this semester. Because the students like it and they recognize that that changes the classroom dynamic. They recognize that suddenly it’s not just one or two students dominating the conversations. So, when they get up to the front of the class and they get to facilitate something I would say, I don’t know, 65, 70 percent of them choose to do think-pair-share because they recognize that it really lowers the barriers of anxiety about participating in class—everyone has 90 seconds to scribble something on their index card and it’s only an index card,— it’s tiny—there’s not any kind of pressure to write something beautiful and then that becomes just such an easy way to really transform the dynamics of the classroom… low cost… low time investment… and even when I’m thinking at the bigger scale of assignments and rethinking the research paper so that it’s not just being submitted to the professor, but it’s for a public audience, I try not to overhaul everything at once. So it’s like each semester I’ll try one new assignment. Not throwing everything away and starting from scratch each time, because it takes a lot of energy to do these things. And so thinking about how we can make small changes and experiments but not overwhelm ourselves or our students… and the other thing that I would suggest if somebody finds themself in a situation where they want to start trying these things but might not have the kinds of tremendous support that I’ve been lucky to have is… there’s just so much research out there on the effectiveness of student-centered pedagogy. We’ve read a lot of it in my course this semester on writing and education. We read a lot of the studies that have shown how positive of an impact it can have on students to discover ideas for themselves and to work in small groups and solve problems and arrive at answers rather than sitting and listening to a lecture. Just kind of having some of those studies in my back pocket—I’ve always felt that if I was called upon… you know, “Why did you do that? I can’t believe you let the students help assess each other’s papers.” I would have some things that I can cite, that I could go back to to say “Well, actually, it’s been shown that asking students to metacognitively reflect on the implications of their writing is a great strategy.” So that kind of thing has been really helpful for me in terms of thinking about relationships to colleagues and different reactions to this kind of pedagogy.

Rebecca: We normally wrap up by asking, what next?

Danica: Well, at the small scale, I guess, we have this digital glossary of keywords that will be coming out from my multicultural literature students… going to learn all about ghosts and power and assimilation and why these words are important for how we think about and analyze literature… so really excited to see what they do with that… and then I guess at the bigger scale I’m working on a book on the activist pedagogy of teacher poets from the 1960s and 1970s, and I’m hoping that some of that work will help us really understand the ways that a lot of contemporary student-centered practices… things that we’ve talked about today… a lot of them emerged in the 60s and 70s and especially in relation to the critiques of power emerging from the social movements of that era from the women’s movement and the civil rights movement and protests against the Vietnam War… and so I’ve been thinking a lot about how those critiques of power necessitated new relationships of teaching and learning and this was especially happening in the work of poets who I’m interested in and so that book is also considering the ways that interactions with students shaped American literature in ways that we rarely consider and also the tremendous role that poets and authors and especially feminist poets have played in creating a lot of the contemporary student-centered pedagogy that we know today to be so effective.

Rebecca: That sounds really exciting.

Danica: Yeah, it’s really fun to work on.

John: In some of your posts you’ve listed a large variety of techniques that people can try and we’ll include links to those in the show notes as well. Thank you, this has been a fascinating discussion and we look forward to hearing more about what you’re doing.

Danica: Thank you so much for having me, this has been really fun.

Rebecca: Yeah, thanks for joining us. [Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

59. Gatekeeping in Math Ed

Teachers at all levels often play an important role in influencing the educational and career paths of our students. In this episode, Dr. Marcia Burrell joins us to discuss how math teachers play a critical role as gatekeepers who may either welcome students to or provide a barrier to student success in all STEM fields. Marcia is the Chair of the Curriculum and Instruction Department at SUNY Oswego.

Show Notes

  • National Council of Teachers of Math (NCTM)
  • Budapest Semesters in Math Education
  • Polya, G. (1973). How to solve it: A new aspect of mathematical method. Princeton: Princeton University Press.
  • The Polya Approach Used at the University of Idaho
  • Bjork, R.A. (1994). “Institutional Impediments to Effective Training”. Learning, remembering, believing: Enhancing human performance.
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Miller, L. & Spiegel, A. (Hosts). (2015, January 23).Invisibilia: How to become Batman pt. 1 [Radio broadcast episode].
  • National Research Council, & Mathematics Learning Study Committee. (2001). Adding it up: Helping children learn mathematics. National Academies Press.
  • Brandsford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. National Academy Press.

Other resources:

  • Larson, M. (2016). The Need to Make Homework Comprehensible. National Council of Teachers of Mathematics.
  • Stinson, D.W (2004). Mathematics as gate-keeper: Three theoretical perspectives that aim toward empowering all children with a Key to the Gate, The Mathematics Educator14 (1), 8–18.
  • Burrell, Marcia (2016) Gatekeeping in Mathematics TEDx talk at OCC. January 29, 2016.

Transcript

John: Teachers at all levels often play an important role in influencing the educational and career paths of our students. In this episode, we examine how math teachers play a critical role as gatekeepers who may either welcome students to or provide a barrier to student success in all STEM fields.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Marcia Burrell, the Chair of the Curriculum and Instruction Department at SUNY Oswego. Welcome, Marcia.

Marcia: Thank you.

Rebecca: Welcome. Today our teas are…

Marcia: Earl Grey with caffeine.

Rebecca: Extra caffeine. [LAUGHTER]

John: Mine is just a pure peppermint tea today.

Rebecca: And I have a jasmine green tea.

John: We’ve invited you here to talk a little bit about the work you’ve done on math instructors as gatekeepers. What does it mean to be a gatekeeper?

Marcia: Well, I like to use the word gatekeeping because sometimes gatekeeping has to do with an open gate, where you can just slide right through, or someone gives you the key, or they’ve given you the secret password, or it’s a barrier, where if you don’t really know what the hidden curriculum is about passing through the gate then you could stay there and be turned away. And in mathematics a lot of times people are afraid of math or they’ve been socialized to think they cannot do math and it’s really a gate that’s been created either by themselves through socialization or it’s been created by a math person or by someone like a parent who said, “oh, don’t worry, I wasn’t good at math either.” So, when I think about gatekeeping and mathematics it’s really about barriers that are created by us or barriers that are created by others, or for people who are really successful in mathematics, they have an opportunity to open the gate; there are certain things that they can do that will make people pass through the gate more easily.

Rebecca: I think our students can empathize with the idea of gatekeeping when it comes to mathematics—you hear them talking about these stories of certain situations where the barriers have been in place for them, or sometimes that’s faculty. For example, I’ve heard many times in creative fields where the creative faculty might say, “yeah, we know you’re not great at math but you have to take math,” or I had a situation when I was a kid in middle school—I remember distinctly middle school teachers saying “the women in this class aren’t going to do as well” and then I remember the few of us banding together and then we got really good grades on this final exam that we were told that we wouldn’t do well in. I think that those narratives are certainly there and it’s interesting to think about it not only from the person coming to the gate but also from the gatekeeper perspective, which leads me to the question of, what are some things that gatekeepers do that keep people out?

Marcia: I’m gonna focus on math people mostly, where sometimes they say things like maybe in a beginning level math course, “Why didn’t you know that? You should’ve learned that before. I don’t understand why you can’t do fractions.” So, there’s vocabulary built into a lot of us where we send out messages which get people to realize, “Oh, there’s something wrong with me; I should know how to do this.” So, they start imposing those same messages on themselves. The other thing that I think is important is in mathematics there’s always been a stratification about who can do math or who should do math and who can be successful in math. Often, as you just said, women have stories about fighting to get into a advanced math class because they didn’t do very well on some class but they were willing to work hard. So, certain populations are harmed because they’re socialized that way that when women have trouble in mathematics we say, “Oh, we should make it easier; you should do a group of courses that are not gonna lead you to calculus in high school,” but sometimes when men struggle we go, “Oh, struggles perfectly fine.” In the U.S., teachers make it easier for students to learn; they give them answers, they work out all the details. When I say give answers, I mean they work out all of the problems so that it’s really just rote, as opposed to in other countries, struggle is actually honored—hard work and struggle is part of the mathematics learning process, where in the U.S. sometimes we don’t allow people to struggle. If you got a B in Algebra I, well, you don’t really need to take Algebra II because the minimum requirement in New York state is Algebra I, and the fact is struggle is a part of the learning process. Historically, we’ve always stratified who is successful in math or who can take math and the level of courses that people can take. Plato 2,300 years ago believed that everybody needed arithmetic, but the advanced math was relegated to philosopher guardians, and in the 1920s the National Council of Teachers of Mathematics argued to have mathematics part of the curriculum, and between 1890 and the 1940s there was a growth in public schools and the perception was that sometimes they weren’t sure that students had the intellectual capability of doing some of the mathematics that NCTM thought was important. But remember in the 1950s the business world and industry said, “What are you guys doing in schools? The people that you’re putting out there can’t do mathematics.” Well, that was mathematics for a purpose and then Sputnik happened and all of a sudden math became this subject that we wanted to make sure people had. But think about how many English classes do people take—one or two in high school, but in high school students often take four or five math courses. I’m not saying they’re not important, but it really forces us to think about mathematics as an elite subject when gatekeeping from my perspective is it’s not about an elite subject, it’s everyone can do math; people are born mathematical and everyone should have an opportunity to do the subject and not fail at it, but struggle and make movements towards whatever learning they need to do.

John: So a lot of this sounds like our society is creating or emphasizing or encouraging the development of fixed mindsets in math where many messages coming through (as you both have mentioned) in early childhood discourage people from thinking that they’re able to do math and only the elite can get through. Is that common in other cultures?

Marcia: I mentioned earlier that in other cultures hard work and struggle are honored and I witnessed in Budapest, when I was visiting there as part of my sabbatical, that students were asked to go up to the board and struggle through a problem, even if they had no idea. And we do a lot less of that because either you know the answer or you don’t; that doesn’t really work that way—it is an iterative process. I used to work on problems and maybe get a little frustrated, put it away and the next day I’d look at it and I go, “Oh, now I get it.” It’s really about process. The NCTM standards talk about process and product, and if you want people to learn mathematics then you really have to emphasize process, working in teams, giving people a chance to try things and fail but also collaborate with others to ensure that maybe there are multiple ways of approaching a problem, but if you’re not allowing students to talk with one another and work it through, then sometimes they think there’s only one way to do it and it really doesn’t improve their mathematical abilities. Mathematicians are about process—there are certain skills that mathematicians use. Good mathematicians persevere through problem-solving. They check their answers using different methods, they plan how to solve a problem versus jumping into a solution, and they justify the answers and communicate with others. Good mathematicians don’t just know the answer; it’s a process, and there’s even collaboration between mathematicians, but when we teach it on the K-12 level, we say, “This is what you need to learn and you need to learn it in a specified amount of time,” and so a lot of times students are turned off by the way we teach mathematics. Opening the gate is really about helping teachers rethink how they actually teach mathematics. We have a lot of data about how to successfully teach math, and it’s about problem-solving, reasoning, communication, connections and representations, but if you’re just gonna stand at the board and write the answer to a problem, that doesn’t help people really connect to how you came to that problem. So, gatekeeping is about getting teachers to rethink how they’re teaching mathematics and what they think is important. Process and products are important, but process is actually more important.

Rebecca: You mentioned mathematics as a collaborative process, but in my experience in K-12 I don’t think I ever worked with another person once.

Marcia: It’s funny you mention that. Again, the stratification stuff is huge. I attended a program called Budapest Semesters in Math Education and it’s geared for Americans, Canadians to come to this program. They’re interested in both juniors and seniors to come and learn about the problem-solving approach to mathematics. These are students who are mostly math majors, but they could be math ed majors, and they are sent to these schools where they’ve selected the top students in mathematics to use a problem-solving approach and what happens is they give them a problem with no background and they ask them to work out these problems. They can use their textbooks, they can use calculators, but the fact is our students—Americans and Canadians—get to witness students almost trying anything to work out these high-level mathematics problems—sometimes they’re theoretical, sometimes they’re applied. But what the students say who are in this particular program—and I got to be in these classes with them—was, “Why can’t we have all students use some of these processes?” And the processes are really just the things we already know that good mathematicians are supposed to do, sort of George Pólya, you know: analyze the problem, look at all the facts, try something, test your answer. But you actually get to witness that. So, when you asked me “None of the classes I ever went to that were collaborative and problem-based and working in teams,” well we seem to have an idea that only the gifted and talented or special programs will allow kids who already show aptitude to do mathematics in that particular way, and the fact is I visited a school in Budapest where this teacher who’s been working with the gifted and talented students got permission from the parents to try this problem-solving approach for a ninth grade through 12th grade. They had to get sign-offs by parents, because of course, in our system, if kids don’t know certain things by the end of certain grades then their opportunities—another gate—for getting into the university and going through the career path are cut off. So, these parents had to sign off that they were going to risk that what she was gonna to do over the next four years was gonna be helpful to their students and that they wouldn’t be harmed by doing this problem-solving approach. I witnessed several math classes where this teacher had been working as part of her dissertation to have students go through this problem-solving approach—it’s not just Pólya; there are other… Pósa there’s a Pósa method—I met this gentleman who, he was in his 80s and he invented the Pósa method and he’s one of the top mathematicians in his age… in his day, but he devoted his life to teaching problem-solving to kindergarten through grade 12. But the point that I’m making is, I witness students who had been through this process, and they were explaining problems to their peers on the board in ways that I haven’t seen good math teachers explain. But they built these kids up from start to finish to be confident about what they knew, to work in groups, not be afraid to make mistakes, and I think that we can do more of allowing students to learn not just at their own pace, but learn what mathematicians do—the process of engaging with one another if we weren’t so afraid of the whole accountability—what do kids know at the end of 12th grade? What do they know at the end of 11th grade? It’s recursive. Some things they learned in ninth grade in Algebra I will come back in Algebra II and when they’re college students they’re gonna pull the algebra and geometry together, if we allow it, as opposed to looking at these areas as completely separate things. One of the things about gatekeeping is that teachers have to think about students as already being competent; they’ve got to provide students with scaffolding so that students that are in different places have an opportunity to demonstrate what they know. I also think that we have to have high expectations, but we have to let students understand that they can extend the learning if they take some risks; that’s what good mathematicians do, and then we have to exhibit in depth knowledge as well as subject matter knowledge. So there are certain things that gatekeepers—math teachers—can do, but they’ve got to trust that students can learn, and we’ve got to keep the expectations high, but also scaffold for them so that they’re successful.

Rebecca: …a lot of evidence-based practices.

John: Yes, I was just going to say a lot of what you’re talking about, there’s a tremendous amount of research supporting that, not just in math instruction but across the board. In terms of providing students with challenging problems—you have the desirable difficulties of Bjork and Bjork, for example, and in terms of learning from mistakes, that shows up in all of the research on teaching and learning and it’s something that Ken Bain talked about when he summarized some of this research in What the Best College Teachers Do, and it’s also shown up in several of the books we’ve used in our reading groups, Make It Stick and Minds Online, for example: that retrieval practice, low stakes testing, where students can make mistakes and learn from mistakes, is effective in all types of instruction. So, these are really good practices that seem to be mostly neglected in math instruction.

Rebecca: I was expecting John to also mention something about growth mindset. [LAUGHTER]

John: I think I already did a while back, but treating math as something you’re either good at or not good at by teachers and by families and by our culture discourages the development of a growth mindset, and that’s really important. This year I’ve completely flipped my large microeconomics class and one of the things I had them do is before each class I asked them to do some readings and then I asked them to work through some problems in the readings; I have students submit a short Google form, where I ask them just two questions before each class. The first question is: “What have they learned from this reading assignment before that day’s class?” And also, “What are they still struggling with or what don’t they fully understand?” And half to two-thirds of them before each and every class list, “I have trouble interpreting graphs;” “I have trouble understanding graphs;” or that “I have trouble computing these things,” and that’s all basic math, and of course they have trouble doing it when it’s the first time they see it, but they see it as a barrier— “I’m just not good at it,” and every day in class I’ve been trying to encourage them to say, “Well, you may not do it now, but you can get better at this;” “You haven’t yet mastered this;” “You’re not yet good at this, but the more you do it the easier it gets,” and we’re not always seeing that happen, and we see that in lots of areas.

Marcia: Yeah, I think that students are more willing to say “I’m not good at math; I don’t have any experience with math,” but they would never say, “I can’t read; I’m not good at reading.” They might say it, but it’s socially acceptable to say “I’m not good at mathematics,” and the fact is when you look at a group of kindergarteners and they’re in a classroom, they’re all learning for the first time how to add and subtract and they slowly… I’m sorry, through some of their elementary school teachers who often are afraid of mathematics, and they say little things, “Oh, don’t worry about that, it’s okay to not be able to do that, we’ll work on that later on,” but they say it in a way that sometimes gives students permission to say, “Oh, I don’t have to learn that—I’m a girl, I’m a student of color, I don’t have to learn that because the teacher said she doesn’t know it either,” and so one of the concerns that I have for how we train childhood educators is we force them through, at least on our campus, these two math classes where they go kicking and screaming, but the fact is we almost need to reprogram them to think about the things that they can do mathematically and then build curriculum around them. It’s not always about the fact that the way you learn is the same way that all the kids that you’re teaching learn; it’s more about how do you change your perceptions about mathematics. There’s something on NPR, and I’ll have to find the reference a little bit later on, where this young man who was blind learned how to ride a bike, was sent to school, and people couldn’t even really understand why he was able to do all of these things as a blind person—well, his mother decided to treat him like he was a sighted person and it’s a Batman series, where the fact is, if you convince someone that they can do something and you believe it, then all of the things that you do to work on their perceptions about their capacity will come through. But first the teacher has to believe it and then they have to do all of these things to scaffold it. The fact is that, and again, I’ll have to find the researcher, but he did this study where he told all of his researchers that these mice were smart mice… these mice were everyday the same mice… what happened is the researchers came in and they treated those mice like they were smart—they handled them differently, they had them run through whatever people do in psychology with mice, and then he came back later on and said “All of these mice have exactly the same capabilities.” Well, that works in exactly the same way in the math classroom; students come, and if we believe that they’re capable and we come off and treat them with respect about what they have learned and how to build on that, then we’re gonna see better progress in their learning. I have to come back to the gate because the teacher has a lot of power to make the gate accessible or make the gate a barrier, and the barrier is really just the messages that the teacher says to the students and to herself about success in mathematics, and we lose entire generations of people when the gate is closed to them mainly because of perception.

Rebecca: So much discussion of gates it should be important to note that in front of Marcia is this picture of so many different kinds of gates in our conversation. Can you talk a little bit about the gates that you have in front of you?

Marcia: Yeah, I decided to Google different kinds of gates and when you think about the Brandenburg Gate or you think about gates like this one —remind me what this is called; this is in Cincinnati—the arch; this is really a gate, but this shows an opening to something, so when you think about gatekeeping in mathematics, I want us to think about people being gatekeepers for accessibility. So when you look at those pictures you think of when you’re going through the turnstile to pay with an EZ Pass. That is a barrier. If you don’t have money, you don’t have an EZ Pass, you’re not getting through, but if you look at the door to no return like in Benin it’s an opening to the next world just like certain pictures of gates just have you think differently about openings and closings.

Rebecca: There’s some like the dog pen where there is no way in or out; it looks like that one’s just closed forever.

Marcia: Yeah, which one is it? This one or this one? Right, I mean this has a gate, but often people are closed inside of thinking that they can’t do math and they can’t be successful. The job of a teacher would be to help them jump over that particular gate or find a different way to think about opening that particular gate. If you’re a dog and you’re inside of a pen, I think you’re just gonna need somebody to lift you up over that gate, and I think about that with teachers that what they have to do with each individual student is completely different, but their responsibility is to help them understand that they’re all mathematicians and they all have capacity for success in mathematics.

Rebecca: You’ve talked a little bit about how gatekeepers can open the gate or provide the leg up over the wall, or whatever it is, right, that’s there. Can you talk a little bit more about how to be inclusive and how faculty and teachers can really support this environment that would allow for problem-solving and allowing students to fail and try again and to iterate and eventually succeed?

Marcia: I’ve thought a lot about elementary school students and middle school students, where you’ve probably heard about the Montessori Method. The Montessori Method, you work with individuals to build from what their interests are and it turns out that students without a lot of direct instruction can complete whatever the curriculum is for that grade level by mapping to their interests, their strengths, and projects that they do where they’re learning the mathematics in ways that might be considered non-traditional. In the Montessori Method, they’re not just looking at memorizing times tables; they’re looking at multiplication as repeated addition, they’re looking at visualizations instead of just looking at a text. And the fact is that sometimes, I think, that if we allowed students to individualize their learning, especially in middle school and high school, that there’d be more progress than forcing students through the curriculum where each week they’re expected to learn something but they’re not learning it, they’re sort of just being dragged through the mud. And I have a lot of respect for my peers who are math teachers. I was a math teacher where I felt like I know what that kid needs, I need to take time to help that kid through what they need, but I didn’t have the courage to stop what I was doing and figure out how to individualize or make them work in small groups. I was a successful K-12 teacher, but I feel like I started to figure out what was needed when I made the decision to leave. So, part of my job as a math educator is to help our candidates who are gonna be teachers in schools to have the courage to do what they know is right: think about their love of mathematics and give kids problems that are theoretical and have them try it; give them applied problems, give them things where they have to use visualizations and not just know the procedures, but also understand the concept.

John: And also perhaps to use peer instruction, as you talked about, where students explaining things to each other reinforces learning for each student.

Marcia: Yeah, and sometimes the things that we expect of what we call the gifted and talented are exactly the things that other students can do but we’re afraid to take a risk, and I met earlier this afternoon with one of our adjuncts that’s teaching math methods to our graduate students and she said her job is to teach her candidates how to be good teachers, and sometimes that means forgoing what they think they wanted accomplished on that day and building something fun that’s gonna get students to see that math has many openings, not just following things through rote or through memorization. So, I had a really nice conversation with her because she does work in the school systems, but she’s teaching a course for us and she uses constructivist approaches. I have many peers that are still engaged in this math war that it has to be rote, it has to be step-by-step. In the constructivist approach, you care more about the process that students engage in and there’s a program that I listen to on Sunday morning it’s on NPR where it’s a puzzle and the puzzle is usually related to a vocabulary puzzle as opposed to a math puzzle, but the type of thinking that you have to engage to solve those puzzles really is mathematical thinking, so I love those puzzles, but they’re all couched in word puzzles… but it’s really mathematical thinking… and so I think the teachers need to use more of those word puzzles to bring people in so that they understand that they’re engaged in mathematical thinking—it’s just not called mathematical thinking. One of the other things that I wanted to mention before I run out of time is we are heavily tracking students into particular tracks. Sometimes you’re in the track where you’re just going to do Algebra I, and sometimes you’re in the track where you’re gonna get to do Algebra I and Algebra II, and maybe you’ll get to do Geometry, but some of the best learning occurs when there’s heterogeneous grouping and there’s less tracking. This gate stuff, these gatekeeping, really reinforces tracking, which when students come to SUNY Oswego and they’re in a remedial class and don’t know why they’re in the remedial class, because they may have been tracked in a particular way and cut off many, many job opportunities or majors because they were tracked in a particular way, and that is gatekeeping that occurs in fourth grade. And again our responsibility for our childhood educators is to get kids to think more broadly about what mathematics is; it’s not just arithmetic, it’s not just geometry, it’s not just theoretical problems; there are many types of problems that childhood people could engage students in that wouldn’t shut the door to possibilities 10 or 12 years later when students find out that they were tracked in a way that makes it so that they could never do graphic design or they could never do engineering or something else that they didn’t really understand was possible because somebody closed the gate early on.

John: …and that’s really important because most of the growth in income inequality is due to differences in educational attainment and the returns to education. And the returns to education in the STEM fields is far above the returns in other areas as well. So, keeping people out of those areas means that the people with those areas end up doing really well, but the people without those skills end up in jobs that are perhaps overcrowded with lower job prospects, lower prospects of growth and it helps reduce social mobility and economic mobility. It’s a serious problem in our society; it’s the worst we’ve ever seen it in the U.S.

Marcia: Yeah, I can’t connect it completely to perceptions, but a long time ago I taught a remedial math course at Clinton Community College and I had a student in that class and she was a smart person—I think everyone is smart—but I walked through how to study math, how to approach it: you are capable, work hard, keep asking questions, and about 10 years later I got a postcard from her—this flabbergasted me; she was in a remedial class and she had entered a PhD program in mathematics and she said it was just about the fact that somebody finally showed her how to study math—it was read the textbook, try the problems, come to class, listen maybe to the lecture, don’t be afraid to make mistakes; when you’re tired take a break. There are certain things We know that people can be successful in mathematics but we keep thinking that it’s this magic wand thing; it’s not a magic wand thing. We actually know —there’s research from Adding it Up —where we know exactly how people learn math well. The stuff research from Bransford, which how to study mathematics, how to learn mathematics, it’s written in black and white from large-scale studies, but then we return to the rote memorization, follow these steps and that’s not the beauty of mathematics at all.

Rebecca: One of the things that I think is really interesting about what you’re saying is that societally we might think, “Oh, fourth grade teacher… not really gonna have a big impact,” but you’re really talking about this fourth grade teacher is not a gatekeeper of the little gate around the garden; this is like the gate to the universe.

Marcia: Absolutely, and most of our math candidates who are not math concentrates—they’ve got to take these two four-credit math courses—will say, “I just need to get through this class; I hate math.” If you hate math it comes through loud and clear in your teaching; it’s really difficult to mask that. I taught a math for diverse learners course that the School of Education and Arts and Sciences Math Department and Curriculum and Instruction collaborated on and it was a math for diverse learner, so some of the things that I’ve been talking about here was in a full graduate course, and students would say, “Well, I never really thought about that; I thought everybody was gonna learn math the same way I learned math”—you’re a math person, I shouldn’t even say that. You’re a math person—you came through the system and you were successful in the current system, but if you want to build the next generation you’ve got to think about some of these other factors—you’re gonna be in a system, and as we’ve talked about systems, you are part of the system and you do have power to make changes to it, even if it’s perceptions, even if it’s just giving students the perception that you care about their learning and that they can succeed, and so this is really important to me. There are three principles: teachers must engage student misconceptions, understanding requires factual knowledge and conceptual understanding, and a metacognitive approach enables students self-monitoring. If I think about gatekeeping, if teachers kept those three principles in mind, they’re not mine—it’s in the research. This is sort of revolutionary because we don’t want to restrict people to thinking that only certain people can do mathematics, but if math teachers, whether they’re childhood or adolescence, or university teachers think about what good mathematicians do, they’ll follow these three principles and it might move us forward. I know it’s a big deal because the successful people want to keep what they have to themselves, but I think we miss out on the potential of entire generations if we don’t give them access to opening the gate through mathematics. When the Common Core came out teachers had the perception that they had to give these problems to students and parents would call and complain—“I can’t even do these problems; these aren’t the problems that I did when I was a kid”—well, the fact is we weren’t supposed to be sending these problems home; we were supposed to be doing those problems in class, and so a lot of the Common Core mathematics was supposed to be using manipulatives and getting kids to talk about how they think through the arithmetic problem. They were sending home problems and parents were complaining they were spending two or three hours to work through these problems, and there was an article put out—it was an NCTM—where they said what is your problem? No, don’t send these problems home for kids to fight with their parents, ‘cause that’s just gonna reinforce, “Oh, I couldn’t do math either;” it was supposed to be completely done in the classroom in collaborative groups, but we’re still not interested in teaching in that way. So, we sent home the homework—well, you could have been sending home memorize these timetables just as we did 20 years ago or 30 years ago, so finally NCTM put something out to help math teachers in the K-12 area not to send home these problems that would take parents two to three hours, but to rethink the organization of their classrooms where students could work on problems and have fun with mathematics, and the fact is that there are reforms that mathematicians fight about; there are a whole host of mathematicians that said Common Core was bad; Common Core is not bad, the way it was implemented was bad, so now we’ve done some backtracking to think about the fact that when you carry, when you’re subtracting or you’re adding, why do we do that? And the Common Core got students to make sense out of place value and make sense out of what it means when we carry this is about the tens place or the hundreds place and whenever you have new curriculum, Common Core or what was the curriculum in the ‘50s, I can’t remember… the new math… there’s always new math, it’s just an approach to make it more inclusive, but sometimes the way we roll things out makes it difficult, at least for the next generation of teachers, so I’m pro-reform movements, but we have to take the time and the energy to implement it in a way that’s actually gonna be useful—we just keep going back to the way we taught math a hundred years ago.

Rebecca: It sounds like what happened was faculty who knew how to do things a particular way get handed something that’s different but not a way of demonstrating or doing the different, right, like…

John: …without the professional development needed to allow them to implement it effectively.

Marcia: Correct. That’s correct.

Rebecca: The method doesn’t match the material.

Marcia: Exactly. At the same time they were putting out that students have to take a main assessment in fourth grade and eighth grade, but those assessments didn’t really align to this new Common Core curriculum, and so lots of things have changed over the last, I’d say seven to ten years, and we’re sort of coming out of that. When students come to the university level we still expect them to know mathematics. Do you remember twenty-five years ago they changed the math curriculum to be Math A and B, Course I, II, and III? New York state was the only state that was really thinking more globally about, “Wow, it doesn’t always have to be about algebra—it could be about statistics, it can be about more applied,” but the fact is universities didn’t change and we were still expecting students to know this narrow curriculum but it did broaden what people thought about mathematics, but it didn’t really help a lot of those students because then they were closed out of particular career areas because they might have been in a school that embraced applied math or embraced business math or something that might not connect to what they would do at the university level.

John: You’ve also been involved with Project Smart here at Oswego. Could you you tell us a little bit about that and how it relates to math instruction.

Marcia: Project Smart was a thirty-year project where teachers came to SUNY Oswego for summers to do professional development, math, science, technology. There are some teachers retiring over the last couple of years that came to Project Smart right from the beginning. We brought people in like Damian Schofield in the early days to learn about human-computer interaction. We brought people in from music and from art to help teachers integrate other things into their teaching, so they used to come for three weeks, then they came for two weeks, then they came for one week, then we built it into the department where faculty got released time to go into schools and work with teachers from the bottom up to think about how to improve teaching in their classroom. Project Smart really honored the work that teachers did because we would say, “What do you want to improve in your classroom? Are there particular things that you know students are struggling with?” This past year, funding for Project Smart ended, but the institution is still supporting individual faculty to go into schools and work with teachers to build classrooms that connect with the learners that they have in front of them. It’s more connected to what’s called a professional development school, where at the university we have the latest about how to teach, whether it’s math or English or social studies or modern language, and then we go into schools where they’re dealing with kids every single day and we try to help them figure out how to improve as a teacher; we meet them where they are; we build from there, so Project Smart is over—I’m not gonna say it’s dead, but we have a different system to work on professional development schools but just in a different way.

John: So you’re still doing the same thing even though it’s not under that official title?

Marcia: Correct. Correct.

Rebecca: We always wrap up our episodes by asking, what next?

Marcia: Oh my goodness, thank you for asking what next. After returning from my sabbatical, where I had the opportunity to be part of Budapest Semesters in Math Education where I got to see classrooms where students were using Pólya’s problem-solving approach in addition to something called the Pósa method, I worked with Josh McKeown, who’s from international education to reduce the cost of the Budapest program, so we’re working to recruit math students, both childhood and adolescence teacher candidates, as well as straight math candidates to consider going to Budapest over a winter course for one or two weeks over winter session or during spring break. What would they experience if they went to a short course? They would visit classrooms using the Pósa method, they would sit in on some of the math courses at BSME, where teachers are actually showing how to use a problem-solving approach in mathematics, where sometimes our students say “You talk about problem solving, you talk about the constructive approach, but no one is doing it so we don’t really know what it is.” The next step is to work with international ed to get a group of students to do the BSME program.

Rebecca: That’s really incredible.

Marcia: I’m excited about it too and I hope to also re-institute my math for diverse learners course because through that course I reinforce that I believe students should have access to high-quality, engaging math instruction. I believe all students should have mathematically rich curriculum. I believe all students should have high expectations and strong support, and we’re all gatekeepers— we are change agents and we control the gate. I think it’s ambitious because many people don’t agree with me saying that mathematics needs to be more inclusive, but that’s what I’ve been working for my entire career and I hope to continue that way.

Rebecca: Your work is incredible and we’re really excited that you’re doing that work.

Marcia: Thank you.

Rebecca: I know as someone who’s in a field that you don’t always associate with math—I believe in math and so I hope we can all help support your initiative.

John: It’s a major social justice issue.

Marcia: It’s a huge social justice issue because, again, what happens is often students of color, students that come from poor families may or may not have had the best math instruction. I mean, it’s a big cycle, and when they come here we should be able to help not just convince them, but this is a public institution. We should be able to provide access for them to reach whatever goals they hope to. We should be able to take students where they are and help them achieve whatever their focus is, whether it’s math related or not.

John: Well, thank you.

Rebecca: Thank you so much.
[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

58. Role-play

Do your students sometimes settle for a superficial understanding of your course content? Role-playing activities can provide an opportunity for students to become more fully immersed in the academic dialog of your discipline. In this episode, Jill Peterfeso joins us discuss a variety of role-playing activities that can be implemented into a single class session or over a more extended period of time. Jill is an Assistant Professor in and the chair of the Department of Religious Studies at Guilford College.

Show Notes

Transcript

John: Do your students sometimes settle for a superficial understanding of your course course content? Role-playing activities can provide an opportunity for students to become more fully immersed in the academic dialog of your discipline. In this episode, we’ll discuss a variety of role-playing activities that can be implemented into a single class session or over a more extended period of time.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Jill Peterfeso, assistant professor in and the chair of the Department of Religious Studies at Guilford College. Welcome, Jill.

John: Welcome, Jill.

Jill: Hi, nice to be here.

John: Our teas today are…

Jill: I am drinking candy cane tea; it’s a black tea. If you like peppermint tea, this takes it up a notch with even more sweetness. It’s really delicious.

John: What brand is that?

Jill: Adagio.

Rebecca: That sounds right up John’s alley, actually.

Jill: Oh really? For Christmas a couple years ago I asked for some and my parents are like, “Oh, what size?” And I was like, “You know what? I don’t know. A pound.” Well, you know how much a pound of tea is? [LAUGHTER] So I have enough to last me like a decade.

John: I had a mix of tea where it was peppermint, spearmint and tarragon and I got a pound of peppermint, a pound of spearmint and a pound of tarragon.

Jill: My gosh. [LAUGHTER] Yeah.

Rebecca: Speaking of lifetime supplies. I have English afternoon.

John: Again?

Rebecca: Yeah.

John: And I have blueberry green tea. We invited you here to talk a bit about how you’ve been using role-play in your classes. Could you give us some examples of what you’ve been doing with this and in what context?

Jill: Yeah, absolutely. This idea of working with role-play comes from my own interest as a theatre person in high school and college and even into my adult years and also just this memory I have of doing theatre where stepping into the role of another person opens up your mind in really different ways. I’ve devised a number of different things that I do in the class, sometimes borrowing from others, sometimes doing completely experimental assignments. So, I think that it’s sort of three different levels of immersion into role-play. A level one thing, for instance, might be I use dialog tests where I have students imagine dialogues. For instance, historical figures John Winthrop and Anne Hutchinson are having a conversation about conversion… what do they say to one another? And so instead of writing an essay on the thesis of conversion in Puritan New England, I have students imagine a conversation between these two historical actors. So, that would be an example of something that’s level one. Something that’s more level two… I often invite students to take on the voices and the ideas of authors, theologians and theorists that we’re studying. For instance, I teach an upper-level Holocaust class where we often read a lot of excerpts from very dense critical theory, for instance. I will assign students to different authors we’ve read… someone will get Habermas, someone will get Adorno, et cetera, et cetera, and then we come together in a colloquium setting and they need to speak in the discussion as that “author” or that character… and then sort of the deeper level of immersion would be something like reacting to the past, which is a very well established pedagogical role-play method in historical game that comes out of Barnard College and about 20 years old now—is developed by history professor named Mark Carnes—and in that students literally are assigned historical characters and then they play out some event from the past. Games that I have used in my classes include the Narrative Life of Frederick Douglass, Anne Hutchinson (who I mentioned earlier), the Council of Nicaea; I’ve done those in my classes, but they have them for all sorts of disciplines in all different time periods. So, that one, it’s several days. Students give speeches and form teams and do some politicking behind the scenes to come together and play their characters in order to see, like, literally playing with history.

John: In that second level of role-play, when they’re in the role of characters do you have students discuss contemporary issues or issues of the historical period?

Jill: When they are speaking as theorists or authors then I have them sort of in the secondary source mode so that they are speaking as contemporaries, even if it’s an Adorno or Habermas —who weren’t so much contemporary for us anymore—they’re still able to speak about contemporary issues. If we’re talking about Holocaust Studies, for instance, they are able to bring some of that to bear on whatever is happening here and now. I do something similar in a feminist theologies class where we read various feminist theologians over the course of the semester and one student is assigned a theologian each semester, one student per one theologian, and when we discuss that theologian the student speaks not as a student but as the theologian. So, it’s this extra meta-level that they that one student wears in this one moment or I should say in this one class period and what that lets them do is have this dexterity where they are connecting the text that they have a certain intimacy with as the “author” but then they’re also able to connect with their classmates who might be bringing up some of these different issues, like how this reading in feminist theology might connect to some of the reproductive issues that are going on now in politics or issues around concern for the planet, et cetera, et cetera. Really to your question, John, the way I see it, especially in that second level, is a hinging where they’re able to sort of pivot between the creation of the text and the application of the text and that’s one of the nice things about it because it keeps them again hinged where they’re connected to both parts and they’re aware of the fact that they’re swiveling, if that makes some sense.

John: It does. It sounds like they’re making some really deep connections.

Jill: Hopefully.

John: It’s a form in a sense of peer instruction.

Jill: Absolutely, yeah, thank you. That’s one of the things that I try to get them to do with role-play… with other activities I do in the class… but the role-play specifically… is to get students to realize that they can be instructors of their peers and just as successfully as I could be in some instances. It lets them feel that they are experts in what they are speaking on. That is ideally very empowering, but it also gives them—and I found this constantly with role-play, and this is something your audience might find interesting—is that when students are wearing a “mask” of someone else’s ideas or someone else’s character, they are much more willing to be directive with their peers and sort of challenge their peers if their peers are not thinking very critically or very clearly… and I’ve heard this from my students for the theologian activity in feminist theology… they get more annoyed if their classmates are not really stepping up and not really engaging “their ideas.” So, they’re able to say, “Well, wait a second, that’s not what I wrote. Look at the bottom of page 36…” and yet it works because no one really feels attacked by someone playing the persona of Elisabeth Schüssler Fiorenza. So, it works really nicely because that framing… that mask… however you want to think about it… creates the opportunity to step into a liminal space where it’s a little safer to push those boundaries, and students tend to do that and that allows them to do that peer instruction even more so than they would otherwise. I don’t think they think of it that way until after when I have them reflect, then they’re like, “Oh, yeah, I felt more defensive of these ideas and I also felt like I knew where they were coming from.” There was a material, historical context that gave rise to the need for me to write this theology or me to write this theory and they felt that attachment to it. To hear them say that I’m like, “Yes, that’s exactly what I would hope would happen.” As somebody who’s done acting and done theater, that’s the best part for me, immersing yourself empathetically in another experience and so it seems to work for students intellectually in a scholarly way.

Rebecca: …seems really powerful, but I can imagine that telling your students that they’re gonna role-play could be really intimidating. So, how do you prepare students for that experience?

Jill: Yeah, this is one of these things that I’m constantly trying to get better about. I tell them early and often. Whoever signs up for early on—so, again, right now I’m thinking about the feminist theology class where they have to step into and embody these ideas one at a time over the course of the semester—what I do is try to get some of my stronger students who might know me or have done this before in other classes to go first and I make sure to give them a special amount of direction and leeway and then after one or two students will go I will do a reflection like a stop, okay, what’s working. Students who are not doing this but were in the classroom discussion with the theologian, what are you noticing? And students who did this, what advice do you have for others? So, then again, the peers become the instructor. When it comes to other things, I mentioned earlier in my Holocaust class, and we do this sometimes in feminist theology, we do this in my Jesus in Film and Pop Culture class, where we really will be in a circle discussion and I mostly teach seminar. Disclaimer: most of my classes are 10 to 25 students, so this works really nicely. We’ll be in a circle and we’ll be looking at each other and channeling historians and scholars of the historical Jesus or Holocaust theory around memory studies. We might get into it and I’ll need to stop and say, “Okay, I want to pause. I noticed some of you are not speaking in the first person. Remember, I want you to be speaking as you’re scholar. Some of you are doing a really nice job with this, but I don’t hear you using quotes from the text, so remember the text is your foundation. The text is what gives you a platform. You don’t have to make up everything. You are using the text as a springboard to merge with your own ideas.” Constantly of doing that, modeling some of it for students as well and then affirming them. I think this all ultimately plays to where the majority of students get it at some point. But, Rebecca, I will tell you, I mean, you’re right. Some students never really get into this. They think it’s too strange or it’s too uncomfortable, or they’re really good students in the traditional way of doing things and they don’t think that this is something that they need or is helpful. That’s fine; not everything is going to work for everybody. What I love about some of these different liminal activities is that they will reach students who otherwise would feel that they can’t step into discussion in the traditional way because they don’t think they’re good at it, but giving them this additional costume of intellectual ideas to wear is liberating for some students, and that’s enough for me to do it once or twice a semester in some classes because it’s gonna invite in people who might not feel invited another way.

John: How long do these activities run? Is it a one day thing or multiple days?

Jill: It all depends again on which activities. When I do the symposia type models where we’re all together that’s usually at the lowest… it would be our 75 minute classes; sometimes I do it in our three hour classes, then it’s more about two and a half hours with a break in between. I’m getting ready to do one of these symposia in my Catholicism course and we’re gonna do it over two 75-minute classes, totaled about two and a half hours. What that does… and this ties to your question, John… what that does is it allows me to not be anxious that, “Oh my gosh, we have so much to cover and we’re not doing it,” and it really pushes me to the side, which is another key issue with this role-play is I as the professor in an ideal world create the settings, create the condition, give the instruction and then get the heck out of their way and let them stumble a little bit, let them struggle with some silence, let them look awkwardly at each other, let them look pleadingly at me but then turn to each other and realize, “okay, this has got to be us.” I encourage folks who want to try these sorts of things to give time because just investing in time means you’re gonna let the silence happen. Some things are much longer, so reacting to the past, for instance, which I’ve been playing with for the past year or more, that’s several weeks. We did the Anne Hutchinson game in my Religion in the U.S. class just last month and that was five 75-minute days and we’re gearing up for the Frederick Douglass game that starts next week. That’s gonna be six days. So, six days of game playing and then prep on the beginning and prep at the end. Doing that role-play meant completely redoing my syllabus for that course. There were reasons that that made sense given my teaching condition for that class, which I can get into if you’re interested, but that was a real total revamping. Everything from little bits to larger bits, depending on what you’re willing to invest in and what you’re looking to do with your students, what kinds of skills you’re trying to emphasize.

Rebecca: Jill, for someone who doesn’t have a background in theater, but…

Jill: Yeah.

Rebecca: …might find this to be a really interesting idea, what would you advise them to look at or how to start or an activity that they might do the first time out to just get their feet wet?

Jill: Reacting to the Past is a premade pedagogy. There are so many games I would recommend anyone who’s listening to this and thinking that sounds interesting to go to their website ‘cause those folks who run the Reacting Consortium will help you and there are so many games. I go to Reacting as a theater person who’s like, “Oh, won’t it be great if we all just sort of immerse ourselves in these characters of this historical moment and then give speeches as these characters,” this is like Jill in high school who did murder mystery weekends with her friend, like it’s getting these characters in and improving dialogue and a relationship and it’s just so fun, but that’s what gets me stuck on Reacting. A lot of folks who do Reacting are more gamers, they like that there are victory conditions and points for winning, or they’re historians who like this different way of doing history. That’s just my hook, but that’s not everybody’s hook. There are plenty of people I’ve met in the Reacting world who would never have thought of themselves as “a theater person.” So, I think that’s a safe one. Reacting has games as short as a day, as long as ten days. It’s good because it’s pre-made and you can go to conferences where you get to play some of the games, so that’s a good place to start. As far as some of the smaller ones, I think a safe and fun place to begin if you’re intrigued by this idea would be the dialogue assignments or the dialogue tests, sort of like I alluded to earlier, inviting students to put authors in conversation with each other… maybe across historical moments… maybe across religious traditions, in my case… maybe inserting themselves as a student into the conversation. And why is this valuable? Well, because when we want academic writing to happen, ideally students are putting different ideas “in conversation with one another”—juxtaposing different ideas—and so with these dialogue tests they were like literally doing that in a dialogue format as opposed to just writing a traditional paper where they may not be so aware that that’s what they’re doing. So… something very meta about all of this role-play stuff where they are with me—with the professor—the students are aware that they are trying on a different voice and that often for students makes something click. This is a different way of engaging. By the end of going through the process they’re like, “Oh, yeah, like I’ve made these discoveries that I didn’t think I would have permission to make otherwise.” Again, it gives them a permission to see and do something differently.

John: Could you tell us a little bit about the two reacting to the past scenarios that you’re running… in terms of what the main issues are that the students will be addressing?

Jill: The first one is the Anne Hutchinson game, which Anne Hutchinson was a Puritan woman… 1630s… Massachusetts Bay Colony… shows up… John Winthrop is the governor and they’re there to be the city on a hill to show the world this is what a true God-dedicated colony should look like; they’re gonna turn the eyes of the world to them and everything’s going to flow smoothly, and yet things start happening and people start getting religious ideas that aren’t quite in line with the orthodox and Anne Hutchinson is one of them… and she’s this woman… she’s a midwife… she starts having prayer meetings in her own home and, come to find out, that she’s having these visions and hearing these voices coming from the Bible and she believes God is speaking to her. And so doing she’s shifting the theology of the colony and people are starting to listen to her—that makes her dangerous. So this whole trial happened in the 1630s in Puritan, New England, and she is ultimately kicked out of the colony. The game scenario that we play in Reacting to the Past—again, I did not write this; I borrow it, I adapt it—much credit to the authors of the game. This game’s been running for many years. What happens is it’s this counterfactual where we imagine that there’s a second trial; she’s been banished but we’re gonna give her another try. What happens is you have the faction that is against Anne Hutchinson and then you have the faction that is Pro-Anne Hutchinson and then you have these indeterminate—this is a pretty standard format for reacting: you get one side, another side, and these indeterminates. The indeterminates are the ones you need to persuade; basically we have about three or four days of debate among these different groups trying to figure out what did she do wrong and do we want to let her back in, and the indeterminates are these immigrants who are arriving from England who want to get in the church. Before they can vote on Anne Hutchinson, they have to get into the church. So, they have their own objective of getting in but once they’re in both sides want these new immigrants to vote with them. It becomes this very fun game and what happens—this happened in my class just last month—a lot of students find themselves making arguments that they don’t agree with. They will step outside of class and say to me like, “I do not agree with what I’m saying; I can’t believe I’m arguing that Anne Hutchinson shouldn’t be speaking because she’s a woman. I believe that women have rights and should be able to have religious ideas and speak to men” and I’m like, “Yes, but you’re in a different historical context, so you need to be able to separate yourself from that.” I love—and I don’t say to students—I love that you’re trying to hold in tension what you think and feel with what you say and isn’t that how we sometimes have to act in the world? They did a really lovely job with that this semester. So, that’s the Anne Hutchinson game. The one we’re launching next week is the Frederick Douglass game which takes place in 1845. Abolitionism is really coming to its own as the political force. Slave owners are getting even more anxious and holding on to their power and the country is really in turmoil, specifically around the publication of the new autobiography by a man named Frederick Douglass. In this game there are even more historical characters—we have students who are about to be assigned the roles of John C. Calhoun and Senator Henry Clay and William Lloyd Garrison and Sojourner Truth and Angelina Grimké and they’re gonna go at it around issues of slavery, what the Constitution does and doesn’t allow about slavery and there will be indeterminates as well who can see the wisdom in both sides. They may not like slavery but is it politically advantageous to go against it at this point? This is a controversial game in some context, as you can imagine, because there will be students playing slave owners, there will be students playing former slaves, so you have to tread really carefully. That’s another thing with role-play, depending on your institutional context and who your students are, you do have to be careful as you ask students to step into roles that are not their own identities, but, again, that takes a lot of prep upfront. I think it can be done, you just have to be delicate with it. So with Frederick Douglass, for instance—I mean, who’s going to be John C. Calhoun? He was one of the biggest, baddest, most racist (by our terms) slave owners in the 19th century. Well, I’ve sold him this way to students. They’re gonna get input on the characters and I have an African American male in the class who said, “I want to play that part; that’s a part that I want,” and in fact when I did this last spring I had an African American male who wanted that part. I talked to them and I say, “are you sure? Why do you want this? And that’s awesome.” They say “I want to understand where they’re coming from, because that’s the worst thinking I can imagine and I want to know more about it.” So my student who did it last spring came to class in costume every single day as John C. Calhoun. I have pictures of the students wearing this wig and invariably he also wore some symbol of Africa on his person, like a shirt or a medallion around his neck. It was great what he was doing… how he was showing resistance to these messages that he was speaking in class—I mean, it was actually deeply profound. It also liberated other students to argue in the voices of pro-slavery advocates to have students of color in the class be willing to do that work too, and frequently at the end of classes—again, Frederick Douglass was about six days—like we would stop maybe five minutes early; I don’t know that the Reacting people would approve of us, but we’d stop and say, “Okay, this is getting heavy—how can we support one another around some of these really difficult conversations? How can we continue to support the pro-slavery students in the class…” because what would happen is the indeterminates in the Frederick Douglass game would just be like, “Oh, well slavery is bad; I know it’s bad ‘cause it’s 2018, slavery’s bad”—we just kept having to note that’s not where you are, but the water in which you swim as a mid-19th century fish is one in which slavery’s just accepted. You can’t don your 21st century hat and argue from that way. So, that’s also part of the learning objective. I think it’s probably clear that student collaboration is a big part of this… students having to work together in teams, having to come together around strategies and how to make an argument and who to target on the other team to try to turn their mind around.

Rebecca: Can you talk a little bit about the prep work before any of these role-playing instances? Clearly there needs to be some groundwork laid before you have a whole class period dedicated to any of these activities that you’re talking about.

Jill: Yeah, definitely. Letting them know in advance it’s coming is a big one. With Reacting it’s on the syllabus. I send out the syllabus about two or three weeks before the semester starts and I’m always like, “Hey, this is coming, so please look at this two-page document I’m sending you about what this entails; if this sounds great to you, wonderful; if it sounds miserable, let’s talk, because I don’t want you to be immersed in something that’s really unpleasant.” With the symposia that we do that’s a heads up in advance on the syllabus and then a constant reminder going into it—I tend to tell them within the days before to start making notes as you’re reading—read with the intent of thinking about how you would talk as this theorist. How would you channel these ideas? …and I also always for the symposia start class, ‘cause usually I can’t just have like one student as one person because they’re too many students, so I usually have two teams, like so there’s the Adorno team, whatever. What then happens is I give them about ten minutes to start, just to work together, to come, to brainstorm some ideas and maybe pull some quotes from the reading, also to prepare questions—that’s always the other thing, so it’s not just what are you going to say, but what are you going to ask of one another to keep the conversation going. Part of being in discussion is knowing how to ask questions and when to ask questions. Also encouraging them to draw connections between what different groups are saying—I never want the role-play to be an opportunity for every group to grandstand and then pass the torch to another group of grandstanders who aren’t really making connection. How does that work with role-play? That… either I’ve been modeling that all semester or I haven’t, but something I consciously do in class, I try to tell students, you know, when we’re building on each other’s comments let’s not change the subject without trying to bring in what has come before; if you’re going to change the subject, announce it and explain why you’re changing the subject. That’s part of the modeling that I try to do really consciously throughout the semester so that students get in the groove of how to have a conversation and then these other things just sort of kick it up several notches where then I’m more out of the way and they are hopefully building on tools they’ve been given and are ready to run with it.

Rebecca: You mentioned reflections earlier; can you talk about what that reflection process looks like in a little more detail?

Jill: Yeah, for the Reacting games, a whole day of debrief is built in; that’s one of the things that the Reacting people are pretty insistent on and I do not disagree with them. That tends to be a “Hey student, we just did this whole several days of this historical event. We changed history a little bit because in history this would not have happened and this would not have happened, so here’s what really happened and here’s why.” The debrief is very important for connecting back to larger class theme because, again, when you are sort of stepping outside of normal classroom behavior for a while it’s good to remind students as they gently re-enter why you’ve done what you’ve done and how it connects to these larger class themes, and that’s what the debrief is able to do. So, what you called reflection, Rebecca, I think of also as a debriefing. I also always have students write about these experiences. With Reacting they write a couple paragraphs reflection; I give them some very directed questions. After we did Anne Hutchinson, for instance, in my 101 class, I said, “What did you do well and what are you going to do better next time? Because we have Frederick Douglass coming up in a month.” …and then I share those with the students, like, “Okay, your classmates are really proud of how you all did this and here are the requests that people have for the class going forward. Many of you would like your classmates to prepare better; many of you would like your classmates to show up on time so you’re ready to give your speech; many of you would like your classmates to put more effort into writing your speeches and then delivering them with more confidence and poise.” In this way I don’t become that naggy teacher saying, “Okay, remember, we watched videos on public speaking before you delivered your speeches, but you are still not standing with much confidence and you are still reading from your paper.” Then it becomes the students doing it. Again, it goes back to John’s point before about peer instruction—the self or peer critique—and students don’t want to look foolish in front of their peers, so that sort of ups the ante there. With, for instance, the be the theologian assignment, and even the dialogue test, I always either give a journal assignment or even at the end of a dialogue test on a test say, “Okay, in three or four sentences what did you learn writing this as a dialogue that’s different from writing it as an essay? Or you’ve just performed the role in class of Mary Daly in feminist theology. What did you learn from being Mary Daly that’s different from you as Caitlyn talking about Mary Daly.” So, I think reflection is always such an important part of putting the lid on the assignment, really making it a full, complete thing so that it’s not that weird thing we did once in class but something that “Oh, like giving them the opportunity to make their own connections.” That’s why creating conditions for them to make discoveries for themselves and the reflection is sort of the last chance to do that and I don’t squander that opportunity. So, I think asking those questions and giving them space to reflect is really key.

John: There’s a lot of research that certainly supports that. Sounds like a great collection of activities. You mentioned of some concerns that students have. But, in general, how have students responded to these activities?

Jill: Yeah, a range of things. Some students, I will admit, seem confused. I’m thinking about the last time I did the be the theologian activity—I would say like the first month of class students were like, “What are we doing? Why are we doing this?” And I was trying to be patient and then I’m like, “What have I not been clear about?” And at some point it clicked and it seems to happen with role-play: at some point it clicked and it usually comes with one or two students and then like a lightbulb goes off and they get it and then everyone starts to get it. So I will say for anybody who’s thinking about these things or any creative pedagogy really from my experience: do it more than once, because the first time might not work, but that doesn’t mean that the pedagogy is not right; it might just mean that students are gonna need a little more time. Some students really thrive in it; they feel—I’ve talked about this earlier—they feel free to do college in a way that they haven’t felt free before and that’s really awesome to see because some of these are students who don’t speak. With Reacting, for instance, sometimes I’ve been in class with these students for a month or two months and suddenly we do this different thing and they just come to life and it’s really exciting. You get to see a different part of their personality. What is also exciting is how they then carry some of what they learned and some of the collaborative work that they did into future things like, “Oh, we really work together on this one game that we did, like maybe we can do our group project together at the end.” They respond really interestingly in that way. What I love is when I see then in their written work going forward how they make allusions to the role-play, even if it’s indirect. They start using some of the language and some of the teaching tools and some of the terms, it’s like they actually got it. So, it’s a real range. I’ve had some of my very best students not love it, but, yeah, I think those are the students who you pull aside and you talk to them about why because you can usually show them why you’re doing this pedagogy and why you’re doing something so different and they tend to have some really interesting ideas too, ultimately, and then they can sometimes help you reframe things. One of the things with this role-play stuff that I’ve been working on the past few years is I try to be creative but also humble. I’m not afraid—I try to not be afraid when students have critiques and suggestions ‘cause often they have some of the best ideas. They’re the ones who are doing it and so I invite them to do it; I think that goes to the reflection part that Rebecca had asked about earlier. Sometimes reflection means how would we do this differently? How can we do this better? …and sometimes that’s not just about students and their peers but also about me and the way the assignment is written.

Rebecca: How have your colleagues responded to what you’ve been doing?

Jill: Oh, good question. I’m fortunate because this year at Guilford we have a Center for Principled Problem Solving and they have faculty fellowships for a year and I was lucky to get one for the 2018-2019 school year focused on this performance and pedagogy stuff and specifically around trying to bring some of these ideas to my faculty colleagues. I should say again, I’m never an evangelist for these kinds of ideas because I think everybody should do them at all; I’m really an evangelist for teachers doing things that they think are cool and might work for their students and, while I’m not trying to force anything on anybody, but I am trying to help some of my colleagues just as they’re helping me to come up with new and creative ways to engage students and engage material and make what we do exciting to us. We’re going through a pretty significant curriculum and schedule revision at Guilford that’s gonna kick off next fall; we’ve got a lot of faculty who are rethinking their courses and course designs and activities and there’s not a small amount of anxiety about this change. So, one of the things I’m saying is, “Hey, this is a good opportunity to do some things that are more experimental and even experiential.” One of the things I did was, with the help of faculty development, brought in a Reacting to the Past Consortium board member who came and did a workshop for faculty development in September, and it was awesome. He was really engaging, gave us a lot to think about, and hopefully he’ll be back in January to do a small Reacting game. Reacting has some micro games that last an hour and a half. I believe some of Guilford faculty are going to go to a regional Reacting workshop in March. So, I’m trying to just invite people in—nobody’s being forced to do anything. I don’t have that kind of power, nor do I want it. I’m just trying to give people some ideas that have worked for me that I think are fun and that students seem to respond to and it helps our students. So, Guilford student population… we have traditional age students. we have very diverse, like ethnically… racially… in terms of class… we have a lot of diversity. We also have an adult population and then we have some high school students that take college students at Guilford in one of the best high schools in the state of North Carolina, so we have so much diversity, so how can we reach everybody? How can we invite everybody to the conversation? And this is one way that’s gotten people to sort of break down their walls. I think my colleagues are—some of them are suspicious and they should be—nobody should listen to this and be like, “Oh, this is brilliant, perfect, like, no, it’s not perfect.” Reacting to the Past is well-established, it’s not perfect. Some of my ideas aren’t perfect, but it’s a starting point and we can keep honing and keep working together to fix some of these ideas and that’s certainly what I’m doing. A lot of this work started with a fellowship I had a couple summers ago with the Wabash Center for Teaching and Learning in Religion and Theology; I got to spend the summer researching performance and pedagogy and that’s where I started to develop a lot of these ideas and some of the folks that run the teaching journal in conjunction with the Wabash Center, it’s called Teaching Theology and Religion—TTR. They’re excited about this; they think it’s worth hearing about so I’m working on an article with them—I’ve already published a few things smaller with them and I’m gonna work on some bigger pieces. There seems to be enthusiasm because I think we’re at a place where we want students to be engaged. The population of students seems to be changing in terms of their preparation for college, what they find interesting, what they’re willing to sit through in class. So, this is just one of many ways to get students trying a new way of learning. I don’t think if everybody did it that it would be awesome; I think it’s fun because they’re gonna remember in five years, “oh, yeah, in Jill Peterfeso’s class we did that really weird thing. It was weird, but it was also really cool.” I’m alright with that, I’m totally okay with that. Yes, I just say also like “I’m not afraid to be a dork about these things” and I think that’s disarming and students respond to that, ‘cause I’m like, “You guys are gonna get to role-play and I don’t get to play, but I get to watch, and do you want invite your friends?” and there at first they’re like, “no” and then by the end of the class, like we did this in the spring with Anne Hutchinson, I said, “So you want to invite like your professors or some of your friends?” and they’re like, “no, no” and then with Frederick Douglass they’re like, “We could invite everybody, like let’s put a message in the college newsletter,” like they got so into it. So, that’s learning and that self confidence and that’s not being afraid of trying new things and that transition over the course of a semester… something’s going on. I haven’t measured it and assessed it yet—don’t tell the administration—but it’s doing something and they’re learning because I read their reflections and what they come up with is pretty profound.

Rebecca: Sounds pretty incredible.

John: It does. I know in my own class I went from having students write papers to have them do a poster session and I asked if they wanted to invite other people and they were thrilled to have people from the department come in and the Dean came over and visited them and they were so much more excited and engaged about it. Small changes can make a big difference.

Jill: Yeah, I think that’s my thing whether you’re doing my level one immersion, level two, level three—those are just my categories—those small changes can mean a lot ‘cause even a little bit of reframing get students’ brains working differently and gets their hearts engage in different ways, so I totally agree with you, John.

Rebecca: I’m just sitting here contemplating how I can add role-playing into my Three Little Pigs exercise.

Jill: Aren’t you already doing it? [LAUGHTER]

Rebecca: A little bit, but I’m thinking about how maybe the students can do it more. I usually have someone come in and be the client and role-play the client role in my design class but the students are still acting as designers as humans but maybe they need to be characters in the Three Little Pigs or something for my assignment.

John: Actually, I was thinking of that—our second most popular episode has a title “The Three Little Pigs” and I can imagine all these parents playing it for their kids and finding out that it was really an exercise for a design class. [LAUGHTER]

Rebecca: Yeah.”

John: Much of what you’re describing in terms of being in this third party role is exactly the same type of thing, where students are able to see things much more clearly and are able to address issues that they’d be really cautious to approach if they were doing it in their own persona and I can see that connection and the benefits of that.

Rebecca: Somehow it’s just okay to embody that…

Jill: Yeah.

Rebecca: …other that they don’t feel connected to and explore and develop empathy and those sorts of things which is pretty powerful but I think the actual acting it out or writing the dialogues would really strengthen some of the things that I was already doing.

Jill: For sure.

John: How did you prepare to introduce this activity?

Jill: When I had my summer fellowship about performance and pedagogy I spent a lot of time doing research, starting to see who’s doing this and where and I was frustrated by the lack of what I could find in humanities classes or in sort of your more traditional classes—you get a lot of great activities coming out of theater classes or some more of the arts classes, but like high school classes or elementary school, like most of the role-play books I was finding were not geared toward college students doing the material that I wanted to do. So, I think there is room still for exploration and creativity here, that’s what I like about this. Reacting to the Past is certainly a place—Mark Carnes who designed it has written this great book called Minds on Fire, which I would recommend to anybody interested in this. There are other books about reacting that really do some pedagogical analysis of student experience in what’s going on. But, I think then within our individual discipline there’s a lot we can still do and I’m trying to think about that for myself as a religious studies scholar. I think there’s got to be stuff with empathy there and belief—I mean it’s really hard for students to try to understand beliefs of religious groups that they don’t subscribe to. This seems to be a way where they can at least intellectually be trying on beliefs of others just as they would an idea and I think that also shifts the location of some of these ideas where students are I find, “okay, I can understand that someone else may have this idea but to think somebody else may have this belief is like not about the head but the heart.” They’re more uncomfortable with that. So, trying to push those ideas of the heart as they see it up into the head, I think, could be really rich and beneficial for them. I’m sort of just riffing as I’m discovering this year but when we were talking about Puritan Theology… this Anne Hutchinson game… I just kept reminding them Puritans were intellectuals. These are highly educated people, so their beliefs weren’t just of an experience of God—it was well researched and reasoned with their relationship with scripture. So, I think there’s got to be some of that too… to think that where our emotions and our motivations come from connects heart and head; there isn’t some bifurcation of the two. I think that might help us as a society moving forward as we think about where some of our ideas and inspirations come from. I hope that what they take from some of these role-plays they’re able to put in other parts of their lives, that’s really the idea, ‘cause it’s more authentic than a classroom environment, this kind of here, I have some ideas and now I’m gonna improv conversation and ask questions and try not to step on toes… that’s life.

Rebecca: Sounds like a lot of interesting research can come out of what you just said. Sounds like you’ve got lots of plans. [LAUGHTER]

Jill: That’s what I’m thinking through right now. I guess that’s hopefully the next paper for the TTR Journal.

John: We always close with the question, what are you going to do next?

Jill: Next… So, I’m currently designing a new class. I alluded earlier to the new schedule that we’re doing, so we’re going to have three-week classes—some three-week classes, some twelve—and one of my three-week classes is going to be a new class called Religion, Voice and Performance, where I’m gonna use one or two Reacting games… we’ll see… in the service of helping students think through some of the things I was just talking about with empathy, compassion, belief, reason, rationality and relief, discovering voice, whether it’s claiming your own voice while speaking for another through Reacting and role-play or whether it’s trying to figure out who you are. I think that’s another beautiful thing about theater and acting is it invites you to figure out who you are while you are dancing around in somebody else’s shoes—that’s one of the things I’m working on now, which hopefully I’ll get to teach next year. Working on this article for Teaching Theology and Religion and I’m getting ready to keep working on these assignments that I’ve designed from the past and keep making them better. There’s always room to improve them, so those are my three things right now… and helping my faculty colleagues, as they may or may not want to try some of this stuff, so four goals.

Rebecca: It’s really exciting work, I’m glad that you were able to share it with us today.

John: Yes, thank you.

Jill: Thank you; thank you for inviting me.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

51. Engaged scholarship

Many of us live and work in communities where there is a strong town and gown divide. Building trust, engaging authentically, and developing deep understanding through intergroup dialogue takes time, patience and the right structure. In this episode, Dr. Khuram Hussain, an Associate Professor of Education and Interim Dean at Hobart College, joins us to explore a model of engaged scholarship that challenges the academy to engage in dialogue with and work alongside the community to address pressing local issues.

Show Notes

Transcript

John:Many of us live and work in communities where there is a strong town and gown divide. Building trust, engaging authentically, and developing deep understanding through intergroup dialogue takes time, patience and the right structure. In this episode, we’ll explore a model of engaged scholarship that challenges the academy to engage in dialogue with and work alongside the community to address pressing local issues.

[Music]

John:Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John:…and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Our guest today is Dr. Khuram Hussain, an Associate Professor of Education and Interim Dean at Hobart College. Welcome, Khuram.

John:Welcome.

Khuram: Thank you for having me.

John:Our teas today are:

Khuram: I’m actually drinking coffee. I hope that’s ok.

Rebecca: You and most other people. [LAUGHTER] We’ll let it go.

Khuram: I will end the day with tea.

Rebecca: Ok, perfect. I think we had a recent guest who also ended the day with tea. Today I have chai.

John:And I have pineapple ginger green tea.

Rebecca: Well, that sounds good. You always are far more adventurous than me.

Khuram: If it’s any consolation, I have a little cardamom in my coffee, which I typically put in my tea, but I really like it in coffee as well.

Rebecca: That sounds good. I should try that.

Khuram: I highly recommend it.

Rebecca: Do you have an advice about how much?

Khuram: One. One is good.

Rebecca: One is good. [LAUGHTER].

Khuram: If you want it a little stronger you can crack it and then let it sit and it’ll be even more cardamom(y). [LAUGHTER]

Rebecca: Perfect. [LAUGHTER]

John:We see you’ve done some work with engaged scholarship and service learning. Could you tell us a little bit about what is meant by engaged scholarship for those who aren’t familiar with it?

Khuram: Engaged scholarship is essentially the integration of community needs with learning and it involves addressing community needs along with whatever respective disciplines and skills a scholar may apply to a particular condition. It could be anything from developing a literacy program that is also being useful and utilized in a community, but drawing from that community in order to make sense of what questions you want to answer. So, you’re not drawing it just from a review of literature or from a body of scholarship that emerges from conferences or a community of scholars, but in fact from a variety of voices within the community itself. It’s a much more community relevant approach to even designing research before you actually do it, and it spills out into community engaged teaching as well.

Rebecca: What got you involved in engaged scholarship?

Khuram: I first had the opportunity to do engaged scholarship as a professor of education at Hobart William Smith. I was teaching a course on the civil rights movement and a colleague approached me about volunteering to serve as a professor at a maximum-security prison, and the program there was run by a Bard Prison Initiative where long term inmates were given the opportunity to enroll in an undergraduate program. And so I taught the exact same course that I was teaching on campus within the educational space that they had created for prisoners (maximum security prison) and that was my first chance to think about the ways in which the needs and realities of communities outside of campus and inform the work in learning on campus and could also inform my notions of scholarship.

John:Your work is a form of service learning in terms of the student involvement in it. How does your approach differ from the more traditional service learning approaches?

Khuram: I think that a lot of what I have seen in traditional or conventional service-learning approaches is that there’s a great focus on the ways in which our students will learn by “doing for” communities. So how can we help children learn how to read? How can we provide food to food-scarce areas? And that becomes such a central narrative and the assumptions that young people have about what service-learning is is that we’re gonna learn through service for, and what I think is unique and special about the kind of work that many folks are doing today and I hope to be a part of that (and I hope I have been a part of that) is to do service with. To move from that model means we are required to collaborate and to take a much more team-based approach to service work and the learning then moves both ways. The service then moves both ways, and that I think is the fundamental difference between what we’ve been trying to do the last few years and what we’ve often seen provided to students.

Rebecca: How does your engaged scholarship relate to the service-learning projects and things that you do with students?

Khuram: In part, the ways in which engaged scholarship works is by providing students and faculty and community members an opportunity to create knowledge out of the questions and concerns that emerge in community related work. So for instance, we started an initiative known as “Tools for Social Change” some years ago, and before we looked at any kind of service project we looked at the ways in which the community saw itself. How did long-term residents see college campus residents? How did college campus residents in the same city see long-term residents of the city? And put them into intentional dialogue, first through interpersonal relationship building and then talking about social and structural issues that have informed their understanding of themselves within the city. And within larger structures of identity, race and class particularly. After they developed that understanding we asked, “Ok, what does this community mean to you? Where do you feel empowered? Where do you feel isolated?” Based on the answers to that, we were able to map out a different kind of geography. Even though we had developed a sense of connection and collectivity as members of a community that had been dialoguing all semester, we were operating within a city that was deeply segregated and divided, and so it was from there that we looked at scholarship. We looked at research that we could pursue, and one of the first things that became really important for us to consider was the way in which the economics of the city and the capacity of some to gain access to jobs opportunity was very different than it was for others. And so we ended up taking that initial group and developing wider groups that would go out into the city and inquire… essentially do a self-study of the city about the economics and economic opportunities that were available. And so essentially it was these two stages: first of engaging in dialogue; coming to an understanding of what shared community work could be and then going out into the city with the same participants and essentially conducting appreciative inquiry and having students and faculty and community members (long-term community members) interviewing members of the community, and we were out at the Salvation Army, we were in barbershops, we were in laundromat, we were in every corner of the city and particularly in corners of the city that didn’t often have a strong voice or were not well represented, I should say, in conversations about economic development. We were able to take those, transcribe them and give them to members of the working group that are trained qualitative researchers. They synthesized that, summarized it, and we were able to present it to the city. So, here we’ve created knowledge and we’ve created it through a certain kind of process, right? You might want to call it bottom-up, but I like to see it as horizontal; it’s relational knowledge, and that, I think, is one of the most powerful things about service-learning with as well as engaged scholarship with.

John:That group that was doing the analysis of the data… Were they faculty? Were they students? Was it some mix?

Khuram: It was some mix, but here you do have kind of a hierarchy of knowledge and skill, I should say, in terms of how to do this, and so students and community members were trained by ethnographers and researchers on how to hold a tape recorder, what kinds of questions, and how to ask questions, the ethics of confidentiality, and then they went out and they conducted (after receiving a few weeks of training) these interviews in the community and it was the researchers, mostly faculty, that then booked and analyzed that data and ultimately synthesized that data, but every turn there was some part of this that was democratic and collaborative. Even the questions themselves were questions that the participants generated in concert with other community members. What is it that we want to know about ourselves? And so those were the questions that were ultimately used when we did the broader interviews.

Rebecca: Sounds like a really powerful way of breaking down the town-gown divide that happens in a lot of communities where there’s an institution of higher education.

Khuram: I think that it was transformational for all of us. I don’t think anyone could truly have appreciated what was going to happen, and I think part of it is because it was an open conversation and we sustained a certain level of openness, curiosity, and vulnerability to each other as well as what we hope would come out of it, and I mean for me it’s transformed the way I think about everything from teaching to service to even social action and the role of institutions of higher education in really engaging in communities, and so the power of it, I think, was also to reveal what’s possible that we are capable of operating on different terms and the institutions of higher education do not need to be paternalistic in their engagement with communities and they do not need to take a charity-based approach in their supportive communities; they can be collaborative, it just requires us to match strength to strength to define the things that are going to be valuable for college students and faculty and staff to learn from communities and what communities will benefit learning with their work with institutions of higher education.

John:It strikes me too that this type of project could be much more sustainable. Many service-learning projects or one-off projects where the students work and do something in the community or to the community or for the community, but when you get the community itself engaged it swould seem that that could, at least for some types of projects, set the stage for continued collaboration, either with later groups of students working with them or with the community itself. Has there been much success in continuing the efforts once the classes ended?

Khuram: I first off want to say that I absolutely agree that service-learning is conventionally structured as a one semester project-based or hour-based experience, and it’s usually focused on alleviating one particular social issue, and what we have found is that it’s necessary to do year-long initiatives and we’ve been very fortunate to see that this initiative has been able to sustain itself for over three years, but that’s required us to allow it to evolve into what it needed to and one of the biggest parts of that has been that it has been untied from any particular course. It used to just be tied to my classes and so students would do service learning project were tied to classes they were taking with me. Now, students are participating as participants in independent studies, they’re participating in different working groups that sustain themselves a little bit more autonomously, and that is also true for a lot of long-term community residents that have joined smaller working groups. There’s a working group on food insecurity, there’s a working group on political representation, there’s a working group on economic empowerment and economic opportunity, and so any one of these working groups becomes its own kind of autonomous community that intersects with long-term residents and college students and faculty and staff and that, I think, is a sign of progress and health, is when the institution of higher ed that’s tied to these projects doesn’t need to own it, control it, and manage every aspect of it. If it can become a little bit more fluid and have its own purpose outside of a predetermined purpose from the institution, it becomes more organic and more impactful often.

Rebecca: The continuity that set up in a structure like that of “community who doesn’t go away” versus students who drop in and out as they go through four years—they’re a member of the community but then they often leave—seems like it’s a really useful model for not only making the learning better but just making the impact better. Can you talk a little bit about the community’s response to these projects.

Khuram: Yes, drive-by service-learning isn’t the way to transform communities or students; it requires a real, authentic engagement, and I think when you put people in real situations you get real outcomes and that’s across the spectrum. So you’re going to get people that are going to collaborate, develop great friendships, but you’re also going to get friction and struggle and honest expressions of frustration with one another. And so that becomes a part of it too, so our students need to learn or end up learning—whether they need to or not—the ways in which their participation is both important but sometimes limited. They are going to sit and be witnesses to long-standing struggles in a community; for instance, long standing struggles between law enforcement and communities of color, and they’re going to find their own footing in those spaces; they’re going to need to make sense of how to be an ally, how to be an advocate for an inclusive community that they now belong to, so the stakes become a little bit more real. But I would be a little bit disingenuous if I was going to imply that it’s neat and tidy. I’ve received pushback at times. I remember we were holding a dialogue and I had said that we’re really starting to build some really empowering opportunities here and someone coughed and said, you’re from the colleges; you have all the power. It was a great check on my own assumptions about how I was being seen in that space… that participating in a community activity while still being associated in some ways representative of a very wealthy, multi-million dollar institution in a post-industrial Rust Belt City is not going to play out in someone else’s mind the way that it might in mine. Now what I’m proud of in that work is that someone felt that they were in a space where they could call out people’s unseen or unacknowledged privilege, and that I thought was really important for other people to see, and for me to experience, but it also means that tension in real relationships is ongoing. Honestly, we are not dealing with a utopian situation where we’re all playing on equal terms; we’re coming with different levels of capital and different levels of support within that community, so even as we do this work, my students are good to remember, as am I, we cannot be tourists in other people’s lives, that if we have certain privileges this is a place to take responsibility for some of them.

Rebecca: In a situation like this where tensions can be high, differences big sometimes, and you’re trying to dialogue, how do you set up that environment so people feel safe, like the situation that you’ve just described.

Khuram: Always sit in a circle. Always begin with some expectations. What do we need from each other to have respectful and productive and meaningful conversations? Let’s create those standards together and revisit them every time we sit in circle together. Have people that are prepared to facilitate, that have training or are getting training in facilitation; that needs to be, I think, a critical piece of that, because while it is important to hear from everyone, there is a lot of value in having someone who can reflect back some of the bigger messages and patterns that are emerging in the conversation, someone that can point to the standards that we’ve set for ourselves and what we expect as our best way of engaging, and to remind people that there are strategies that we’ve identified when things get really heated where we want to go with that. So, I think being very intentional about creating a dialogical space, and for us, the use of intergroup dialogue and a lot of the pedagogical strategies developed by the University of Michigan Intergroup Dialogue were very important and helpful resources to get started.

Rebecca: I think one of the things that I was hearing here that I want to just note, is if you’re having one of these conversations that you should have a facilitator and that the facilitator is not really participating in the conversation but rather facilitating the conversation. I think that can be challenging if we want to be involved in those conversations, but you need to make sure that you’ve picked that person and that person is staying as a third party.

Khuram: Yes, absolutely. And we typically have two people that will facilitate and that way there’s still some opportunity to give feedback or response or to slightly move out of a facilitator role, at least in terms of being able to share some ideas. But yeah, it does require you to pull back a bit. But having two facilitators… and it isn’t something that can’t be learned; I don’t think that people have to be lifelong professional facilitators. Most teachers are facilitators, and most of us have some experience facilitating or mediating conversations between others. As much as it’s important to start with people that have a background in facilitation, I think ultimately you want to end in a place where many of the participants feel comfortable and can contribute to the facilitation process over time, so we would meet every week. Ideally, we wanted to prepare people for their opportunity to do some facilitating. At this point we’ve seen dozens of participants go on to do much more formal facilitation in other spaces. That’s something that I’m very proud of and I’m very proud of them, I should say, for what they’ve accomplished.

John:You had mentioned some broad categories of tasks and working groups. What were some of the specific projects that were undertaken by people working in these projects in the community?

Khuram: All of these emerged dialogically as members of the campus community and long-term residents of the community talk through ways in which they felt connected and disconnected. We had four big ones, I’d say. We had community police relations, economic opportunity, food justice and food insecurity, and political representation. I’ll touch on each of them a little bit and then if you want to know a little bit more about any one of them I can pause. So, for food justice and insecurity, part of the challenge was an immediate one where it was about galvanizing community members to glean food and to increase access to fresh food, so we had volunteers doing gleaning. In the midst of that they were also looking at the president’s food deserts and dialoging along with community members about their access to nutrition and presenting some of those findings to the City Council and the Mayor. Or police community relations, we had two dedicated members who were part of a standing committee known as the Community Compact that met with different members of law enforcement and city government on a regular basis to talk about police-community relations and to develop programs to engage the community as well as to address certain policies. Then we have political representation, and for that what we saw was a wonderful volunteer energy of members of our entire group that went out and facilitated dialogues between political candidates and community members. Unlike conventional town halls where you’d have people sitting behind a table or behind a podium, we chat in circle with political candidates, and we had facilitators asking questions and facilitating dialogue in a pretty different kind of environment than I think a lot of us have when we engage with people that want to be elected, as well as elected officials. So we ran those, along with giving people an opportunity to register to vote. For economic empowerment, we trained facilitators to go out into the community in pairs and to hold circles in different corners of the community… in laundromats… in a variety of public spaces… to ask them what were the ways in which they were experiencing opportunity and what were the ways in which they were limited from economic opportunity. We also explored with them if they could wake up tomorrow to a different city, what would it look like? What opportunities would exist? And we took all of that and made it a final document called the “Big Talk in a Little City,” which has become an important and integral part of the city’s long-term commitment to economic empowerment, and so, not only are those voices and stories included in an official document, those voices and stories are now helping to shape policy and resource distribution in the city.

John:How have students reacted to this? Have any of them considered careers as working with communities and such things?

Khuram: For some of our graduates this has been life-changing. I think that one of the most fundamental things that we did well was simply to put people that would otherwise never have encountered each other in the same room and to ask them to share their stories and to talk about themselves. Developing those personal relationships between people that would otherwise pass each other on the street without a glance. People that had age differences, 40, 50, 60 years, people that had racial and socio-economic differences and geographic differences were suddenly having dinner at each other’s table, knew the names of family members, and knew the smallest things about one another were coming to their respective graduations and ceremonies and really becoming participants in each other’s lives. So, for a lot of our undergraduate students, having an opportunity like that is so deeply transformative because now policy is not just a matter of abstract equity and justice; it’s a matter of empathy and equity. You feel differently for someone who feels like a friend or family when they are in need and that informs your approach to policy and your approach to work in a community differently. So, we’ve had students that have gone on to do some really powerful work in law clinics, AmeriCorps and have stayed in the community to do some of that work because it was so transformational and they committed so much of their learning to this kind of engagement that they want to continue it. We do have a few folks that took a gap year between graduate school and stayed on, or decided to pursue a different kind of professional path because of the work they did.

John:That’s impressive.

Rebecca: It’s really exciting.

Khuram: I wouldn’t believe it if I hadn’t seen it.

John:Could you give us some idea of the scale of this—how many students are involved and how has it grown?

Khuram: We started with a relatively small group of about 20 students and 20 long-term community members, and in terms of active participants, it never really went much bigger than that, but it sustained itself over time and it also engaged a lot of other students and long-term community members for months at a time. What I mean by that, for instance, is a lot of our sustained participants would engage their friends, their roommates, their neighbors to come to our weekly sessions. So, we would oftentimes have topical session that were open to the public and those open sessions we could have up to 60, 70, 100, 200 people at those sessions, and so we had an active presence for quite a long time in the community when the courses were running, and now that we have the working groups there’s smaller numbers, but again, their impact, I think, in some ways is deeper because they’ve sustained some really deep work. One of the most incredible things that I saw the students do was they developed a course that would involve high school and college students learning together; so they essentially wanted to do what we were doing through these community dialogues in the high school. They wrote a course proposal, they submitted the course proposal, and after a few revisions and edits it was approved by both the college and the high school and we had a small group of about a half-dozen college students and a half-dozen high school students that took a course together at the high school. And that’s not a lot of people—but that doesn’t—what an incredible experience that they’re participating in something they helped codesign in order to address an issue that they perceive to be real across these age differences and community differences; that these teenagers and these college students together identified this town-gown divide and saw high school and college as a way to build bridges and constructed a course to do that and then participated in that course together. To me, that’s a kind of deep, transformative, impact that doesn’t quite reflect big numbers, but big experiences.

John:It’s certainly a testament to the impact that it had on those students that they were willing to do this and interested and motivated to do this.

Khuram: Absolutely.

John:How have your colleagues responded?

Khuram: I think that my colleagues have been excited, and I think that for many of them it created a new opportunity for them to engage. So, we’ve had faculty that have come in as participants, we’ve had them lead certain workshops and activities. They’ve come in with their expertise within their respective disciplines and fields. So, we’ve had a really great showing of faculty support. And part of it is we did not host this work on campus. We were very intentional about finding a place and space that was both a place that could be shared as well as a place that was easily accessible for long-term community residents, and so we found ourselves at the oldest black church in the city and a place that many of my colleagues had never been… that many people in the community had never been, and it was in the part of the city that is still segregated across a number of lines of race and class, and yet it was one of the most racially and socioeconomically diverse spaces that you could sit in in the city and here it was in a historically or at least currently segregated space… and so I think the opportunity for faculty and for staff to engage with a community that they’re really caring about in a context that seemed more inclusive was really exciting and affirmed a lot of their values. I think this is something that people really want, it’s just a matter of creating the opportunity so that they can engage in it. I don’t think that most faculty or staff want to engage in these kind of vertical relationships with communities. It’s just how we’ve been doing things for so long.

Rebecca: Seems like your background in teaching about equity and teaching about intersectionality and doing some research in the classroom about these topics set you up really well to do this work. Are there tips or other things that could provide faculty who don’t have that same background that you could share to give us a doorway in?

Khuram: I think that in some ways having a background as a scholar in any kind of social justice or equity field can be a barrier, and here’s why. That work is always in your head and it is disembodied in the institution, and the institution is, by its very nature, disembodied from the communities that it surrounds. And so you can very easily be a deft and prolific scholar of social inequity and convey and facilitate inequity in your actual life. So really it’s not a guarantee of anything. I think the measure of your capacity is in the doing, and I think it’s really about addressing questions. Who am I inviting to the table? Where is the table? Who is not here? What do I need to ask now to get who’s not here, here? Those are the more important questions, and I think if we don’t presume that there’s a certain kind of institutional privilege that comes even with being able to wax philosophical about questions of equity, then we’ve already lost the plot. We’ve got to honestly think about the spaces and places in which we’re doing our work and the kinds of privileges that we need to interrogate about ourselves before we can do any of this work in equitable and meaningful ways, and so I would say this work is for everybody, and this work is for anybody who is willing to really work with community members and to find shared purpose with community members. It’s willing to listen and learn from… and is not just interested in providing to.

Rebecca: Those are such great reminders… and empowering to make sure that we can all find a way to help and work with the communities that we live in.

Khuram: Yeah, and sometimes it does mean maybe rethinking a service-learning project that’s a semester long and seeing if you can map it out over a year. Would you spend a semester just creating relationships between students, yourself and long-term residents of a community just in that exploratory project? and then say, “Ok, out of this what have we identified collectively as a community need that we can address as a class?” …so that you get, of course, that buy-in, which is so important, but there’s a truly transformative possibility that is emerged that simply wasn’t there until you took the time to really connect and build that relationship, so I’m also in practical terms a really big proponent of year-long service-learning initiatives and moving away from the pressures of a semester-long initiative, unless you’re willing to do half a semester of really just relationship building and collective meaning-making and then cut the service piece a little shorter.

John:We usually wrap up the podcast with a question: “What are you going to do next?”

Khuram: What I would like to do next is to start preparing and supporting students to be the initiators of this work. I am currently working with a couple student groups that are creating their own curriculum and their own activities to engage people in the community with. Right now it’s a youth-to-youth, college student and high school student initiative, and the aim there is to just be a guide on the side, to really maximize whatever space and context I can help create for students to develop their own initiatives for engagement. Again, along these principles of working with, but to see our students become the guides that they need that our students can be the leaders that they’re looking for and that they can help develop leadership in their communities, and so for me right now what that involves is again having college students and high school students connect and collaborate and learn from each other with really very little use of faculty and take from us what you need and build what you must.

Rebecca: Sounds really exciting. Thank you so much for all that you shared today; I think it gives us all a lot to think about. Not just think about it; we need to take action too. [LAUGHTER]

Khuram:Thank you.

John:Thank you.
[MUSIC]

John:If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

49. Closing the performance gap

Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, Dr. Angela Bauer, the chair of the Biology Department at High Point University, joins us to discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

Show Notes

  • “Success for all Students: TOSS workshops” – Inside UW-Green Bay News (This includes a short video clip in which Dr. Bauer describes TOSS workshops)
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc.
  • Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative learning techniques: A handbook for college faculty. John Wiley & Sons.
  • Life Sciences Education
  • Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.
  • Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6), 613.
  • The Teaching Lab Podcast – Angela Bauer’s new podcast series. (Coming soon to iTunes and other podcast services)

Transcript

Coming Soon!

46. Creative risk-taking

When you teach the same classes every year, it’s easy to fall into routines. Classes, though, can be much more fun for you and your students if you are willing to take some risk by experimenting with new teaching approaches. In this episode, Dr. Wendy Watson, a a senior lecturer of political science and pre-law advisor at the University of North Texas. joins us to discuss how she has engaged her students by introducing some very creative and fun assignments in her classes.

Show Notes

  • Ishiyama, J., & Watson, W. L. (2014). Using Computer-Based Writing Software to Facilitate Writing Assignments in Large Political Science Classes. Journal of Political Science Education, 10(1), 93-101.
  • Watson, W. L., Hamner, J., Oldmixon, E. A., & King, K. (2015). 14. After the apocalypse: a simulation for Introduction to Politics classes. Handbook on Teaching and Learning in Political Science and International Relations, 157.
  • Wendy Watson (2016) Best and Worst Teaching Moments (Center for Learning Enhancement, Assessment, and Redesign, UNT video) – This contains a description of the zombie apocalypse project.
  • Center for Learning Enhancement, Assessment, and Redesign at UNT
  • Olson, Katie (2017). “Local Author Gets Cozy with Mystery Genre.” The Dentonite. October 3, 2017
  • Wendy Lyn Watson – author website

Transcript

John: When you teach the same classes every year, it’s very easy to fall into routines. Classes, though, can be much more fun for you and your students if you are willing to take some risk by experimenting with new teaching approaches. In this episode, we examine how one professor has engaged her students by introducing some very creative and fun assignments in her classes.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]
Rebecca: Today our guest is Dr. Wendy Watson, a senior lecturer in political science and pre-law advisor at the University of North Texas. Welcome, Wendy.
Wendy: Hi, thank you for having me.

John: We’re glad to have you here.
Our teas today are:
Wendy: I am drinking Paris. It’s a blend from Harney and Sons.

John: We have that next door.
Rebecca: Yeah, a tasty one. I have Irish breakfast tea today.

John: …and I have ginger peach green tea.
We invited you here to talk a little bit about some of the interesting things you’re doing with your classes. Could you tell us first a little bit about the classes that you normally teach.
Wendy: Sure. In the state of Texas there is a requirement that every student take two Introduction to American Politics courses in our department. We refer to that as the full employment plan. So, I teach both of those courses and then, other than that, I teach all of our law related courses. I’m not the only one, but I teach all of the law related courses: our legal systems course, civil rights and civil liberties, the rights of criminal defendants, constitutional law, an LSAT prep class, gay rights in the Constitution, and a seminar on the death penalty, in varying cycles.

John: You do quite a few innovative things in your classes, and one of those is having your students rewrite the Constitution after a zombie apocalypse. Could you tell us a little bit about that activity?
Wendy: Yeah, the idea is that the zombie apocalypse has occurred. This is actually for one of the flavors of Introduction to American politics, and this particular course deals with institutions: the founding of the Constitution, federalism, the legislature, the executive, the judiciary, and civil rights and civil liberties. The idea is that the zombie apocalypse has occurred. Huge portions of the population of the US have been destroyed and the remaining members of the country are required to rebuild the United States and part of that is rewriting the Constitution. Essentially, what they’re doing is building a government from the state of nature, but they don’t know that. They think they’re building a constitution after the zombie apocalypse, and that’s way more fun. It’s a guided exercise; they get worksheets every week making them think about “What is bicameralism? What are the benefits of bicameralism? What are the drawbacks of bicameralism? etc. They don’t just get to go off and write crazy things. They actually have to think about stuff and then they work in groups to create these Constitutions. One of the things that I really love about the course is that they actually do have to grapple with these issues. They sometimes get pretty heated.

John: How large are the groups?
Wendy: Usually these introductory courses are about a hundred and twenty five students and I put them in groups of about five to seven.
Rebecca: And are these things that happened outside of class, in class, online?
Wendy: No. I’ve taught the class as an online class in which case it obviously happens online, but when I’m teaching the class as a face-to-face course I actually do give them class time. Having them do it outside of the class nothing ever happened, giving them the time in class keeps them from hating me and also ensures that they actually do provide some sort of useful product at the end.
Rebecca: What assignments or exercises or things that you would normally do in class does this exercise replace?
Wendy: You know it doesn’t actually replace any exercises because if I weren’t using this activity, all of their homework would be outside of class and they’re still doing all of that. So what it’s really replacing is me lecturing and I’ve got no problem with that and I don’t think they have a problem with that. It’s more exciting or more interesting for them to be doing something, talking to each other than it is to be sitting in a seat listening to me, I think, I’m pretty sure. And I think it’s actually more educational for them to be engaging in the material as opposed to passively sitting and listening to me. Yeah. So although all they’re missing out on is me talking.
Rebecca: How did you how did you decide to go in this direction and develop this particular activity?
Wendy: I was trying to think of a way to create a simulation that would last throughout that semester, so something that kind of continued over the course of a term. And I wasn’t really sure what that would be, and I think we were watching The Walking Dead. But honestly how that all came together I couldn’t tell you, but yeah I’m really happy with that. It’s been adopted by several of my colleagues and by a professor at University of Whitewater. She used it in a summer program for high school students, and yeah, I’m really happy with that how that one turned out.
Rebecca: How did the students respond?
Wendy: You know,of course there are always students who are not going to respond at all. But I’ve never had a student who actively said that they hated it which is, I guess, good. I’ve had a lot of positive feedback and I had you know one of my best student interactions ever over this particular assignment. Again, I’m going to apologize to all of my biology friends out there. One of the features of the assignment is that the zombies fall into two categories. The type one zombies who are traditional brain-eating zombies and then the type two zombies who have developed a lesser mutated form of the virus. And so they have features of zombies, they have the shuffling gait and the slurred speech, but they have higher order cognitive thinking and they don’t eat brains, and they’re just generally safe. But if two type two zombies have a child together there is a probability or a possibility that their child will be a type 1 zombie. Again, this makes no sense at all, since it’s a virus and that just doesn’t make any sense. But, it raises this question of what do you do with type 2 zombies? Do you sterilize them? Do you kill them? What do you do with them? And they were grappling with this issue one day. And this poor student comes in, and he was, I swear to God, he was almost in tears. Because his group had decided collectively to exterminate the type 2 zombies and he said,” what do you do when you encounter people who are terrible?” And so he ended up having this long talk about how do you deal with the notion that there are Hitler’s in the world. I was like “Well, you have to remember that there are Gandhi’s in the world.” It was a long and lovely conversation about the essence of mankind and the balance of good and evil. And I kept emphasizing to him that this wasn’t real and that his friends were not evil, but anyway it was it was a great conversation and I was so touched that he took it so seriously. It’s just a testament to me of the fact that students really are interested in the material if you give them an opportunity to be interested in the material.
Rebecca: It sounds to me too like it allows them to really grapple with the really difficult conversations that are around rights and lack of rights and who gets those rights. That might be really uncomfortable if you talk about it in a in a real situation, but in this safe simulation you can have some of those challenging conversations that you might not be able to have as effectively.
Wendy: Yeah, I think that’s right. If you’re talking about things like race or sexual orientation, you’re always confronted with the fact that there are people in the room whose actual rights are implicated, and that does tend to make people sent to themselves perhaps, and that’s not necessarily what you want in real active discourse. So, when you’re talking about something that is seemingly unreal, it is unreal… they’re zombies… it’s not real. I do think that it gives people the opportunity to think through issues in a way that is safer, but also more honest.

John: The type 2 zombies add to the degree of difficulty or the level of challenge there.
Wendy: Yes, exactly.

John: You’ve also created a 500-person learning community, could you tell us a little bit about that?
Wendy: Yeah, that was nuts! My university decided to try to create a variety of different models of learning communities, sort of all at once, that alone was nuts. But I was going to be involved in a combined course learning community, so without any residential component. And I found this wonderful man in the psychology department who probably had no idea what he was getting into, and we created this community that was 500 students. His Introduction to Psychology course and my Introduction of Political Behavior course, that’s the other half of our introductory American politics duo. And our courses were back-to-back, so there were times when he could have two hours, and times when I could have two hours. And we focused on political psychology, specifically as it related to campaigns. And over the course of the semester, they each had to read three or four articles and write one page papers about them, little summaries, and then they came together and they shared their information, and they had to come up with the campaign strategy for either one or two presumed political presidential candidates. At the time we thought that was going to be Clinton and Rubio… that obviously didn’t happen. But they created these poster presentations and then we picked from among those poster presentations the 10 best, and we took those to UNT on the square which is a little gallery space in downtown Denton. And we invited faculty and university administration and we invited the Denton Record Chronicle which is our local newspaper. And the students really got into it, the ones who won showed up with their little red bow ties if they were representing Rubio and they had candy at their stations. And it was really awesome. It was great.
Rebecca: What do you think one of the biggest learning gains was for students who were in this learning community scenario where you were diving into something in depth from two different points of view?
Wendy: I think one of the things that they gained was an understanding that these two disciplines actually interacted with one another, that psychology and political science weren’t sort of siloed ideas, that they actually were related to one another. And I think one of the other things that they learned is that what they learned in class actually had implications for the real world. That things that we were learning in psychology and political science had implications for how politicians were actually running their campaigns. And that they could take the skills that they were learning at UNT and potentially apply them to a job, which is always a big thing. [LAUGHTER] Getting a job is good.
Rebecca: What level are the students in these classes?
Wendy: In those particular learning communities, most of the students were freshmen, first-year students, because they had to be advised into them, somebody had to sort of point them towards this pair of courses, so they tended to be freshmen. Otherwise these courses actually tend to draw students all the way up to their senior year, because they put them off until they have to graduate. But for these particular communities, they pretty much have an advisor say, “Hey, here’s a good idea. Take both of these courses.” They tended to be freshmen.
Rebecca: Did you find that the learning community method works particularly well with first-year students?
Wendy: I think for a lot of types of innovation it doesn’t necessarily, but I think for this, it did, because I think their desire to please was strong. And I think that they didn’t any preconceived notions of what college classes were supposed to be like, so they were maybe more receptive to the idea of doing something different. For all they knew this is what it was supposed to be like. [LAUGHTER]

John: …and getting that introduction to an interdisciplinary view of the world is probably good to do before they get too deeply into the silos of their major.
Wendy: Yeah, I agree.
Rebecca: So you’re full of brainy ideas and another one that you pulled off was an online Electoral College simulation game, can you tell us about that too?
Wendy: Yeah. So that was a lot of that was a lot of fun. I actually have to give most of the credit for the online component to our office, here it’s called CLEAR the Center for Learning Enhancement Assessment and Redesign. The assessment component has largely gone out of clear, but that’s still what we call them. They do all of our online support, learning management system, redesigned helping us create online courses, all of that sort of work. And I had a sort of a low-tech version of this course. Originally they were working in groups, I always make them work in groups, I don’t know why. But they had groups and the idea was I used the map from 270 to win, which has sort of the baseline Electoral College predictions, and which states are going red, which states are going blue, etc. And then students had campaign money and they could essentially bet their money on individual states. And if you were the Republican Party and you bet fifty dollars here, but then the Democrats get 51, then the Democrats won the state, so whoever bet more money in a state won the state. And so you could see the strategy of betting in different states of spending more campaign money more campaign resources in each state, and as you won a state, the states that were blue moved around or the states that were red moved around and you could see the total – who was winning the electoral college. And it was played in three rounds. But this was a huge pain to implement in the classroom with having to update this Excel spreadsheet every round and get people’s votes every round. It was a nightmare. So CLEAR created an online version for us that allowed students to play against each other online and it was really slick, it was beautiful, I loved it.
Rebecca: So, I’m noticing the “loved” as opposed to “I love it”.
Wendy: You notice that didn’t you.[LAUGHTER]…… Yeah, so I think another point to make here is that if you’re going to launch into one of these grand plans, you really do want to have some long-term commitment from your University. I love my university but long-term commitment is not their forte and for the learning community, for example, Adriel and I (my co-conspirator and I), we put a lot of effort into that course and we ended up offering it twice. It went really well both times but to the extent we needed money it came from a Title III grant that ended. So, we didn’t have the money anymore and then we also depended very much on help from the registrar, from advising, and from admissions to help us coordinate all of the the details. Because it was no small matter, right? It was actually very difficult. It wasn’t just us. There are all sorts of offices that had to help us out with this. And the university basically was like, “Oh, we’re done.” That was difficult and so we just lost the necessary institutional support for maintaining that program. And with the electoral college I went for like a year and a half without teaching that course, so it didn’t get used because nobody else was using it. And so CLEAR stopped supporting it on their website. It just went away and it’s just gone. So, it’s just one of those things. You kind of need to get it in writing, because there’s a tremendous amount of start-up costs associated with these programs and unless you know that that’s going to carry forth and this investment is going to pay out over an extended period of time, tt could be a little bit demoralizing.

John: In one of your other experiments in class, you did something with a mystery room. Could you tell us how that worked?
Wendy: Oh yeah, that was this last year. That was so much fun. Yeah, so the game was actually called Free Lucky. Lucky is UNT’s unofficial mascot. He’s an albino squirrel; he’s actually not lucky at all. We’ve had a series of Lucky’s on campus and the only two that I’m aware of… one got carried off by a red tail hawk and the other one got hit by a car, so they’re not lucky. [LAUGHTER] But we call him Lucky and you can get little lucky dolls. And so I got little Lucky dolls and I shoved them in little cloth pencil cases and I put combination locks on the pencil cases, so he had to get him out by undoing the lock. And I’m put my groups of students… groups again… in various study rooms in the library and they each had a little encased enshrouded Lucky in their room. And then they started the game with a question on their learning management system on Blackboard. This was for an LSAT prep course and the beautiful thing about the LSAT is that you have these questions with very specific answers. No question… here’s the answer… that’s it. The first question, if they got it right, it led them to a webpage with another question; if they got it wrong it led them to a webpage that had nothing and then it sent them back to the original page, and so forth and so on. It sent them to various pages around the web, some of them with clues, some of them with other questions, eventually it would’ve taken them off of the web and sometimes it pointed them to different clues around the room. There were various and sundry things on the table, some of them which mattered… there was a playing card… it actually was a clue, but then there were things like spools of thread that meant nothing. There was envelopes taped under the table that had a whole series of questions. And the questions there, if you answered them all, there were four of them and those gave you letters and then there was a tongue depressor on the table that helped you translate the letters into numbers and that was the code to the combination lock and that allowed you to free Lucky. And the first team that got Lucky to me… I was sitting in the lobby of the library, first team that got Lucky to me won… and they won packets of colored highlighters, which doesn’t sound exciting but they were all pre-law students and that’s like gold in the legal community… is colored highlighters. So it was exciting, they were really thrilled.

John: It sounds like fun.
Rebecca: It sounds like a lot of fun.
Wendy: It was.
Rebecca: What made you decide to do a mystery room?
Wendy: Well, you know, we have one here in Denton, and I think it looks really cool and I want to go, but I can never get people to go with me, and so I decided well I’m just gonna create my own. I wanted to do something, again, that was interesting. As much as the LSAT prep stuff was really interesting and important for my students, it’s not super engaging. We could stand up there and write logic game trees on the board, for hours on end, but that’s not exciting. That’s not even lecture exciting, that’s just really really boring. So I wanted to at least break up the class a little bit by having something that was more engaging, more active, something that was interesting.

John: And it brings in gamification too, where there’s some incentives and competition.
Wendy: Yeah. Oh yeah, the competition was big. I had one group that came down with Lucky after about a minute and a half. I was like, “You did not answer all those questions.” The guy who handed me Lucky, he’s like, “You gave this puzzle to a marine .” [LAUGHTER] I was like, “So, did you just bust the lock?” He’s like “No, I didn’t have to bust the lock. I could get him out without busting the lock.” I was like “You have to open the lock, you can’t cheat.” [LAUGHTER]. So they went back, they did it. But anyway, yeah, it was definitely a game to them. They were serious about it.
Rebecca: That’s hysterical and unexpected, right? [LAUGHTER]
Wendy: Completely.

John: A common theme of all this is that you seem to experiment with your classes and take some risks in trying new things. Could you tell us a little bit about what prompts you to do that?
Wendy: A couple of things. One, is that honestly it keeps me interested in the courses. I can get bored with the material as much as they can. In fact, they sit through it for a single semester, I sit through it for semester, after semester, after semester. And you can only talk about the appointments clause for one or two times before you’re like “Oh my god, I’m gonna dig my eyes out. This is really dull.” And that’s something I actually enjoy, right? I think the appointments clause is interesting. You still want to shake it up a little bit. And the other reason is that I really do believe that students learn better if they are engaged. As much as I love to hear myself speak, I don’t necessarily think that they love to hear me speak. I think that they get more out of my class if they are doing something. If they are seeing some connection between what we’re doing in the real world. If they can see themselves actively engaged. If they have a sense that they have power in the class. Some sense of control over their own education. I think all of those things are really valuable to them. So it’s a little more effort for me, but I think the payoffs are worth it.
Rebecca: So all of these examples that you’ve shared with us today are really different from one another: they use different technologies, different setups. What is your advice to someone who wants to take some risks and try something new, but it’s something that they’ve never done before?
Wendy: Start small. Don’t start with a 500 person learning community, which is what I did. That was dumb. It worked out, but it was dumb. Yeah, start small. Collaborate with somebody so you have somebody to lean on and share ideas. That’s maybe why the learning community worked, is that I had something called the Core Academy, so we were focused on these sorts of things together. And then I had my my co-teacher, Adriel, to work with. I think having a support system and starting small is the way to go. You don’t have to do a semester long simulation, you can devote one class to something. Use a method that lots of people are using, like team-based learning. You don’t have to do that all semester you could do it for one class. There’s nothing wrong with starting small and then getting bigger.
Rebecca: Did you start small?
Wendy: I did not [Laughter].

John: Somehow I suspected that would be the answer.
Wendy: Yeah, that’s not my style. But again, I think that if you’re worried about getting started, if you are less stupid than I am, then don’t hesitate to start small. There’s nothing wrong with that.
Rebecca: Have you had any student resistance to some of the alternative or non-traditional methods that you’ve been using in your classes?
Wendy: I get a little resistance sometimes. For the most part, they actually seem to enjoy it. Every now and then I’ll get a student who seems to think that I’m not doing my job. I mean I’ve had students who flat out on evaluation have said “I expected to come to class and hear you talk and you didn’t.” Like “Really? That was what you expected?” I mean, yeah, I assumed that is the expectation, but like, “You’re disappointed that didn’t happen?” I can’t imagine that. And of course there’s always, as I mentioned, a lot of these things involve group work, and a lot of students have resistance to group work. Even when the group work ultimately works out okay, they still are annoyed that I put them in groups. Just the anxiety associated with group work carries over to the end of the semester. Of course, some groups don’t work out. You’ve always got somebody in some group that either doesn’t pull their weight, or is responsible for a part of the project and fails to turn it in, or somebody in the group who is bossy. You always have some group that’s got a problem and I usually try to mediate that situation, but sometimes they don’t come to me until it’s too late. There are always points of contention. But they’re relatively few, and honestly I’ve always got a few complaints when I lecture too. I’d like to say I never have complaints there, but I do.
Rebecca: I read this really great article about you being a mystery novelist.
Wendy: I am.
Rebecca: Can you talk a little bit about that?
Wendy: Sure, yeah. I am a mystery writer. I started writing a long time ago, right around 2001 actually. A bad year. But I had my first novel published in 2009. I write a type of mystery called a Cozy, which is exactly what it sounds like, it’s cozy. They are light, often funny mysteries. Amateur sleuths, so no cops or private investigators. They can be in the book, but they’re not the primary character. Usually female sleuths, small town, no sex or violence on the page. I mean obviously somebody dies but it happens off the screen. They’re really quite delightful. I said PG-13, I actually included the word “bitch” in my first book, and it wasn’t even calling somebody bitch. It was like son of a bitch. I hope I’m not destroying your podcast by using that word [Laughter]. I actually got nasty emails about using that word. Really? Oh my goodness. I don’t use that word there anymore. Yeah so I started writing I’m working on number seven right now and, that’s that.

John: How do you manage that along with all your innovation in class? It seems like that’s a lot of demands on your time. How do you allocate your time?
Wendy: Not well. Yeah, I was talking about this with a colleague this morning, we were talking about this LSAT prep course (she’s teaching it this semester) about the fact that prelaw students really should be studying a lot for the LSAT. It’s a huge portion of their application. Yet, for some reason, they don’t and instead they focus so much on their GPA, which is important, but honestly, not as important as their LSAT score. They shouldn’t let their GPA slide either, let’s be clear. But in the grand scheme they should be focusing on their LSAT score. We were discussing the fact that the LSAT is way far away but their GPA is right in front of them, and so that just feels like the thing they need to tackle right now. And for me my deadlines are way far away and my courses right in front of me. So I tend to focus on my coursework and I’m not so great about meeting my deadlines, and I apologize deeply to my editor, but that’s just the way it is. I do though have a calendar, a very detailed calendar, that I keep, that has specific time set aside for every single thing that I do. Not always true to that calendar, but I do have a calendar, and it includes time set aside for writing.
Rebecca: Do you find that your writing life and your teaching life influence one another?
Wendy: Yes. Certainly my academic life has influenced my writing life. One of my books was set on college campus and I got to kill off a couple people that I didn’t like so much, which was awesome [Laughter]. Certainly, I think that my tendency toward narrative, toward storytelling, influences my use of hypotheticals in my classes. To the extent that I’m sort of telling stories. Like the zombie apocalypse, I didn’t just write a paragraph: there has been a zombie apocalypse. It’s this, probably too long story, about this has happened, and it’s all dramatic, and that’s definitely a carryover from my writing life.
Rebecca: I imagine that those details though and that spike in the climax to a story, are all the things that get students really engaged and interested and and buy into the simulation and take it seriously. As opposed to something that’s a little more surface level and that it’s a little harder to imagine.
Wendy: Yeah, and I think sometimes one of the things my cozies tend to include is humor, at least I hope it’s humor. I tend to inject that into my hypotheticals a lot and I think that that helps. One of the simulations that I do in my legal systems class is a negotiation divorce case. Each side in the negotiation has information about their client. Some of its common knowledge, that both sides have, and the wife’s attorney has knowledge that only the wife has provided and the husband’s attorney has information that only the husband has provided, and they know that that information is going to come up during the negotiation in a series of PowerPoint slides. They don’t know when that’s going to happen, but the idea is that all the sudden the wife is going to blurt something out during the negotiation. They also don’t know that there’s information that the husband and wife have not told their own attorney and that’s going to come out in the course of the negotiation. So I had great fun crafting the simulation; like the things that the husband and wife have done, and the pieces of information that come out are delicious, and the students have so much fun finding out about these details. And yeah, I think that that makes the whole simulation so much more engaging, instead of just calculating the appropriate alimony. I think it’s a lot more fun.
Rebecca: Can you share a couple of tips from your creative writing self that might help other people come up with hypotheticals or examples that they could use in their classes?
Wendy: Yeah, I think one thing that you want to do is provide detail. If you’re going to create a hypothetical, create a character to go with the hypothetical, and then provide some detail about the character and the setting and those sorts of things. It really enriches the hypothetical. It doesn’t all have to be completely relevant. In fact, sometimes it’s better if it’s not all relevant because then it forces the student to look past the things that aren’t relevant to find the things that are. I think that’s probably the key is to include at least one person in your narrative and then provide some detail. Provide a setting, provide some description of your character, provide some element of detail about what’s happening, so that it’s not sterile or clinical. Because that’s, like you said, that’s really going to draw the student in, in a way that’s sort of, A happened, B happened, C happened, or not.
Rebecca: That’s great advice [LAUGHTER].

John: We always end with the question, what are you going to do next?
Wendy: So this year I’m actually not teaching, which it is really weird for me. Last year this time, I took a position as the director of the university’s core curriculum. So, this year I’m going to be continuing with my pre-law advising but otherwise I’m focused on the university’s core curriculum. I will be engaged in assessment, which is everybody’s favorite thing, but I’m also gonna be developing a lot of programs related to our cores. So some programs related to writing across the curriculum, some programs related to bringing back, I hope, some of our learning community endeavors, and possibly exploring some other options that would allow us to really enrich our university core curriculum for our students. When I talk to students now they talk about them as the basics or the things that they have to check off, and I want them to think of those classes as something more than that. So that’s what’s next for me.
Rebecca: Sounds like the right person might be in that job to help inspire students. [LAUGHTER] I think sometimes that’s a hard sell these days, helping students recognize the value of a liberal education, and get them excited about it and help them find connections.
Wendy: Yeah, I agree. I think I have a tough road ahead of me but I’m going to do my best.
Rebecca: I look forward to hearing more about it.
Wendy: Yeah, thank you. I’d love to come back sometime.

John: We’d love to have you back.
Rebecca: Thank you so much for spending time with us this afternoon and sharing all your great initiatives in your classes, I hope it’ll inspire a lot of our listeners.
Wendy: Thank you, I really enjoyed it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

45. Opening the STEM Pipeline

Preschool through high school experiences have a direct impact on the majors and disciplines that students want to study and engage with in college. Designing these experiences to invite underrepresented groups into the discipline early can help to inspire and motivate a new generation of professionals. In this episode, Dr. Stacy Klein-Gardner joins us to discuss how engineers are attempting to diversify the field.

Show Notes

Related publications:

  • Parry, EA, PS Lottero-Perdue, SS Klein-Gardner.  Engineering Professional Societies and Pre-university Engineering Education.  In M. deVries, L. Gumaelius, and I.-B Skogh (Eds.) Pre-university Engineering Education.  Rotterdam, Netherlands: Sense Publishers. 2016.
  • Reimers, J. E., Farmer, C. L., & Klein-Gardner, S. S. (2015). An introduction to the standards for preparation and professional development for teachers of engineering. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), 5.
  • Klein-Gardner, S. S., Johnston, M. E., & Benson, L. (2012). Impact of RET teacher-developed curriculum units on classroom experiences for teachers and students. Journal of Pre-College Engineering Education Research (J-PEER), 2(2), 4.
  • Klein-Gardner, SS, ME Johnston, L Benson. Impact of the RET Teacher-Developed Curriculum on their teaching strategies and student motivation.  Journal of Pre-College Engineering Education Research. 2(2):21-35. 2012.
  • Faber, C., Hardin, E., Klein-Gardner, S., & Benson, L. (2014). Development of teachers as scientists in research experiences for teachers programs. Journal of Science Teacher Education, 25(7), 785-806.
  • Mckay, M., Klein-Gardner, S. S., Zook, K. A., Yoder, M., Moskal, B. M., Hacker, M., … & Houchens, B. C. (2011). Best Practices in K-12 and University Partnerships Panel Winners ASEE K-12 and Pre-College Engineering Division. In American Society for Engineering Education. American Society for Engineering Education.

Transcript

Rebecca: Preschool through high school experiences have a direct impact on the majors and disciplines that students want to study and engage with in college. Designing these experiences to invite underrepresented groups into the discipline early can help to inspire and motivate a new generation of professionals. In this episode, we explore how engineers are attempting to diversify the field.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Dr. Stacy Klein-Gardner, the founding director of the Center for STEM Education for Girls, and currently an Adoint Associate Professor of Biomedical Engineering at Vanderbilt University, and a Senior Professional Development Provider with Engineering is Elementary, at the Museum of Science in Boston. She recently was appointed as a Fellow of the American Society for Engineering Education.
Welcome, Stacy.

Stacy: Thank you. I’m delighted to be here.

Rebecca: Welcome. Today our teas are…

Stacy: Well, I have to confess that I don’t care for tea. So, I had some lemonade with lunch and I’m good to go now. [LAUGHTER]

John: I’m drinking Tea Forte black currant tea.

Rebecca: …and I’m having Lady Grey today.

John: We’ve invited you to join us because of your very extensive work in improving educational, P to 12 STEM and STEAM education pathways in many ways. First, though, could you talk a little bit about your own pathway to a career in engineering and engineering education?

Stacy: Sure. I’d be happy to. I grew up in the American South…actually went to junior high and high school in Oxford, Mississippi. I wasn’t always satisfied with my educational opportunities there, so I spent every summer possible at the Duke University talent identification program, or Duke TIP. Which is where I made some wonderful lifelong friends that have influenced my personal life and career since then. I did go to Duke University, where I double majored in biomedical and electrical engineering. I spent my summers working at Duke TIP, really falling in love with education and realizing my passion for that. I did a masters and a PhD in biomedical engineering from Drexel and Vanderbilt University, respectively. Then, I always thought I would retire to teach high school one day and realized that was stupid, and if that’s what I really wanted to do, I should go do it. So, in the same Fall, I defended my dissertation, I started teaching high school full time and fell in love with being in the classroom and working with teachers. Since then, I’ve been a high school teacher, full or part-time, for over 20 years now and I’ve been on the Vanderbilt University faculty since 1999…and I’ve done everything from being Associate Dean to research track professor to adjoint professor now…but really have enjoyed creating my own career in engineering education.

Rebecca: You mentioned being the Associate Dean for Outreach at Vanderbilt School of Engineering. Can you describe what your role was like? I think it’s a little unusual, perhaps, to have an outreach dean so I think it’d be interesting to hear about that.

Stacy: Yeah, the title was definitely unusual at the time. You do find more positions now, often maybe at the assistant level. But, I had a really diverse group of things I was in charge of. I worked with our Career Center on setting up appropriate opportunities for the undergraduate and graduate engineers coming out. I managed a big sponsored lecture we had every year. My favorite part was definitely doing K-12 outreach for the School of Engineering and reaching out to local communities and schools and students. Another favorite part, one that maybe surprised me a bit that I ended up really loving, was study abroad for engineers and finding ways to help engineers figure out a way to get abroad. ‘Cause the rumor used to be that engineers couldn’t study abroad, but there’s so many more types of programs that you can go to and so mine was, finding the right kind of programs and aligning those with the degree requirements of Engineers and then helping the engineers know how to plan ahead to actually travel on them.

Rebecca: So, can you talk a little bit more about your work in K-12 and also the study abroad stuff because in fields where we might not usually think about these as being good matches, like engineering, we’re always looking for new strategies to find those relationships and what have you. So, can you talk a little bit about some of the strategies or things that you developed?

Stacy: Sure. In study abroad, a lot of it was doing the logistics, but some of it was also helping engineers realize that in order to come up with good engineering solutions, you have to really understand the client for whom you’re working; the person who’s found the problem that you’re looking to solve. So you need to not just understand the straight up science, technology, engineering, and math, but you also need to understand the culture of the person, perhaps the language…What is it about their environment that makes different design constraints? So, I think, having engineers study abroad, in such an international world that we live in, is crucial now. I’ve really seen it grow in popularity which has been really fun, even though I’m not in charge of it anymore. We have a very high percentage of students at Vanderbilt who now study abroad as engineers. The second half of that question, or maybe I took him in out of order, was around K-12. You know, at the time I was doing a lot of funded work by the National Science Foundation. My favorite project was a Research Experiences for Teachers program (RET). This is a program where you bring, typically, high school faculty (although that’s broadened some since then) onto your university campus, for six weeks during the summer. Then I would place those teachers into different labs that I had picked very carefully and they would have an assigned project that they worked on full-time, for most of those six weeks…and then at the end of that time, I would work with them on designing curriculum that would be both standards-based (so they would be allowed to teach it in their classroom) as well as based in the research of their labs. So that they were bringing in real-world, current research that was going on, and often the people from that lab would come to the high school as well. Then we would publish those units through a wonderful national digital library called TeachEngineering.org. So that was definitely my favorite piece. I did some other work. I designed some high school level medical imaging curriculum units, and getting to where people have a better grasp of “what is ultrasound? or MRI? and how do those things work?” and actually motivate you to want to study high school physics or math or something like that.

Rebecca: That sounds really exciting and a great way to get people involved in fields they might not know that much about.

Stacy: It’s definitely important, especially when you’re thinking about subjects that sometimes get a bad rap for being particularly challenging. It’s good to let people see why it is they’re learning those and to put that, when am I ever going to use that, upfront so they know exactly when and how they’re going to use that.

John: Has there been any follow up in terms of following students to see how many of them did go into careers in STEM fields?

Stacy: It’s a little hard to get some of that data because I often work at the teacher level and it’s a whole other level of IRB [LAUGHTER] to get at student level data.

John: That’s true.

Stacy: You know, I think it’s somewhat depressing in that the numbers for engineering percentage-wise aren’t increasing rapidly at all, even though a lot of people are putting a lot of time and effort into it. So, not always, I mean I definitely have a lot more confident teachers in the Middle Tennessee area who are integrating what is going on in engineering into their classrooms. Of course it helps now that the next generation science standards have engineering embedded into them and just recently in my state the Tennessee state science standards do as well.

John: In 2010 to 11, you established the Metropolitan Nashville Public Schools K to 12 Engineering Pathways. What problems did this address? How has it worked and are you still working with the Nashville public schools?

Stacy: I think one of the biggest problems it was created to address is a misalignment between what different careers and companies are looking for in their high school graduates, as well as probably colleges too, with what the high schools were producing. There’s such an emphasis now on STEM, and problem solving, and computational thinking that really wasn’t being addressed by the schools and so, with Race to the Top money, Metro Nashville Public Schools set out to form this engineering pathway. I was heavily involved with it for that particular year. I did a lot of professional development. I did all of the professional development for the elementary school that was part of this K through 12 pathway, using the Engineering As Elementary curricula and integrating that. Then, at the high school level, I actually co-taught a ninth grade engineering course at this particular school. So, I was helping another teacher who was an engineer by training but didn’t have as much of pedagogy, and that sort of thing. Trying to help her build up her skills and left her rolling. That high school, Stratford High School, is still clicking along and doing really well with STEM education. It’s growing in reputation and now has a middle school that’s been integrated into the campus as well. So, I think it’s it’s been a success and will continue to be. One of the teachers with whom I worked with the most there is now the STEM director for the entire district. It’s been nice to watch her come from being one of my RET teachers to that position at Stratford, now to leading our entire district. My involvement with the district kind of waxes and wanes over the years. You know, I’ll get really involved for a while, and then I’ll be less involved for a while. I’m not working with Metro Nashville Public Schools right now although I’m always available if they call on me for anything in particular. I’ve actually just joined an advisory board for the Williamson County Public Schools which is just south of here. So, I like to keep my finger in the pie in something locally, but then I often try to work more on a national level.

Rebecca: I wanted to follow up a little bit on what you were talking about elementary education and engineering. For many of us, perhaps, when we went to elementary school, engineering wasn’t a part of that curriculum. So, for those of us that aren’t in engineering can you talk a little bit about what that even looks like?

Stacy: I’d be delighted to. If you think about what the characteristics of an engineer really are…it’s around someone who’s creative, and who thinks outside of the box, and brings in different kinds of solutions, and doesn’t have a lot of preconceived notions. If anything, that’s exactly what a preschool to elementary age child does. They haven’t sort of been beaten down by the system to think in a particular way. They still have that inherent creativity. So, the ideal time to introduce the field of engineering is at the preschool through elementary levels…so that they learn what the field is about, can identify what an engineer does, and have positive feelings towards it, and that we’re creating them to be more STEM literate citizens. There are multiple programs out there. The one with which I’m most familiar, and have even liked so much I’ve joined their staff, is with Engineering is Elementary. But, with any of them you find an authentic but sort of compacted version of the engineering design process. I might look at what a college student would use, or even a high school student might use. and we might call out 12 different phases of the engineering design process. But, in elementary school we have five fingers, so we have five steps to the engineering design process, [LAUGHTER], and in preschool we have three steps. So, just kind of compacting it a little bit…always providing an accurate view of the field. Then giving the kids age-appropriate challenges, things that might happen to an elementary aged child, and then asking them to problem-solve.

Rebecca: Can you give an example?

Stacy: Oh sure. There’s one of the EIE units that comes to mind, where the kids are out there playing a sport and their team needs to be cheered on. They find this little turtle nearby, and they win the game, and so they decide that they’re gonna keep the turtle, and they have to bring the turtle back for the next round of the playoffs. Somebody’s got to keep the turtle in a place where the turtle can not die, because that would not be good for school spirit at all. So, the whole question becomes around, what do you need to design in order to have a habitat that this turtle can live in? They draw upon the appropriate science in this particular unit…and a lot of its around membranes and creating a habitat that has enough water but not too much water. So they draw upon things they’re already learning through the science standards for elementary age children, but they’re putting them to use, and they’re working to save the turtle. Of course they do. It’s an exciting unit, it’s based on a story book that sets the stage for it so you get a lot of your reading and ELA minutes and that sort of thing in it, but then really does bring in science and math as they use the engineering design process.

Rebecca: Sounds really fun.

John: It does.

Stacy: It is a lot of fun [LAUGHTER].

Rebecca: I mean I have to admit I asked that question just because I have a toddler and I was just really curious [LAUGHTER].

Stacy: Talking about the new Wee Engineer, WEE, it’s very cute its for preschool kids.

Rebecca: Yes, yes. Yeah, I want to hear about it. [LAUGHTER]

Stacy: Oh, you really do want to hear about it?

Rebecca: No, I really do.

John: She does [LAUGHTER].

Stacy: The new Wee Engineer units that are coming out are meant for the preschool setting where the teacher introduces the problem…and it’s actually not a teacher, it’s a puppet…and so the puppet comes and introduces and says something like “I want to throw a party for my friends, and I want to make this noise maker really loud, and what do you think of my noisemaker?” …and of course it makes no noise. The puppet then says, “Can you help me?“ …and so the students go through an explore stage, where they explore the materials that are available. A lot of the work at this age focuses on helping students think about how a material is made and how that affects its function. So, they explore different materials and then they get to create their own noisemaker in small groups… and they test it… and then they do it again…and they improve (which is a big part of the engineering design process), until they all have really loud noise makers which they then share with each other and they of course give back to the puppet so the puppet can help throw a good party.

Rebecca: I like that it’s given back to the puppet so that the teachers don’t go crazy. [LAUGHTER]

Stacy: Yes, that would be a critical part of not driving the poor preschool teacher insane.

Rebecca: That sounds like a lot of fun. Maybe I need to go back and teach preschool engineering instead of web design. [LAUGHTER]

John: But, so many students get turned off early on and reaching them early can be really effective in stimulating later interest.

Stacy: They do.

Rebecca: Can you talk a little bit about what you’ve done with STEM for girls and other underrepresented groups and how to get them interested and excited about STEM?

Stacy: Absolutely. My study of the literature shows that a lot of the things that motivate girls also motivates different underrepresented groups, particularly underrepresented minorities/ethnicities, and is often just generally in line with what is good pedagogy, if people actually stopped and thought about that in engineering. I’ll focus on the girls just because that’s been my wheelhouse for the last seven or eight years now. But, a lot of the research shows that girls are interested in helping people or the environment, or something like that. If we can help frame STEM as being something in which you can help people, we will inherently pull a lot more girls into that field. So, that’s kind of one basic way. Often, if I talk to a girl, she’ll say she wants to be a doctor or nurse or something like that in the medical field, because it’s so painfully obvious of how you help people. I try to turn her into thinking about engineering and what engineering really is. Majors like biomedical engineering and environmental engineering are often popular with women, because again, it’s obvious how you’re helping people or things. But if engineers are good, and there’s actually a whole book called Changing the Conversation published by the National Academy of Engineering, on how do we praise engineering appropriately, because it is all about solving human needs and want. If we can present the field more accurately and more fully that will help. I think also when I look at a lot of these things, I like to be very explicit about things like stereotype threat, or implicit bias, or imposter syndrome, and I try to be very overt about teaching students what these things are so that they can recognize it in themselves, know what it is. There’s something about identifying it. I even still have imposter syndrome at times, where I feel like somebody’s gonna figure me out…that I’m not actually that good at engineering education, despite having just been named a fellow of a prestigious Society, I feel like still somebody might figure that out. But I know what it is, I can call it out and say you’re just having a case of imposter syndrome and, somehow, it’s easier to move aside and move along if you know that it’s a real thing and you’re not the only person who has some of these…I call them issues, I’m not sure that’s the right word.

Rebecca: I agree with you. The ability to name it out and file it away allows you to move forward. When I finally learned what some of those things were as a designer, I too, was able to overcome some of those hurdles.

Stacy: I guess the other thing I’ll add, is Carol Dweck’s work with growth mindset, has really put a name to something…about having the ability to think of your brain as a muscle that you can flex and you can grow and it can get stronger. I think letting students know that that’s a thing. Or, at the school where I worked most recently, you were not allowed to say “I’m not good at whatever it was,” you were only allowed to say “I’m not good at _____ yet.” …and I really appreciated that word “yet” there, and the implication that you can and will be good with it, but it’s going to take some hard work, and things don’t always come easily…whether you’re gifted or not doesn’t really matter, you still have to work to accomplish anything good.

John: Besides stereotype threat, implicit bias, and imposter syndrome, what are some of the things that are being done in classes now that deter women and minorities from entering engineering and other STEM fields?

Stacy: The first thing that pops in my mind there is thinking about the examples that are used in a classroom. If there are examples that are supposed to illustrate some concept, yet they are completely unfamiliar to you because the situation in which you’re growing up provides you no context for experiencing that or understanding that, you’re immediately set at a disadvantage in the classroom, and that’s not going to encourage anyone to want to continue in that field. I think there’s also some cases of just downright bias. I had a professor in college that didn’t really seem to think women should be engineers, and well I do know that that is improved, that’s not gone. There are cases of bias that are still out there. I also think a lot about parents and the role of parents, and what they believe their children, their daughters especially, can do…and what’s appropriate for them. Because there are some cultures that have a lot of bias kind of built into them and so it’s about changing the way parents think. Because if a student…if her parents don’t think she should study engineering or science or something, she’s probably not gonna go study that in college. So, we need our parents to understand what these fields are about…educate them…and then get them as a part of our moving more and more diverse people into these STEM fields.

Rebecca: One of the things that I think a lot about is the relationship between design and engineering. As a visual designer, I know that I end up with a lot of students who seem to have a fear of math, or a belief that they just can’t do math, which the process of engineering in the process of visual design is, I don’t know, almost exactly the same. So it’s always interesting to me that they err on the side of the arts thinking that they’re somehow avoiding math, but then of course they discover that there is math there too. Are there things that we can do to help overcome this…I don’t know…. it’s like almost like a preemptive strike, that like “Oh, there’s math. so I obviously can’t participate in this.”

Stacy: I hope so. I feel like we’re making some strides in that area, because you’re right, it is often math that is the big hang-up on why people don’t stay in STEM. Some of that is from having one of your parents, especially the mom, saying “Oh, Honey, I wasn’t good at math, you don’t have to be either.” …which, of course, we would never in a million years say about reading or a lot of other areas. So, I’m not sure why we say that about math sometimes. I think we’ve got to figure out how to let the math come naturally; that, if it’s a part of some problem that you are actually interested in solving, you have empathy for your client, and you’re invested in it, the teachers picked a good problem…“Oh gosh, look we’re gonna have to do this math here” and suddenly it makes sense why you’re doing the math…and you have a reason to want to do it. I think those are critical things that we need to have in our math sequences from elementary on up, so that students don’t develop this hatred or fear of it that is somehow irrational. I also think that while there is math in engineering, not every engineer does mathematics all day long. So, there is some conceptual understandings you have to obtain in order to become an engineer, but it doesn’t mean you sit around and solve differential equations all day long, necessarily. Some can, but many don’t.

Rebecca: I think that’s an important thing. I think there’s a lot of fields where we just assume that people just do math all day and it’s just a misunderstanding or misconception about the field. I think it’s also, sometimes, we present some things in such an abstract way that it doesn’t seem relevant. So, I always like to share with my students the experience that I had around geometry. When I was learning geometry in high school, I could do geometry, I could answer the questions correctly, but I never really understood what the point was and like “I’m never gonna use this.” Then I started doing more programming stuff and made visual interfaces and then all of a sudden was like: “I understand why this is relevant” [LAUGHTER] and I had that breakthrough moment where I was using all kinds of different geometry equations and things to create visual interfaces, essentially.” So, I share that with students and that sometimes helps a little bit. I could put in the math, and then all of a sudden I saw a visual, and then it just clicked and made sense.

Stacy: …made sense…it had purpose to it.

Rebecca: Yeah.

Stacy: I think a lot of math traditionally has been taught like as this separate silo, never to be used…and sometimes I think it’s because the math teachers themselves don’t know when it’s used. They don’t know the science or whatever the other field is…or psychology…and there plenty of places with statistics that use math. But, I feel like we have to lead with those things. So, when I was a high school math teacher and I wanted to teach sinusoids, I would lead with “What’s the temperature gonna be on your graduation day?” …and so we would have to develop this whole model to predict what the temperature was going to be on their high school graduation day – which was not just in a few months, most of them are juniors taking the class. So, we would have to develop this whole mathematical model which involves sinusoids and all the different parameters of one, and then on the test I would give for that unit they had to answer that question for me. It was always fun on their actual graduation day to see how close we had come.
[LAUGHTER]

Rebecca: That sounds like a lot of fun and a great way to make things seem relevant.

Stacy: mm-hmm.

John: So, one thing that would help is if math instruction and science instruction was made more meaningful by using more meaningful examples, so that the math is motivated…so people can get past the fear, because they see that there’s a purpose to it it’s difficult to do that. But, you’re doing some of that with the Nashville schools. I hope we see that more nationwide.
Could you talk a little bit about your work with Engineering is Elementary in designing curriculum?

Stacy: Sure. I have been affiliated with the Engineering is Elementary program for about a decade now. I have followed their work…their research as it was designed and presented at the American Society for Engineering Education. I was super impressed along the way that they were actually doing real education research and they weren’t just developing some curriculum and going “Oh, look how many people use our curriculum.” They were actually looking to see if learning objectives were being met and things like that. So, I’ve had a lot of respect for them. About ten years ago, I affiliated myself with them and became one of their endorsed network providers, such that I could provide their workshops whenever I wanted to around the country, I always really loved and admired how they set up their professional development…how its implemented…that the PD itself is based in research. So, when I was looking for a new educational intellectual pursuit to take on in my career, I approached them and asked if I could come work for them, and to my delight they said “Yes.” So, I started working there part time in January and have enjoyed that. There are separate teams within EIE and I serve on the professional development team right now. So, I’m enjoying working with our extended network of partners, so the people I used to be one of, I now work with and help to take the best of what we’re doing in house in Boston and get that material and the best practices for engineering education out to all these people across the country who can help spread the word and get more kids into engineering. So, I’m really enjoying that piece. I’m gonna be developing some of my first, I’ll call it online PD, but it’ll have some hands-on components to it also, for the adult learners. I think that’s a fun new pursuit for me. In house at EIE, they’ve just created (as I was mentioning earlier) this new Wee Engineer program for preschool and pre-K and there’s also a new EIE for Kindergarten, which I’m thrilled to see, because those age groups desperately need some authentic well-designed, well-researched curriculum. That’s kind of been my role right now. We’ve got some middle school projects as well, but I’m really enjoying starting young and going all the way up through eighth grade and looking at how do you do that best.

Rebecca: As a college faculty member, I’m interested in how you got involved in more of the P-12 things. How might you encourage other faculty, no matter what their discipline is, to get involved at those earlier levels?

Stacy: Great question. I think it’s kind of fun to see, once it clicks to faculty members that what goes on in P-12 definitely affects what can go on at the university level. Some of it can even be slightly self-serving in that they want more students or more diverse students to enter into the university process. So, I think that’s part of it. I think that it’s fun to help a university faculty member see how they can take their passion and enthusiasm that they have for whatever their research field is and take that down to younger kids and distill it to the basics, but increase people’s understanding of their field overall. So, I like that piece of it. There’s the other motivating factor, if you’re gonna apply for like a National Science Foundation grant you have to have an education and outreach component. So, that there’s that external motivator as well to think about how could I be involved in this process. I also worry a lot about we have these new next generation science standards. which I think are quite good but they have a lot of engineering in them. So, who exactly do we think is going to help the K-12 teachers know how to do that, and do it authentically, if we ourselves are not out there helping them and teaching them.

Rebecca: What would you encourage a faculty to do as their first step to get involved in P-12
STACY. I’m trying to think of one single first step…probably, reach out to your kids’ local school and listen…ask the teachers and the administrators…but especially the teachers…ask them what they need. Because the teacher will know. He or she may not know how to go about getting it, but they will know what they need. Don’t go in and be like “I know everything” when in fact you really don’t know everything about what it’s like to be a K-12 teacher…but go in and ask how to be helpful. Listen to what they say and honor the fact that they have to be standards-based.

Rebecca: Sounds like really good advice, but at times, it’s just that little encouragement of what that step could be is helpful. So, thanks for that nugget.

Stacy: Yeah. I mean, go ask. People really want you to.

John: …and in many disciplines, I think, there are organizations that work with elementary and secondary schools. In economics, there’s the centers for economic education spread throughout the country, who do work with middle schools and high schools in providing some educational resources. I don’t know how common that is in other disciplines. Is there anything like that in art?

Rebecca: There is something more general for art, but not for design. Design stuff is kind of under the umbrella of art which, depending on what your position is, you might not think that they’re entirely related. They are, but they aren’t, you know. It’s kind of complicated. {LAUGHTER]

John: You’re working on a new advanced course in engineering for high school students. Could you talk a little bit about that?

Stacy: Yeah, this is a project that has been near and dear to my heart, and some of my wonderful colleagues, particularly from the University of Maryland (Dr. Leigh Abts, in particular). We have been trying to get an AP engineering course started for 14 years now, and when we started, really, the colleges weren’t ready for it. The schools of engineering were not interested in it. They weren’t interested in accepting credit for it. They didn’t really see the value of it. Thankfully, that has changed over time, which I’m really excited about. The College Board got interested in AP engineering and created a framework for the course. They had to put the brakes on that for a little while while they focused on the redesign of the SAT, but now they’re back to being interested in it. So, my team approached the National Science Foundation who said “Yes, we’re on board with this. This needs to happen.” So, we’re modeling our upcoming work off the very successful work of creating the new AP computer science principles course which has been a highly successful course for AP…and it’s also really successful in that a lot of women and underrepresented students are taking the course and taking the test, so that’s super exciting. So, we’re trying to draw some best practices from that and, knock on wood, I hope that the NSF will approve our final proposal to begin the work later this fall. The basic idea around it is we’re going to finish the framework…make sure it’s right…and we’re going to be developing a sample curricula and sample professional development for that. As you may know, with AP courses there isn’t any one set curricula that you’re expected to follow. You can do it however you see fit. You just have to make sure it fits the framework. So, we’ll be developing some sample ones and then we’ll be partnering with high schools. We’re hoping for about 70 high schools all over the country and a lot of diverse settings to train their teachers and have them work with our students and do engineering design at that level. We’re looking at having ultimately an assessment that is a bit like AP art studio actually in that we’re hoping for a portfolio process where you would submit engineering design work that you’ve done over the course of the semester or the year, and then that work would later be evaluated for possible engineering credit.

Rebecca: That sounds like an exciting advancement.

Stacy: I hope so. we’ve had over 110 Deans so far say that they’ll be interested in giving some sort of credit for it. I think it’ll be interesting to see how the universities handle it. I think some might give credit for their actual Intro to Engineering course while others might give it as more general science and engineering credit. I think that the universities now see it as a great way to get more STEM literacy in our population and I think they’ve started to see it as a great way to get more diverse students into their programs, because they will have done engineering at a younger age and done it in the more friendly confines of their local high school.

Rebecca: …and perhaps introduced populations who aren’t really familiar with the field at all to what the field is rather than expecting college students to just magically know what all of our disciplines are.

Stacy: Right, that’s true of a lot of disciplines, so it’s not just engineering.

Rebecca: Yeah, exactly.

Stacy: There aren’t tons of schools that have economics in them but you’re probably not gonna major in it if you don’t know what it is.

John: Actually, I think most schools do now have economics. It’s part of the New York State curriculum and I think most states do have at least one semester economics course. But, it took many many years before that was widespread.

Stacy: We’re trying to catch up with you, John.

John: It’s not always taught by people who know much about economics, unfortunately…. [LAUGHTER] …which is one of the reasons why the Centers do so much work.

Stacy: I would mention that if someone is interested in engineering education, I know that there are now actual programs in engineering education. You can get graduate degrees in it, and I would also steer them towards the American Society for Engineering Education. It’s a wonderful Society. It is the place to go to for what the best pedagogy is in engineering education, and to find your people there. They have lots of divisions, some are specific to your field of engineering and then we have a wonderful pre-college division there as well. So, it can be a great resource if anybody who’s listening wants to jump in jump in and join it.

Rebecca: So, we generally wrap up our podcast by asking what’s next?

Stacy: What’s next? Well, I feel like I’ve hit some of the “what’s next” because I’m in this great transition period in my career, which I’m excited about. So, I’m hoping that what’s next is that personally I’m able to view engineering education from preschool all the way through 12th grade and then into college and look for “How does that pathway work?” Are there things that are missing? Are there things we should be doing differently? So, I’m excited about taking that long view across engineering education and I’m always looking for new collaborators and people who are as excited about the field as I am.

Rebecca: Thank you so much for joining us. I hope that you’ve motivated a lot of others to think about their own disciplines plans from the elementary level all the way up through the university level. Sometimes, that longitudinal perspective can really help us have better perspective on what we’re teaching in higher ed.

Stacy: Definitely. Just to think about like “What matters? “What’s actually important?” It’s easy to get caught up in the minutiae of these little things you have to be sure you’ve mentioned to your students. Not really. Focus in on the big thing.

John: …and if you really want to do something about the gender imbalance in STEM fields you do have to reach out earlier because by the time you get to high school, people have already been sorted out. So, it’s really important to do that sort of work early.

Stacy: Very true. Most of the girls are getting sorted out late elementary to middle school, at the latest. So, you’re absolutely right there.

John: Well, thank you.

Stacy: Well, thanks for having me. This was fun. I appreciate you reaching out to me John, I was flattered.

John: Thank you, Stacy. We very much appreciate you joining us today.
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

44. Industry realistic experiences

Student motivation is enhanced when students see that the work they are doing is relevant to their future careers. In this episode, Dr. Bastian Tenbergen, an assistant professor of Computer Science at the State University of New York at Oswego, joins us to discuss how industry realistic projects may be used to enhance learning in software engineering classes.

Show Notes

  • Daun, M., Salmon, A., Tenbergen, B., Weyer, T., & Pohl, K. (2014, April). Industrial case studies in graduate requirements engineering courses: The impact on student motivation. In Software Engineering Education and Training (CSEE&T), 2014 IEEE 27th Conference on (pp. 3-12). IEEE.
  • Daun, M., Salmon, A., Weyer, T., Pohl, K., & Tenbergen, B. (2016, April). Project-based learning with examples from industry in university courses: an experience report from an undergraduate requirements engineering course. In Software Engineering Education and Training (CSEET), 2016 IEEE 29th International Conference on (pp. 184-193). IEEE.
  • Dijkstra, E. W. (1959). “A Note on Two Problems in Connection with Graphs.” Numerische Math. 1, 269-271.

Transcript

John: Student motivation is enhanced when students see that the work they’re doing in their classes is relevant to their future careers. In this episode we examine how industry realistic projects may be used to enhance learning in software engineering classes.
Thanks for joining us for “Tea for Teaching,” an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego [Music]

John: Today our guest is Dr. Bastian Tenbergen, an assistant professor of computer science at the State University of New York at Oswego. Welcome!

Bastian: Thank you, thanks for having me.

Rebecca: Today our teas are…

Bastian: Well, upon John’s recommendation, I’m having the mint herbal mix tea, which is excellent! I’m a peppermint tea drinker, so this is blowing my mind right now.

Rebecca: Excellent!

John: I’m having ginger tea.

Rebecca: I’m having Prince of Wales today.

Bastian: I like the ginger tea, that is my favorite tea.

John: It’s good.

Bastian: Ginger and fennel and peppermint, those are my three.

John: We invited you here to talk a bit about the projects that you have students do in your computer science classes. What classes do you generally teach?

Bastian: I’m teaching in the computer science department, but I’m mostly teaching software engineering courses. We actually have two separate majors: we have computer science majors (Bachelor of Arts and Bachelor of Science) and we also have a software engineering Bachelor of Science program. People usually confuse software engineering and computer science or at the very least don’t really know what the differentiation is. In contrast to computer science where it’s really all about programming and all about finding optimal algorithms to solve problem x for person Y, software engineering is concerned with the process of development from A to Z. So from requirements all the way to programming which is a small part of it, all the way to Quality Assurance and also budgeting. Also, the business aspect of it, so it has a wider focus.

Rebecca: It’s a little more client facing?

Bastian: Very much client facing, yes. By trade I’m a requirements engineer you can say and a very smart person who very recently submitted his PhD dissertation (which I’m very proud of him that he did finally did that). He wants to find requirements engineering as a socio technical process that implements the vision of a system given the time and budget constraints that you have. They usually also call us the context of the system, the developmental context of a system. It’s the budget, the time, the resources you have and such things. Those are considerations during software engineering.

John: In what classes do you have students engage in projects?

Bastian: Well it is very hard to teach computer science without actually using projects. You can teach the skills but at some point the art of making software becomes more than the alignment of skills in a particular way. Legitimately almost all classes we teach have a very heavy focus on projects. I’m teaching a software and safety requirements engineering course which is project-based, at least a quarter to half the students grades depends on the project. I’m also teaching a software quality assurance class where at least a quarter, sometimes half of the grade depends on project performance. I’m also teaching occasionally capstone courses, where the capstone experience in the software engineering program really tries to simulate how an independent developer develops a spoke software for one individual client and one of my favorite things to teach is a class called “Software Design”. The term design implies software architecture but it’s not just that. For those software engineers out there listening, this particular course is called that for historic reasons, but it’s really a design process class. The entire class collaborates together on producing one substantial piece of software, which is usually on the first day of class. I demand like big evil Papa Smurf that this project could be marketable, so the explicit goal is we want to market it, which sometimes works and sometimes doesn’t, but that’s the goal. Then we differentiate the students into teams and have a database team, a GUI team, we have graduate students at our university that specifically focus on usability and human factors so we have those as a team, we have requirements teams, we have Quality Assurance teams. They have to learn not only how to work together, they also have to learn how to apply their skills, have to learn how to best make design decisions, how to communicate them and not only how to communicate them with like-minded peers that are also scientifically or engineering capable but also with a stakeholder. Software engineering in general is very focused on the people who are giving the money for a project. In my classes I really focus on the fact that students should be able to argue their rationales, not to other engineers and not to other technicians, but to their grandmother because if you can explain it to your grandma, you can explain it to the person who gives you money in the project; and that usually worked well.

John: How early in the term do students decide on the project?

Bastian: So, It depends. It depends on the course. In my requirements engineering and software quality assurance class where we also teach skills, we also teach requirements, solicitation, or you teach let’s say data flow based testing, which is a new technique for them to pick up. There, I usually pick the projects for them or if they have a particular good idea we’ll discuss it, but usually it’s in the first week or so that they finalize the project. In capstone classes and in the software design process class, I usually conceive the project ideas and then we make the necessary choices, let’s say the necessary preliminary choices in the first week. What I mean by necessary and preliminary choices it’s this; I basically say “I want a universal all-transfunctionater” and no one has any clue what that is and I say “great it’s your job to ask the stakeholder, who is also me, what I mean by that.” Then the requirements team would differentiate the people into teams and the people who self-select into requirements they say: “Ok, well Bastian, what did you mean by that?” …and I say “Well, I meant… really… whatever… a cow milking device.” So the project kind of takes shape. So, I force them to come up with the requirements and to get them out of me, so that, as an instructor I basically have a dual role… or actually triple role, sometimes quadruple role and I’m project manager for them. I’m also the stakeholder, I’m also the person who gives them advice and the instructor that says “dude you shouldn’t do this because X & Y & Z or whatever. Or, maybe here’s a great idea that someone else just had and maybe try this.” More often than not I’m also the conflict solver and a psychologist that lets them cry on the shoulder because at some point during the semester everyone is just frustrated. This is part of the experience I guess but that’s why I usually tell my students the trick is to be successful despite other humans and once that idea clicks, working together never becomes a problem ever again. So as you lose one conflict earlier in the semester and then it kind of dissolves and this is when you see the students go from students to professionals. It’s my favorite class to teach because you can see how the students go from “professor, how do you want this” to “well Bastian I know you said you want a cow milking device but see we don’t have any cows, so how about we build you this instead”. It’s important in these kinds of projects for them to be able to communicate what the stakeholder wanted versus what we can conceivably give to the stakeholder given the time and the budget and the people that we have on staff.

Rebecca: Or what this stakeholder may actually need and doesn’t realize that they need.

Bastian: That’s right! Two years ago, I co-taught to this class for the first time which was great because then we could literally play good-cop and bad-cop. One stakeholder and one instructor will always be against the ideas, which believe it or not wasn’t necessarily me, and the other one was always in favor and would always say “oh yes that’s fine, that’s fine, Keep going”. But you know even if you have someone who constantly approves of what you do you don’t know whether or not you’re actually making any good progress. So it may feel good to have your ego stroked and be told that yes everything is great but that doesn’t necessarily mean that you’re making useful progress. Really in the end the only way you learn is if you make mistakes. On the other hand of course being told everything is bad or everything is completely horrible and how dare you even propose this doesn’t help either. So the truth is somewhere in the middle and it’s for the students to find out what goes. That’s the tricky part about teaching this kind of class, is to guide the knowledge discovery process such that they find it but they can still be successful despite having to do all the work themselves really.

Rebecca: So you’re describing mostly the setup for your software design class right? Which is a big team right that has small teams on it, but you’re all working on the same project.

Bastian: Yes.

Rebecca: Are your other projects and your other classes also set up so that everyone’s working on the same project or individuals working on a project? How are the setup similar are different?

Bastian: You have teams of students I have a very much focused on that that students would at least together with one other person. And the reason is, four eyes see more than two eyes, that’s why. Plus I encourage them like, hey, you know if you talk to another person, if you vocalize your problems, it helps, it stimulates your thinking. So that’s why I do this for example my requirements class, I give the general theme of the project and then let the students do some of it on their own. For example, a little while ago when I taught this software and safety the requirements class first here in the US, I gave the students the opportunity to I said, “okay, we have these cyber physical Rovers or robots, never mind what cyber physical systems are but it’s a buzzword and they can do certain things something makes them special”. We discussed this in class and I said, “we have these robots, and I want you to do something cool with them.” “They each have individual functionalities, pick one for different sensors, different robots had different sensors, pick one and do something fun with it”. And they pitched the project ideas. For example, one of them said, “I want my robot to exit a maze.” Great idea do it. Another person said, “I want my robot to use the camera and use computer vision to recognize another robot and drive after him”. And it was a cool project. Another team of thing was three students actually said, “no we don’t like the robots we’d much rather do something else and here’s an idea”, and I said “okay”. Soon as the learning objectives that I have to find in my syllabus are roughly aligned, I’ll let them go. My general philosophy is if the student has a better idea than me and can argue it, ok. Because I want to learn something too, right? (laughter). So I let them do it and let him explore it if they have the idea right.

John: The students would have more ownership till when they come up with the idea.

Bastian: True. Usually I’m not sure if it’s me over the project or it’s just those cute little robots that we have, but usually students are quite enthusiastic about projects. For the coming semester believe it or not we bought programmable slot cars. Remember those slot cars that you used to race on the like little tracks, you a little controller in your hand you can push more and less gas and throttle. We bought programmable ones and we’re gonna be using that in a project. I’m super excited about this and can’t wait to play with that. I’m hoping students will be excited about this too. And if they’re not then fine they’re not expensive.(Laughter). Plus we have several other faculty in our department who are quite excited about these. I’m not going to tell you the name of the manufacturer but they have a very cool API, which is an application programming interface, which is really simple and open. I haven’t tried them out yet, so I’m hoping it that’s a needle platform to automotive software engineering projects which would be cool.

Rebecca: So, as your students are working in teams and you’re trying to make sure that they’re prepared for professional life, right? You’ve talked a little bit about thinking about clients and things like that. How do you make sure that the problem that they’re solving is realistic and it’s not pared down so much that it’s unrealistic? Sometimes when students self define a project, it’s in a context that wouldn’t generally exist when they are working on their own unless they’re at a startup.

Bastian: That is so true. I would argue that finding the project not necessarily the scope, but the project domain is probably one of the two hardest things about doing the project. In fact, I’m not sure if I’m allowed to say this and make some advertisement on my own behalf here, but colleagues of mine and I wrote two academic papers and we’ve just submitted the third one on project-based learning in industry-realistic case examples in software engineering to a fairly substantial fairly high ranking conference. The industry realistic examples, they usually reflect one or two aspects that you would commonly find in let’s say industrial development projects. For example, the problem of, let’s say sensor integration. If you have a little robot and you tell the robot to rotate 90 degrees, you can know whether or not that thing actually turned 90 degrees because the one motor if you have two wheels, assuming you have a two-wheeled robot one motor might be have different manufacturing tolerances and maybe a little bit stronger than the other one, so you may be turned 89 degrees, maybe you turn 94 degrees. So how do you fix that? Well you could put a little sensor on it that does that, but the only rotational sensors you have they are going to be inaccurate too. Especially if you have let’s say have the robot run on carpet rather than tile. All of a sudden the physical setting and that the robot is in has a great impact on the software that you’re developing, and that is an industry realistic problem. Let’s say you fly an autonomous aerial vehicle somewhere and try to detect wildfires, which we are currently experiencing a very hot summer with a lot of drought. So they do this, they use drones to detect wildfires. How do you know you’re actually currently flying through smoke as opposed to through humidity or through fog or through a regular cloud? You have to use sensors. It’s a realistic problem. So the domain flying an actual drone is hard, so we use a little robot which however has the same kind of problems. I was very fortunate that earlier in life, I was working with some industrial companies in research projects and so it’s relatively easy for me to figure out what could be a challenge that the software developer or software engineers is going to be facing. So in those two papers that are just described, we focused on how to apply industry realistic case examples and we figured out what kind of properties these have. For example, you want to be sure that the project that you give to your students doesn’t have a bunch of challenges, but just one is usually enough, just to focus on one little challenge. For example, get the little robot to rotate accurately, but you don’t tell them make a project that lets the robot rotate, because that’s boring. Instead you say, “hey, why don’t you write an overtaking algorithm for robots?” And usually you know full well that in order to make those robots actually overtake one another like cars on a highway, a lot of things have to fall into place. First for example, you have to figure out how to make this robot drive straight and that is already a project in an art of itself. So the other important criterion for these industry realistic projects is to have the project scalable. So toward the end of the semester I usually joke with my students and say, “well, if you can’t finish your project in time, it’s either because you didn’t scope the requirements right, or because you bit off more you can chew, development is harder than you initially thought, or maybe because we haven’t redefined success yet.” So if you can’t be successful redefine success. Which when I say that really what I mean is I tell them, listen, you can’t deliver what you wanted to deliver, fine, not a problem happens all the time in reality, instead tell me what we can expect. Given the time that’s left what can we expect. “Well, we can actually make the robot overtake”, they will say, “but we can make it drive straight with a certain level of accuracy.” That seems boring and uninteresting when I say it like this but it’s actually a remarkable feat. At the end of the semester, two kinds of students those that are happy to be done because this was horrible experience, the minority thankfully, or you have the people that say, “oh my god, had no idea how hard it actually is to interface hardware and software.”

Rebecca: Really a big lesson in scoping, it’s like how do you break a big project into small pieces.

Bastian: Absolutely.

Rebecca: Understand that small pieces have to be completed before you can put them together to make a big piece. It’s like modular design.

Bastian: Yeah, absolutely. Modular design is one of those keywords buzzwords almost from the 90s, but they were right. You divide and conquer is a recurring theme in computer science that works everywhere. If you want to sort numbers you divide and you conquer it’s the fastest way to do it and if you want to develop a software project you divide and you conquer. Your first build project one and project two. You can scope this whatever way you want. Very often actually I have students who halfway through the project realize the potential that the project has that they’re working on and say, “hey Bastion, I really would like to bring this project into this direction instead I know you said overtake algorithm, but let’s do a path finding algorithm instead.” Esker Dijkstra in the 1960s wrote basically the silver bullet of shortest path algorithms and, can I implement that and put it in the robot? And why not? Just last semester I had someone interested in that doing it. The third characteristic about these projects is don’t be a stickler too much for what the industry really experiences and let the student figure it out on their own. And the one hand you could simulate what companies develop software to particular degree. So you could say, oh we are all now going to fill out application slips or vacation slips or things like this right, but that this misleading from the art of developing software. On the other hand when you tell the student hey listen or when the student asks, “hey listen, I want to bring this in another direction because I find this really interesting,” usually what comes out is something really rewarding, In my experience at least. So the third concept is don’t overdo it students will by themselves, with enough enthusiasm, drive it into a direction that is going to blow your mind, theirs and yours.

Rebecca: So when students are working together in teams and they’re taking on kind of different roles. How do you help the students divide those rules but then also make sure that they’re learning all of the skills or techniques that you want them to learn.

Bastian: That’s hard it’s really really hard and I would say that there’s no silver bullet of how to do this. It is an unfortunate truth that the larger the project is the more people are working on the same project, the higher the chance that at least one person is simply left out and you can be the kind of person that says okay, let’s try to live this person up to make sure that they learn something, but to be entirely honest, in part, in my opinion it’s a component of the experience to make yourself available to your team. So what I do throughout the semester is encourage students to contribute any way they can and students miss understand sometimes from a grading perspective that contributing means being the natural-born leader. In my experience, every team no matter what has one or maybe two people who are really great at the technology and also really great with people and their form naturally adopt the role of the leader. Assigning a leader doesn’t really work all that often. You can say okay you’re a graduate student so you’ll have more management responsibilities and that usually works. But often there’s one non graduate student who’s also fulfilling this managerial role so part of the experience is to find any way you can possibly be helpful for your team this doesn’t necessarily have to be the leader role. You cannot be a leader and be a rather shy quiet person and still get an A in project based courses, the way I teach him. Simply because what does an A mean? An A means here an excellent outstanding student and when are you excellent outstanding student? Well, in these cases when you’re an excellent outstanding team member for your team. When are you doing that? Well, when you contribute stuff any way you can to your team such that your team can continue. I’ll give you an example, if you are the kind of person that never volunteers presentations in class, that never contacts me as the instructor with questions, that never has an management important role in the team but manages all the background communication, implements all the code, and does all the right things in the simply couldn’t contribute couldn’t do what they’re supposed to do if it wasn’t for your input; you’re an A student, regardless of whether or not you’re very outspoken and outgoing or not. On the other hand, if you are a student who talks a lot and who is volunteering a lot, and who is putting themselves in the limelight a lot, but at the end of the day your team can’t count on you because you didn’t show up for the team meeting or because you promised something but never delivered or because the stuff that you deliver is of poor quality and your team decides to drop it and not use your work. Then you’re clearly on the other end of the grading spectrum. So I have a rubric, a rubric system where I say oh can a student clearly is the backbone of the team any way possible a B student is delivering useful stuff in regular intervals and C student is well useful when being assigned work, right, and a D student is unfortunately not useful even when prompted and an e student is the kind of student where the team said listen we’ve asked you 15 times you haven’t done a darn thing we’re done with you.

John: We should know that as we go for some reason we use E’s instead of F’s.

Bastian: Oh that’s right. I’m sorry.

Rebecca: Its alphabetical.

John: It doesn’t make sense to any of us but it’s been done here for a long time.

Bastian: It’s true. So a student that is failing the course with an E or other universities with an F usually those students know that they are. Usually before they are even assigned a failing grade I’ve had numerous conversations with them not as the manager, not asked stakeholder, but as the Papa Smurf (laughter) who says listen, if you want to pass this class, and for software engineering students in our university this class is a core requirement, so they have to have a passing grade in this class to graduate. I say listen, right now you’re not. We’re also doing peer evaluations so some people could say well if you were the one that subjectively evaluates the students isn’t that unfair and the answer is yes, of course. So I’ve experimented with this, just evaluations by me, and I had some good experiences with it and also some very bad ones, unfortunately. So within disputes, and it happens occasionally. What I like to do is peer evaluations where students within the same team evaluate other team members on a scale of say 25 points and usually, and remarkably, these peer evaluations match my subjective opinion almost all the time, 100 %. Students when they evaluate others are usually little positively biased and they are reluctant to evaluate people really badly, but if you ignore that, the subjective evaluation students have of each other are matching my observations very well.

John: How often do they get feedback in terms of how well they’re doing?

Bastian: Every day, every day. We meet usually in this class, we are meeting three times a week or the university has allotted three meeting times a week. I like to schedule two meetings where I’m there and they are reporting to me in daily scrums, those of you who are software engineers,yes we’re doing AGILE methodology specifically scrum. We do daily scrum so it’s basically, you stand up when and you say this is what we have done from last time until today, this is what we’re currently working on, this is what we’ll do next, these are the roadblocks, these are possible problems, and these are questions that we have. Five minutes, everyone does it and usually takes the entire class period to figure out problems, to resolve roadblocks, and most of the time it’s minor things but gotta get done because it’s the planning for the rest. So, during that is when I provide feedback by saying hey have you done this yet or have you thought about that yet, or John Doe here, was supposed to deliver this and that, did they? On the other hand, I’ve very often we have experiences that students say well, see our friend Jane Doe here foresaw two weeks ago that this is going to be a problem, so she already did this and that in anticipation. That’s how you know you have a really great student at hand, right, when they can anticipate problems in the future but would usually only experienced engineers are able to do. So they get feedback every time. What I do however, is the third class meeting that we have, I usually reserve for project work. Because that is the one day in the academic schedule for all students in the class, and if you have 30 people in the class, that I know they have time. Especially at the beginning of the semester I often hear things like, oh we don’t have class on Friday. I’m like, no, no, no, no no, you have class. I might not be there and the reality is that of course I’m there, I’m just then the next room letting them duke it out, and when the shouting or the crying gets too loud, I walk in. Or they decide on things and they have a question and needed it answered right then right there, so they walk over to the other room, or wherever, I am and they ask me. Or I just sit quietly in the room and let the students plan the work on their own. So, the idea is that the third meeting of the week is usually when they get to make progress when they need other people to be present. We also usually coordinate using online chat functions, we’ve used Discord.

John: This is used in a lot of gaming.

Bastian: Yes yes I use them gaming a lot right? Plus all my students they’re all familiar with it because they’re usually all gamers. And we even have a little Steam community going because, you’re nerds like that. So they coordinate through Discord and sometimes they say, hey Bastian is a fine if we don’t meet in person because John and Jane are out of town because, whatever, wedding or sick or whatever, is it okay if we do this online? I say sure, I don’t care how you get it done, just get it done. That’s all I care about. I care about you make progress any way you can. Next semester I’m actually preparing for having this class for the first time in a sort of hybrid fashion. Hybrid in how a university means a portion of the course is online the other portion is a physical in class meetings and what I want to experiment with is, moving this course to an entirely online fashion. Basically simulate how offshore development works. Let’s say you have a team working in Atlanta, you have a team that works here in upstate New York, and you have another team in India or Poland or Germany, and they work together they have to coordinate somehow. So we’re gonna do this next semester. I’m excited, really excited for that.

John: Interesting. Will there be a synchronous component where you have everyone report?

John: Absolutely. So the reason why I said hybrid is because we’re gonna meet exactly twice in person. It’s going to be at the first class we’re going to actually physically meet. I tell them that from now on we’re not going to meet anymore. Instead, we’re going to meet online using an online meeting tool. The university has a couple of licenses that we’re friendly enough to allow me to use one. So we’re using this tool, we’re doing online meetings where everyone has to be present and has to do the same things we would otherwise do if we had physical, in-class meetings; the daily scrum, this is what I’ve been working on, this is what I’m gonna do next, this is what we as a team have been doing. So we still have the immediate feedback component, we can still plan ahead and we can still do all of this. The second time we meet will be at the end of the semester when we present the final project and when we show the final implementation to the stakeholder. Basically like a sales pitch. Of course that’s gonna be problematic because specially the usability folks, those part of the team who are going to be conducting actual usability tests with human subjects committee approval and everything, so we do it the actual way that a company does it, they of course have to meet. This is for next semester I’m actually thinking about having them fill out mock travel requests just to get them accustomed to this. So we’ll see how this work. I’m quite excited about this prospect. I looked at the class roster the other day and I think I have a really cool crew of really capable people and as things gonna be great.

Rebecca: What are some challenges that you’ve run across teaching project-based classes and some advice that maybe you could give to a faculty who’s newer to this methodology?

Bastian: I would still consider myself new to this. I’m actually junior faculty so I’m only, in quotation marks, an assistant professor at this university for just about three years. But our department usually have four as project involve classes taught by more senior faculty. One of the most significant challenges that have experienced this when you have disruptive students. Every once in a while you have a student who completely hates the idea of projects and frankly I was one of them when I was in grad school, I was I was one of them because at the end of grad school I was like if I hear the word project one more time I’m going to flip out. These days I have a different opinion of this. I understand that some people are just fed up with it and I understand. Especially when they have to work with other people that they don’t know that don’t have the same work ethic that they do, they get frustrated a lot. So a recurring problem is student frustration with other students. That’s why I joke with them and say well this class is not about skill acquisition, I don’t need you to know how to compile code, at this point I expect you know how to do this. I need you to learn how to be successful despite other people in your group. You need to be successful despite the fact that you’re running out of time. That kind of stuff. So it takes a little bit of convincing sometimes but usually you’ll find the trick is to find an amicable solution. Then if there’s conflict between people then talk to both sides and say listen, I’m not your enemy, I’m not here to point fingers, I’m not here to agree with you or disagree with you, I’m here to help you facilitate a compromise. That is sometimes challenging. It happens every single semester, but it’s challenging. My strategy usually is to listen to both sides and say okay and maybe you just used the word, the wrong words, maybe you use the wrong language, maybe there’s cultural differences, you have students from other countries and they might not have the same work ethic that you do they may work 24/7 it feels like and you will really appreciate your weekends off. That is fine that is a fine, thing to do we just need to be upfront about it we just say, listen Jane, I’m not gonna work Sunday nights because Sunday night’s is when I relax. Or hey, I’m sorry Wayne, tomorrow morning 8 o’clock is the only time we can meet, can you somehow make it happen? So it’s really about compromise and it’s the case-by-case thing but my strategy is listen to them all and if they can’t make a decision on their own, then I make one, and they just have to abide by it. Usually it’s not a problem.

John: Which is also a useful job skill because they’re going to be in these environments.

Bastian: Exactly. In fact, when I say we simulate the way a software company develops software, I’m not joking. We really do it. These conflicts that you have in a class like this are literally the same. Most students really appreciate the experience, they may hate going through it but they usually love it at the end. In fact two years ago, I had a graduate student who was a graduate student of human-computer interaction, of which our University has a master’s program, but her background I believe it was art. She came from an art background.

Rebecca: Probably a graphic design student.

Bastian: Um, I’m not certain about that, but probably. The strength of the HCI graduate program is that it has so many people from so many different backgrounds, which is a great asset, and you can draw from really greatly talented people. Unfortunately, the downside is well these people they may have taken exactly one computer science class ,namely introduction to programming, and they have never done anything software, ever, ever again. But this person she hated going through this class she hated every single second of it but now she is working for a rather renown company here in upstate New York and she says I’m really experiencing this every day of my life, and I’m so thankful we went through this. This is the best worst class you’ll ever take in your entire life. It’s not about making students suffer of course it is about making them experience something in a realistic fashion, and tone it down a little bit. I don’t want to be the evil boss, I don’t want to be the guy who okay’s everything, and the truth is somewhere in the middle and usually that kind of pans out. Another really challenging thing though is when you have the disruptive student. Not just someone who’s fed up with projects or fed up with people in the project but actually tries to sabotage it. Not too long ago I had a student who was let’s say, extremely convinced of their own opinion, and this person, they were very sure of their own abilities. They were very keen on arguing they would argue everything until you’re blue in the face. They would misinterpret people stopping to argue because they just fed up with it, with oh they just conceded, I won the argument. So I had this person actually say, what everyone is praising me for my great ideas. I said well, sure, but you’ve done these three components that you’ve developed for this project, and your team has used none of them. Your team is no longer inviting you to team meetings, on my recommendation, because whenever you were at a team meeting they would not get anything done. So what do you think, what do you think this is, this is not okay, this isn’t an okay behavior. So in the end we found a way to help this person become useful after all, for the team, but it was very very challenging. In this particular semester I would think that unfortunately half of my teaching load was probably just taking care of this one person. Later I found out from other faculty that they were difficult in other classes also, so it wasn’t really me or the class, it was just personal issue. Even though this person took a lot of my time, ordinarily this class is the easiest to teach because, I don’t need to prepare anything, I have no preparation some grading afterwards but no preparation. On the other hand, you also have to be ready to face anything. You walk in a classroom and you don’t know what fresh hell awaits you that morning in terms of conflicts, but as I said, it’s only experienced as conflict while you were in it, afterwards you’re laughing it off and everyone is usually happy that it happened this way. So that’s what I’m saying is like a rewarding class to teach, but it’s kind of tough.

Rebecca: I imagine you probably have busy office hours as well with project based learning.

Bastian: Oh yeah. So much so that my faculty website says, office hours by appointment only. In reality it means, if I’m in, I’ll probably have time for you. Because with classes like these problems emerge right then and there, and I don’t mean interpersonal problems I mean, oh snap, we really need to use this one server but the server just went down. What do we do now? Or, we’re using this Google API and Google did what Google loves to do, namely change their API, what do we do now? Or, not too long ago, we were developing Facebook integration and Facebook from one day to the next took away the ability to post across pages on Facebook. So the project was kind of dead in the water, what do you do now? And that’s the problem that emerges immediately and you have to fix it, the students can’t fix it. When the resources that they need vanish, they can’t help themselves, there’s no way they can recover on their own. So that’s when after a short brief moment of panic, where I panic myself, we have to fix it somehow.

Rebecca: And you become the magic wand. [laughter] That’s what my students think when they’re standing in line for project-based learning. It’s like they come in it’s like, please I can’t move forward.

John: Those are all realistic type problems that they will be facing.

Bastian: It happens all the time happens to companies all the time, if you’re in the reality of the situation is Facebook doesn’t just take this away neither does Google. Google as opposed to, for example Oracle, they don’t really change their Java API all that much and if they do they have support for the things that you use to use,it call it deprecated, Google just switches it off. But they don’t do it from one day to the next there is usually a period where they tell you, oh by the way in a year or so we’re gonna switch over this in that server. So technically as a student you could be prepared if you did enough research but realistically, they have to complete this project, and our semesters are 15 weeks long, they have to complete this in 15 weeks so you have to make some concessions. Then we’ll just redefine the scope we just focus on something else. For example, a little while ago Google took away the opportunity of making your own google map, and when I say that is not a google map of let’s say, I don’t know, Oswego New York. Using the Google map engine, make a map of your bedroom, that’s what I meant. So they took away that opportunity or they took away certain functionalities that we wanted in one of our robot projects. I said well, they can’t do that so what I’m gonna do instead? One student suggested, hey, can we use the Unity engine to model a room that robot moves in? I said sure. Unity is a game engine to make video games. I said okay sure, you can do that, but I don’t know unity very well. Actually, I don’t know it at all. So, we have people here on campus who do know this, but I’m having a feeling to become good enough at unity to make this project work we’ll take another semester of itself. So why don’t you do it the easy way? Take a picture of the room that you want to use, and then “restorize” it and just fake it till you make it. So in the end the project was successful despite Google’s API being on.

John: What are some examples of specific topics that are used in design class?

Bastian: So in the software design process class, the first time I taught it here in Oswego, we did a family tree website, like those find your ancestry websites that you can find on the internet. Mainly because my Dad, he now passed away, but my dad was really into that and he wanted a website just to show our own family tree. We did that which was marginally successful. It was a decent family tree some of the features that we initially shot for were not delivered but, you know, we can safely say it was a family tree. A year after that we did an automated clicker system and I know that John here, is very much a proponent of using clickers and classrooms. If you have seen that millionaire quiz show on TV, they have little devices, and you can basically poll the audience in the classroom or in a question or multiple choice type answers. So we implemented it, and I’m of the firm opinion that no student should have to pay money or anything because tuition is already high enough, so we implemented a free one. That was using students own cell phones and wireless network they could poll.

John: You had some classes actually use it as clients for protocall.

Bastian: That’s right. So I used it in my own introduction to programming class. I used that semester, I used them as guinea pigs. They were excited beyond belief. They kind of liked it. It was very buggy of course mainly because doing it over wireless is really bad protocol. Plus if you have a wireless network in a large lecture hall it is an even worse protocol. So there were some problems with it that we couldn’t just solve, that were just unsolvable to us. But in principle, in a small enough audience, let’s say inside of 20 students, it would work great. Last semester was particularly exciting due to a scheduling error by, I’m not gonna say whom, but say by certain administrative forces, I unfortunately and accidentally had twice as many students in this class as I was supposed to. I like to teach this class with like between 15 and let’s say 25. Because we have a lot of students sometimes we have to unfortunately have 30 students in this class. Last semester I had 50, so yeah.

Rebecca: Oops.

Bastian: That was awful. But I decided after I talked to our department head, Doug Lea, and he says well, what you’re gonna do, pick up people and kick them out? We decided that this is a really evil thing to do to students so we just bit in the sour apple and said okay fine, let’s do a red team blue team approach. Where we had the same project and we split the class in half saying you’re team blue, you’re choosing a different design solution than team red. They both implemented a Scrabble clone. Those of you have played Scrabble board game, and we can use words and play words, and the idea is that people would walk by a kiosk system, which is actually running right now and the entrance of our science building here, is a computer in a display case. It’s running a cloned version of Scrabble. People can walk by with their cell phones connect to a little wireless that is emitted there and then they get a hand dealt on their cell phone, then they can play words. Of course they’d have the usual problems like, the first person that walks by plays an unspeakable word, so we made it Oswego themed and say if you play certain words you would get bonuses and such things. I would just mentioned in the coming semester I’m going to teach this class for the first time mainly online and I’m thinking about doing a Productivity type software. Something like it connects to your email account and looks for what your emails are actually about; how much time do you spend in your emails, how much time do you waste? For me, as faculty I always feel like I’m doing 5 % teaching, 3 % research, and 97 million % of miscellaneous administrative stuff, so mostly probably emails.

Rebecca: Mostly email. [laughter]

Bastian: I want to know if that’s true. I want to see what do my email say I am communicating about the most? On the one hand you have to connect to Google’s IMAP account and download emails and then you’d have to some natural language processing to parts of speech in the email and so on. Of course there gonna be privacy issues with this. These days everyone is really concerned about privacy, as they should, so we’re gonna have a little team that is gonna be specifically concerned with making sure that we abide by ISO 27000 privacy regulations. Unless the students have a better idea of course. [laughter]

John: So our last question is, what are you going to do next?

Bastian: I’m really excited. I had a student, I was successful in obtaining funding for a student project over the summer, and this student built an indoor GPS navigation system for robots. Now when I say that I mean mainly the API. So from this grant money we bought a little ultrasonic location beacons, you could say, which can be distributed around the room and the robot gets another location beacon slapped on top of itself, and then the robot knows in relationship to all the other beacons, where it is. Using this little system he implemented a GPS type API that allows us to say, robot go exactly there, and the robot will drive up to two centimeters precise to that position. The robot has obstacle avoidance, it has pathfinding capabilities, and all that stuff. So one of the things that I want to do next is have a fleet of those robots, we have several of those robots, but only one of them is location aware right now. When I put location awareness on several other robots and then simulate let’s say exoplanet exploration, using those little things. Let’s say you have three or four or five or 20 of those robots roaming around in a large room and one of them finds an obstacle and says, hey guys, here’s an obstacle don’t run into. It tells all the other robots where that obstacle is and then the next time when the next robot comes around, to a similar location, and says oh here’s an obstacle, here’s the question; is it the same obstacle? Because if it is, then we don’t have to put two obstacles on the conceptual map, we have to do just one. So it’s something I want to do it also ties into into my research. Like one of the things that I’m really, really focusing on is to make sure that the students just don’t do boring little projects. Every student in computer science has implemented a library system or an ATM, you know boring, been done before. I’ve worked, as I said earlier, in cyber-physical systems and safety-critical requirements and such things, so I use those ideas in my classes and I want them to solve tiny little projects therein. I just mentioned earlier, we bought these programmable slot cars. What I want to do next is do obstacle avoidance and automatic cruise controls with those slot cars and just automotive type software engineering projects. That’s what’s happening. I’m really excited about that too.

Rebecca: Great. Thanks for joining us today.

Bastian: Thank you for having me, I’ve really enjoyed being here.

John: You’re doing some really interesting things there.

Bastian: I’m not doing any of them. [laughter] The students are doing them. I’m just there for the ride, really. [Music]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on Teaforteaching.com. Music by Michael Gary Brewer. [Music]

42. Flipping the classroom

Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode, Dr. Dominick Casadonte, a Chemistry Professor at Texas Tech University, joins us to discuss research and best practices related to flipped classrooms.

Show Notes

  • Camtasia
  • Mediasite
  • Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement. International Society for Technology in Education.
  • Sams, A., & Bergmann, J. (2013). Flip your students’ learning. Educational leadership, 70(6), 16-20.
  • Bowen, J. A. (2011). Rethinking technology outside the classroom. Journal of Music History Pedagogy, 2(1), 43-59.
  • Bowen, J. A. (2014). The Teaching Naked Cycle: Technology Is a Tool, but Psychology Is the New Pedagogy. Liberal Education, 100(2), n2.
  • Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve student learning (ed.). Jossey-Bass
  • Belford, R. E., Stoltzfus, M., & Houseknecht, J. B. (2015). ConfChem Conference on Flipped Classroom: Spring 2014 ConfChem Virtual Poster Session. Journal of Chemical Education, 92(9), 1582-1583.
  • Stoltzfus, J. R., & Libarkin, J. (2016). Does the room matter? Active learning in traditional and enhanced lecture spaces. CBE—Life Sciences Education, 15(4), ar68.
  • Stoltzfus, Matthew (2016). Engaging Students in the Flipped Classroom (video)
  • Coats, H. J. (2016). A study on the effect of lecture length in the flipped classroom (Doctoral dissertation).
  • Casadonte, D. J. (2016). “The Effectiveness of Course Flipping in General Chemistry – Does It Work?” ACS Symposium Series, “The Flipped Classroom”, December 2016 (Book Chapter) The Flipped Classroom Volume 2: Results from Practice, Chapter 2, pp 19–37, Chapter DOI: 10.1021/bk-2016-1228.ch002, ACS Symposium Series, Vol. 1228, ISBN13:9780841231627eISBN:9780841231610,
  • POGIL.org

Transcript

Rebecca: Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode we discuss research and best practices.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is dr. Dominick Casadonte, a chemistry professor at Texas Tech University. Dr. Casadonte is recognized as a global leader in flipped learning by the Flipped Learning Global Initiative.
Welcome.

Dominick: Thank you very much.

Rebecca: Today our teas are:

Dominick: Well, my usual afternoon beverage is an iced green tea with three pumps of raspberry, but since I’m getting over a cold, today I’m drinking a hot green tea with vanilla, lavender, and honey.

Rebecca: That sounds really nice. Finally, a tea drinker.

[LAUGHTER]

John: I’m drinking Irish Breakfast tea.

Rebecca: …and I’m drinking chai today.

John: You’re recognized as a an expert on flipping the classroom, and you’ve been doing it for a while. Could you tell us a little bit about what a flipped classroom is?

Dominick: Sure. The usual paradigm, if you think about it with regard to teaching, is lecture, homework, lecture, homework, lecture, homework, tests, lecture, homework, lecture, homework, and so on… and the problem with that format is that it’s not conducive to deep learning… with the exception of maybe very bright students or very bright asynchronous learners. Trying to keep up with a lecturer is often difficult. A student, for example, may not have gotten down everything that they wanted to write down… they may not have gotten it down correctly… or it might be fragmented… or perhaps they just didn’t understand well enough to write it down in a meaningful way, or a way that’s meaningful for them. Then they try to go home and they try to do homework, but since they didn’t really understand the lecture in the first place, the homework is again difficult and they might only be able to work part of it. So then, the very next day, there’s a new lecture and a new topic. The normal sort of didactic approach to education often leads to what I’ll call fragmented learning, in the sense that there isn’t enough time in class to practice many of the active learning strategies that work so well because faculty are so concerned with getting the lecture done. In the flipped model, on the other hand, we flip the homework-lecture-homework paradigm so that the students’ homework is to watch a lecture online. Now it can either be PowerPoint or a video and different people use different techniques. It can be done on Camtasia, it can be done through Mediasite, it can be done through a variety of different platforms. It could be done on YouTube for example. So, they’ll watch usually a video or a lecture online before they come to class, and then they may or may not do online homework prior to coming to class as a part of the pre-class experience. Then, once in class, time can be spent, for example, checking the students knowledge… clearing up muddy points or any misconceptions they might have… working advanced problems. But now, in this particular case, class becomes more of a discussion and you can use a lot of the active learning strategies that really engage the students in learning the material. By flipping the classroom, we freed up time to really engage and interact with our students and it helps with their learning in significant ways which I can talk about later.

John: So, in a traditional class, you tell students what the content is in class, and they come in with very different backgrounds and some of them are able to pick it up, others get lost along the way, and then they’re sent out to do homework… where students seem to have the most difficulty on homework or tests… and we’re pretty much leaving them alone to do that. But, here in the classroom, you’re able to work with them and help them through some of the issues.

Dominick: Right… and so, for example, in the classroom with homework, now they have a mentor who can help them work through the more difficult problems to help guide them along. There are models of the flipped classroom where there are peer mentors in the class who can help and walk around and mill around as well. So, there are opportunities for group work. There are opportunities for group discussion. You can engage the topics in a much deeper way than you can by just simply lecturing at students. So, it’s a very very non-passive way of teaching.

Rebecca: What prompted you to get into a flipped classroom model?

Dominick: Desperation, I would say. [LAUGHTER] I first started teaching an online class in the fall of 2007 as part of a multidisciplinary science master’s program of which I was a part here at Texas Tech… and getting everyone on the same platform at the same time, my thinking was that lecturing to them live online was a waste of time, basically. The question that motivated me was the same one that motivated John Bergmann and Aaron Sams around the same time at the high school level up in Colorado. Namely, what could I do to use my classroom time more effectively? What could I do to use my online time more effectively? …and I thought, “Well, let’s take the lecture out of the classroom.” So, I pre-recorded my lectures. I had the students watch the lectures before we got together in the online environment, and then we spent the majority of our time having discussion and working problems…. and these were teachers at the time and they were very very enthusiastic about the ability to actually discuss the material that they were trying to learn… and so then I thought “Well, hmm, if that works really well on an online format why don’t I try it face to face?” …and so then, starting in 2008, I adopted this to my general chemistry class. The following summer, a high school teacher that I had as part of a workshop said “Oh, I see you’re flipping your class” and I said “Flipping, what’s that?” ‘Cause at the time, there were a lot of different terms for flipping. It was called: time-shifted instruction, reverse instruction, blended instruction, all sorts of things. Flipping is a term that’s really sort of stuck.

So, I started out of a sense again of “How can I engage my students more effectively in the classroom?” and once I realized that it worked swimmingly in the online environment and then I said “Well, okay, will this work in the face-to-face?” …and it worked even better there. By that point, there are pockets of people around the country who are doing this, and they used all sorts of interesting terms. Jose Bowen at SMU, who’s an art professor, used the term “naked teaching” because it can be unsettling when you walk into a classroom and not have the comfort of your lecture notes to be able to project or read to the students. So, very often I will walk into the classroom and say “What are we going to talk about today? or “What would you like to talk about today?” You can’t do that unless you feel comfortable about the subject material and you have some expertise, but it’s a great way, in the long run, to really, really impact students.

Rebecca: You mentioned both doing this in an online environment and also in the classroom. Is there a difference between your experience in both?

Dominick: Yes, it’s so much easier face-to-face. To be able to walk around to gauge what’s going on in the classroom, their level of understanding, and how learning is happening. The power of direct peer-to-peer contact should never be underestimated, I think. Now, in the online format, my focus was more on the development of a learning community working on a particular topic, rather than in peer-to-peer mentoring and things like that. So, they’re very different approaches, but you do develop a sense of community in both, I think.

John: Now, with your online classes are they synchronous or asynchronous?

Dominick: My online classes are synchronous, in the sense that everybody meets in one place at one time. We have various software programs that allow us all to be in the same place. Yeah, so they’re all together at one point, and that’s kind of interesting because I have people from all over the country who are taking these classes. They’ve never actually physically met, at least for the first class that I teach. It’s interesting trying to, in the discussions, get a sense of the personalities of the people who were providing discussion.

John: Are they participating in video formats or is it just audio or text?

Dominick: I’ve done both. I’ve done video and audio and depending on the bandwidth and the number of students in the class both can work well.

John: How large are your classes?

Dominick: Well, my online class had 24 students in it. So, a fairly large, I think, from an online perspective class. It wasn’t one of these very very large classes like you might see at MIT, for example, but 24 is a good class in terms of bandwidth… trying to get everybody in the same room at the same time… and my face-to-face classes… they’ve ranged anywhere from a low of 25 to a high of 150.

Rebecca: You mentioned two different techniques that you use both in online versus in person. So, online you mentioned community formation and then in person you mentioned peer to peer. Can you expand upon each of those?

Dominick: Yeah, my first adage for teaching is know your audience, and in the face-to-face environment I was working largely with in-service teachers who were trying to develop a more significant content knowledge of chemistry, and so there the idea was the development of a supportive community where these teachers could bring their ideas to the table in terms of not only the content that I was teaching them, but also how they could then apply that content in their own classroom settings… and in that regard they were able to help teach each other techniques that they could use in the classrooms based on what they were learning in terms of the content. There really was a community focus, sharing knowledge as opposed to just gaining knowledge. In the face-to-face classroom, it’s more a sense of the students trying to understand the content at a deep level. There I found that it is true that when you teach something you really hopefully really understand it and so peer-to-peer mentoring is much more effective there. So, we tend to work in groups… we tend to work with dyads… two people working next to each other… and then sometimes I will just have people go to the board… but once again it was in that context of community because I think it’s very important, if you’re going to do flipping, and do it well, that it’s an active, encouraging, engaging classroom experience. If my students go to the board, whether they get the problem right or wrong, the class gets into a habit very very quickly of applauding the student for their attempt… whether or not it’s right or wrong… and then we debrief. We talk about what works… what doesn’t work… So, it really is both peer to peer and community building there as well. But, the emphasis is more on the individual in the face-to-face classroom, I think.

Rebecca: It sounds to me a little bit like the choices that you’re making online and in-person aren’t necessarily because of the medium, but rather who’s in those particular classes. Am I hearing that correctly?

Dominick: That’s right, and one of the nice things about flipping is that it is such a rich environment in which to work. As I mentioned earlier, the pre-class videos can be video lectures, they can be audio lectures (if that’s appropriate), they can be PowerPoint presentations, they can be any number of things. In the in-class experience, it’s an active learning environment. so you tailor it to the people in the classroom. For example, if you’re trying to teach in a flipped environment of the class of 24, there you can do all sorts of things that promote individual learning in ways that it’s a little bit more difficult to do in a class of 300, for example. But, I have a good friend, Matt Stoltzfus at Ohio State, for example, who routinely flips like general chemistry class of 600 students, and he’s able to give them as close to a personal learning experience as one can, I think. The point that I’m trying to make is that the flipped environment is a very rich one and it allows you to tailor the learning experience to your class specifically.

John: Do you create your own videos or do you use ones created by other people?

Dominick: I actually create my own videos. I have my own recording studio in my office… I have a Mediasite setup…. I have a video camera… I have a document camera… and I have a wonderful microphone…. and so every couple of years or so I re-record all of my videos. Now, I do use a mixture of other formats. So, for example I do have post-video homework that the students have to do online before they show up to class… and that’s done using an online learning platform that we have here at Texas Tech through a national distributor. The advanced problems that we work in class come both from the textbook that we use and also from problems that I develop. So, it’s sort of a hybrid. But, the pre-lecture videos I actually produce…… and one of the things that studies have shown is that students develop a certain sense of identity with regard to the person teaching the class. Some people, when they’re starting to flip, might want to just use Khan Academy videos, for example, or things like that. But the studies show that the class wants to see folks on the videos (or your voiceovers if you’re using PowerPoint with voiceover)… they want to see the professor who’s teaching the class, because that’s their professor… and so they develop ownership, if you will. Plus if you’re using other media, for example a Khan Academy, they may not be teaching it exactly the same way that you want to teach it, and so then you have to either reteach or undo. There’s a quality control issue there. So, it’s just easier, if you’re gonna do flipping, to make your own videos… and there are so many different ways of doing that now that there’s really no excuse.

Rebecca: That had been my experience as well when I’ve done videos in my classes. The students really liked the quirkiness or knowing that it’s the same person that they had in their classroom and if you try to slip in something else occasionally they really didn’t like it.

Dominick: …and a lot of people have a perfectionist tendency and really want their videos to be really super perfect. Well, once again, studies have shown that that’s not really what students want. They want to see the foibles. They want to see you as you are. If they know that you’re going to say “um” or “uh” in the classroom then if you don’t say “um” or “uh” in your videos, then they’re gonna say “Is that a robot teaching the class that looks like my professor?” So, it’s okay to be human when you’re doing the pre-lecture videos. But, I think one of the things that often hangs up people when they’re starting to do flipping is this notion that it has to be perfect, and it really doesn’t.

Rebecca: I think they really appreciate when you make mistakes and things, too. I know that my students did when I do like a coding mistake, I’m like “Whoops, I made a mistake..” and go back and fix it and explain what I did and why it was wrong. We all make those kinds of slips and errors and things and we would do it live. So, it’s kind of nice to do it in videos, too.

John: It makes you seem more human by doing that.

Dominick: One of the nice things about them seeing you make mistakes is that it gives them permission to make mistakes. They don’t have to be perfect. I’ve had experience with students in classrooms where they come in and they’re intimidated… they’re shy… they’re just afraid that they can’t master the materials. So, seeing somebody make a mistake who is an expert gives them permission to make mistakes… and one of the things that that really does is it empowers them to learn, because at the end of the day when somebody’s trying to learn they’re going to make mistakes and I give my class permission to make mistakes. In fact, I tell them you have permission to make mistakes while you’re learning. After you’ve learned something, that’s a little different. If you’re an engineer, for example, and you’re building a bridge, I don’t want you making mistakes. But while you’re learning… absolutely, make mistakes. Part of education is this movement from novice to expert and in that process one makes mistakes. So, I told my class “You have the right to make mistakes” and I use an example. So, let’s say you’re a five-year-old and you’re trying to learn how to ice-skate you fall down, what does a five-year-old do? They laugh, they brush themselves off, they might giggle a little bit, they get up and then they just skate some more. Now, imagine you’re an eighteen year old and you’re learning to ice skate and you fall down. Most of the time, people stand up and say “Did anybody see me?” and they worry about what people are gonna think, instead of just getting up, laughing, and moving on. So, what often happens is the 18-year old never learns to ice skate, whereas the five year old, who’s willing to make mistakes, learns. So, I tell them it’s okay to make mistakes while they’re trying to learn… and also it’s about empowering students to be able to have confidence in themselves… and we talk about this a lot. We did a study of what motivates students in the flipped environment, and part of it is the confidence… the autonomy… that they develop in the flipped environment. When they really think they’ve got and they really understand, there’s a certain level of “Geez, I understand this. I can understand the next thing…” and so on and so forth. That sense of confidence really improves their educational experience.

John: How long are your videos? Do you tend to have very long ones or do you chunk them up into smaller chunks? and what would you recommend in terms of video length?

Dominick: We did a study a few years ago on the optimum length for flip videos, and it came about because our book dealers: McGraw-Hill, Prentice-Hall, Cengage… those are the top three that we have here at Texas Tech… were telling us that they were creating videos and they’re creating these five to seven-minute videos… and I said why are you making five to seven-minute videos?” and they said “Well, everybody knows that the Millennial generation has a five- to seven-minute attention span and so they want five to seven minute videos.” I said: “But, what is best for learning and improving learning outcomes? I don’t care what their preference is. What’s best for them to learn?” He said “Well, we don’t know.” I said “Well, do you have any data or evidence that shows the five- to seven-minute videos are really great for learning?” and they said “No, but we would be interested if someone would do a study and tell us.” So, I had a graduate student who embarked on a study of lecture video length. We set up essentially short video people… those are people to watch five to seven-minute videos… and we did this with master videos, so each video was probably 40 minutes to an hour long. We put stop signs in the video and when they hit a stop sign they would stop. They would work some online homework and then they could pick up again… or they could go off and do something else. but the short videos were five to seven minutes and then we had a long view group, as it were. We allowed them to decide which videos they wanted to do and not surprisingly, 62% decided to do short videos. So, I have no problem with the book dealers’ notion that Millennials prefer shorter videos. But, then we let the semester go on and we didn’t force them to stay into one group or the other. We let them move and we watched their video habits. We weren’t video stalking them, but we could watch their lecture habits using the Mediasite analytics down to the millisecond.

What we found was that 60% of that short video group switched to the long video length, which is very surprising to us, and we did also a variety of assessments. We looked at online homework grades. We looked at quiz grades. We looked at exam grades. We looked at final exam grades. We looked at the American Chemical Society standardized tests that we give as a pre-post and what we found was that there was a subgroup of the long-view group that watched the video as a long video, but then they stopped at specific points to either have a snack with a friend, go to the bathroom, whatever they needed to do… and we called them the long pause people. It turned out that in every assessment that required global understanding… so final exams, ACS exam individual exams… the long-pause viewers actually scored one standard deviation higher than the short viewers. In fact, the short viewers had the worst learning outcomes of all three groups. Then we gave them Likert scale questionnaires, and we also gave them open-ended questions, and we said “Why did you make the switch? What do you find?” and they said “Well, it just got too fragmented to look at these short videos and we couldn’t take what we saw in video A and connect it to video B and so on and so forth, especially if several hours had gone by, because we then had to go back and watch video A again to remember what we forgot. But, if we did the long videos we were able to just put it all together.” Also in that we found that the best optimum video time. So, what constitutes a short video versus a long video for Millennials is 20 minutes or less is short, 30 minutes or more is long. So, 20 to 30 minutes is the sweet spot for video length. That’s what the study showed us and we’ve just submitted that for publication.

Rebecca: Have you adjusted how you’re teaching based on that information?

Dominick: Yes, my videos are roughly in the range of 30 to 40 minutes, but I tell my students “Take the time you need. Take breaks. It’s going to help your learning outcomes.” I also share with them the results of our studies, because I think if you’re going to, in my case if you’re gonna be a scientist, you should be data-driven. So we want our students to know that we’re not just telling them this because it’s something anecdotal, but rather it comes from data that we’ve collected.

John: Have you thought about controlling for self selection and randomly putting students in groups? Because one concern I’d have with that is that it could be the case that those students who select the short videos might have done less well no matter which group they were in or vice versa… although you do have switchers in there.

Dominick: Yeah, I’ve actually been very lucky in my studies that I teach our honors general chemistry sections… and so I look at the SAT scores… I look at their previous class performance… and it’s a very very, as much one can have, a homogeneous group. So there’s really not much of a selection bias, I think, as far as the study goes.

Rebecca: You’ve also done some research on the flipped classroom approach in general, not just the video length. Can you share some of your findings?

Dominick: Sure. We found out a lot about the flipped environment over the past 10 years or so. As I mentioned a minute ago, I’ve been very lucky in that I tend to teach very bright students. I did a five-year longitudinal study on the effect of flipping which has been published in American Chemical Society monograph and we found that the average exam grade increased by 9.2 percent over that five-year period, and that the largest gains in learning came during summative assessments (for example during final exams and externally developed independently normed exams like our ACS exam). We also have done work on what motivates students to do well in the flipped classroom and we just recently presented at the biennial conference on chemistry education regarding the effectiveness of peer mentoring during the flipped classroom. The results there were very astounding to me. It shouldn’t have been because peer-led team learning has been around for more than 15 years, but nonetheless, in trying to tweak the classroom, the addition of the peer mentor took what was already much better than had been before and improved it dramatically. I’ll give you an example. On the American Chemical Society end-of-term exam, without the peer mentor present… So, I give my students incentives if they score above the 95th percentile on that exam, and it’s a challenging exam, they get an automatic “A” in my class. They don’t have to take the final because this exam is normed against thousands of students around the country in a variety of different university settings. Pre-flipping, I had zero to one student scoring above the 95th percentile; post-flipping the average was roughly nine. So, it increased tremendously. With the presence of the peer mentor, the number went from 9 to 34 and so almost a six-fold increase in the number of students scoring above the 95th percentile. The only difference being, since I used to run the discussion sections that I subsequently allowed the peer mentor to do, that was really the only difference in the environment as far as I could see in terms of controlling for the different factors, except for the obvious one of different students. But she did this in the fall of 2016 and the fall of 2017 and there were 34 in one in the fall of 2017, 33 in the fall of 2016. The percentages of the class actually increased. This is kind of reproducible, if you will. So, we’ve looked at a variety of different things, and with regard to motivation we looked at that from self-determinacy theory and found essentially three things that really sort of motivate people: one is autonomy, second is pace, and the third is responsibility.

In the flipped environment, this sense of autonomy… the sense that “Oh yeah, I am really learning something” is very important to the students. The pace, the fact that they could watch the videos in their pajamas, for example, was very important to the students as a motivating factor… and responsibility, the fact that they had to take responsibility for their own education as opposed to being sort of spoon fed in a lecture format was something that motivated them as well. That student just graduated in December, so we’re now preparing that for publication. So, we looked at a variety of different aspects of flipping.

Rebecca: Can you clarify about your peer mentor model? The students are watching videos outside of class, and then they’re coming to a class with you, and then also recitation session with another student?

Dominick: They watch a pre-lecture video. They do online homework. We have a class discussion… work problems… and then there’s a separate recitation section… and historically I have done all of that. I had a very bright student who really wanted some teaching experience and she said “Would you let me run your recitation?” and I, like many faculty perhaps, don’t really want to give up control of my classroom environment. So, it took a lot of cajoling on her part to get me to do that…. and I said “Well, let’s look at it and see what happens.” She took over the recitations and… talk about a motivated young lady… she would provide review sheets for them… she would do all sorts of things that I would do, but the way she would do it, I think, spoke to the students so much more effectively than I was able to do. I think that’s one of the real reasons why their scores went up.

Rebecca: She knew how to meet them where they were at in a way that, as we become more of an expert in our field, we lose touch with that.

Dominick: Yeah, that’s exactly right. So, one of the things that I’ll be interested in following is that there was a recent study that Prentice-Hall published about the Millennials versus the Gen Z students… and one of the things they noticed is that, while there was about a sixty or so percent “like” amongst the Millennials for using computers and computerization and video and things like that, that number was almost half for the gen Z population. It’s almost as though it’s such a normal part of their life, the gen Z population, that there’s nothing special… there’s nothing unique about it… there’s no value-added to doing videos… it’s the sort of normal expectation. So, I’ll be very curious to see how this sort of flipped environment works over the next 10 to 15 years when the expectation is that they’ll be seeing videos at some point either prior or post classroom experience.

Rebecca: …definitely an interesting question.

John: You’ve organized several symposium on flipped classrooms. What are some of the biggest takeaways from those symposia when you bring people from many different disciplines together? Are the results pretty similar across disciplines or do they vary substantially?

Dominick: Well, I think the first thing that struck me with regard to these symposia is how diverse the flipped environment really can be. Since active learning occurs during the classroom time, there’s almost as many different active learning strategies as there are teachers, and so no two flipped classrooms are the same… and that’s the first thing that I learned. The other thing that I learned is that you have to be committed to flip. You can’t do it half-baked. If you try to do flipping, there was one example of a professor who said “Well, all this is is what we’ve always done. You tell the students to read the material before they come to class and then we have a discussion.” So, he said “Okay, I’m going to ‘flip my classroom’ and just have them read the textbook before they come to class and then we’ll have a discussion.” He found very quickly that the students weren’t reading the textbook, so there was very little discussion going on, which frustrated the professor and frustrated the classroom and set up more of an adversarial relationship.. and it was the worst teaching experience he ever had and he said “I’m never going to flip my classroom again.” So, one of the takeaway messages, and this was reported at one of the symposia, is that if you’re really going to flip your classroom, you’re in for a dime, you’re in for a dollar. You do it as well as you can and be very concerned about what you’re putting in and what you’re expecting to get out or it can be a very very bad experience. Now, I will say this, that almost everybody when they get up on board the flipped bandwagon, especially if you’re using technology prior to classroom, it’s hard at first. That’s the other real take-home message. It takes a lot of time to flip your class the first time. But, once you do it, it actually is much more enjoyable. It’s actually easier, I think, than the regular didactic approach. Those are some of the take-home messages.

Rebecca: Can you talk a little bit more about some of the other challenges faculty might face if they’re doing it for the first time?

Dominick: Sure, yeah. The first and biggest challenge that people have is time. Don’t decide you’re gonna flip your classroom two weeks before the start of a semester. It can be the most horrible experience you’ve ever had as a teacher, because students have an expectation that each lecture is going to be there for them. As we all have various things come up or university requirements… meetings… things like that… and you may find yourself at 3 o’clock in the morning trying to record a video for dispersal at 8 o’clock in the morning. So, don’t wait till the last minute. That’s the biggest sort of thing. That’s one of the reasons why people are sometimes a little bit risk-averse with regard to flipping their classroom.

Other things that have come up: saying “Oh, I can just use Khan Academy videos or videos that are on the web” …and they haven’t previewed them, for example, and then they find that the students have a very different concept of the material than the professor has and then you end up spending a lot of time: “Well, which one do we believe? Do we believe the video that you showed us? or do we believe what you’re telling us?” and so it can create an environment which doesn’t propagate trust in the learning experience. Other issues are: online homework versus homework that you put together for the students… what are you gonna do during class time? how are you gonna fill that time. If you’re not familiar with active learning strategies, that can be very daunting. For example, you walk into a classroom and the students have already watched a video… they’ve done some homework… So, what’s your value added? If you’re not used to active learning strategies, that can be very difficult the first time that you’re doing it…. and then do I need to change my assessment strategy based on the fact that now I’m using a different kind of pedagogy? So, there are a lot of different moving parts and I think putting all those moving parts together can somewhat be inhibitory for people who are trying to flip for the first time.

John: What types of active learning strategies do you use? You’ve mentioned some group work on problems, but could you give a few examples of types of activities you use during the class sections?

Dominick: Sure, there’s group work first of all. I send students to the board. I’ll pass out file cards, for example, and ask the students to put down one thing that was unclear in the video or something that they really would like to learn about in addition to the video. So, it’s not just muddy points, but it’s also how do we expand and extend. Because in my case it’s an honors class and I want to give them a little bit more than the normal amount of material and experience, and so those are some of the things that we do. We have to make molecules and structures and do all sorts of things, and so I use human atoms. I have volunteers come up to the front of the class and they have to then make molecules. They have to develop particular structures. They have to show how they bond, how they vibrate, how they move. What they do. So, they have to actually sort of insert themselves into the molecular dynamic, if you will. They’re trying to understand that, as an atom won’t understand its environment. The first couple of times, students aren’t used to that level of kinesthetic learning, but once they get it, then I usually have a fair number of students who are willing to volunteer and come up. Because, while they’re doing that, we’re also discussing, and I’m asking the class: “Okay, so why is it that this particular atom won’t bond here? What’s wrong with this?” and so it’s a real discussion… but now using human beings as the models rather than just making stick figures or things like that.

So, I try to move students through a variety of different learning environments that engage them not only visually, auditorily, but also tactilely. and kinesthetically, and that’s somewhat unusual, I think, in chemistry classes. Because, once again, most people don’t think of chemistry as a visceral activity in many cases.

Rebecca: Actually, it sounds like a lot of fun.

Dominick: Well, and it is, and I’ll tell you I walk into my class the very first day and they’re all very respectful, because Texas students typically are very respectful and they’re honors students. So, they really want to make a good impression. I usually start my class laughing, telling them this is the last time this class will be quiet. A noisy class is a learning class, a quiet class can be a sleeping class, I don’t know. So, yeah, my class is always very, very engaging, I hope.

Rebecca: Have you ever had the moment when you’ve asked your students “What do you want to talk about today?” and then not have anything they want to talk about?

Dominick: Yes, I think every faculty member has that “Oh, my gosh” moment. I always come prepared with questions, so if they don’t have anything to talk about, then I’ll ask them “Well, what did you think about this particular part of the video?” or “Did you really understand this?” or “Let’s take this concept and move it farther” because one of the things I never do in my classroom is just do a rehash of my video. I figure they’ve watched the video. So, I might say “Well, okay, you saw this but how could we apply this in this other setting?” …and then if they really didn’t understand it, then I’ll be able to tell in a heartbeat whether or not there are real questions because they really don’t have a good sense of understanding… and then I can go back and say “Okay, at what point is this breaking down for you? How did this not work?” Silence kind of tells me that there is usually a breakdown somewhere… and so I try to address what that breakdown is and try to correct it. So, that’s what I meant earlier when I say one of the things I do during my class is try to clear up misconceptions… try to address muddy points… and just make sure that they really understand the lecture part as well. That can take anywhere from five minutes to 45 minutes. My classes are an hour and a half long. Depending on how difficult a lecture was… on what their level of understanding is… So, I want them to have deep understanding of the content and so if they’re coming in silent then I worry that that depth is not there.

John: When they see the videos, you have them take tests. Are the tests the same for each student? or do you vary the questions? Is there some randomization there?

Dominick: Well, no. So, we give departmental exams, so the exams are the same for every student, basically. I know many universities have a format and they can vary the numbers that are put in, for example, but no, my students really… they’re fairly separated when it comes to the exam, so I don’t really feel the need to give each one a separate numerical set of calculations.

John: …and it sounds like many of your students are honor students where that might be less of an issue.

Dominick: Right, and we have an honors code through the Honors College, and I tell them the first lecture if you’re cheating you’re out of here. If you’re cheating you’re out of the University… and I’ve had the unfortunate occasion to have students suspended from universities… not at Texas Tech, but at other places I’ve been… and so they understand that I’m very serious about that.

John: On those tests, do they have multiple attempts or just a single take on the test?

Dominick: Well, my tests are usually set up so that they’re half multiple choice, half free response. Because at the end of the day, I tell my students “Qe don’t live in a multiple-choice world… A. agree, B disagree… you know, we don’t… and so I need to know how you’re thinking. While multiple choice exams are expedient in terms of grading, they don’t let me know what you know. So, I give them both because some students like multiple choice exams. They think they’re really good at them. Some students really want the free response. So, it’s a mixture of both. I try to give them a rich assessment environment as much as possible.

Rebecca: So, to follow up on what John was asking, are your homework assignments kind of a multiple attempt to help learning? or is it a one attempt kind of thing?

Dominick: It’s a one attempt. Yeah. Now, what I do allow them to do after they’ve provided their answer, is that I’ll allow the question to be open so they can go back and review it, especially if the answer is wrong…. and I allow them to do that to help them review for the exams as well.

John: So, they can go back and retake it, but only the first attempt counts towards their grade?

Dominick: Right. So, they have to think about what they’re putting in there before they put it in. Because in some cases, I’ve heard stories of students who put a wrong answer in purposefully and then the online learning environment gives them hints or tells them how to work a problem just like that… and then they go back and they have another problem with just different numbers but they’ve already been coached essentially in terms of how to answer the problem. Once again, perhaps it’s because I think you have to do things right eventually at the end of the day, I really want them to get it right… and these are… once again… they’re relatively straightforward pre-class questions. Their designed as just-in-time or warm-up questions. They’re not multifaceted. The questions we’re getting in the class are really challenging problems. They’re challenging problems using an honors book. So, hopefully they differentiate between those.

John: Are they graded on the problems they do in class as well?

Dominick: They’re not. Because part of that environment, once again, is to have groups… have community… have a mentoring process… and so, the ultimate goal in that whole process is the solution of the problem. So, I’ve already tested them in the online learning environment. I’ll test them on the exams. I’ll test them on quizzes. But, in the class I want the process of how to solve the problem come forth and not the grade be the most important thing.

Rebecca: So, we always end or wrap up our podcast with the question: what are you gonna work on next?

Dominick: Well, I have a pretty active and diverse chemical education research group and with regard to flipping, specifically, I mentioned that I had a recent PhD who looked at motivation in the classroom and what we found there were there were basically three reasons that motivates students to want to do well: autonomy, pace, and responsibility. Through the flipped environment they learn how to develop confidence in their ability to learn, and secondly they liked that the class is largely self-paced and they get to watch the videos in their pajamas if they want to, for example… and finally they really appreciate the fact that they are responsible for their own learning. So, we’re going to be looking at the role of metacognitive intervention as an autonomy motivator in the flipped classroom. That is to say, if we help them think about how they’re thinking during the early parts of the flipped classroom, do they proceed to confidence in their ability to learn that much faster… and we’ll also be looking at how the flipped classroom, especially with community building activities and community building learning strategies can improve the learning outcomes among historically underrepresented communities in the sciences… though especially communities where the notion of family and community is so important in their lives… that are not necessarily in the classroom. So, those are the two areas that we’re going to be moving into with regard to flipping… and I have a number of other projects that are not related to flipping as well. So, it’s a very diverse group of questions that we’re trying to answer, but once again I think that the flipped environment is a very value-added environment for both the students and the faculty… and so I think it’s a mature pedagogy in the sense that we talk about process oriented guided inquiry learning (POGIL) being mature and peer led team learning activities (PLTL) as mature pedagogy. Service learning is another mature pedagogy that has matured over the last 20 years or so… and I think it’s now safe to say that flipping is a mature pedagogy. In fact, at the biennial conference on chemistry education, there was a wonderful paper doing a meta-analysis on flipping and the presenter showed that in terms of looking.. I think he looked at 18 or 19 different papers on the flipped environment… and he found that, in general, there’s about a 30% improvement in student learning outcomes and it’s even better in organic chemistry than general chemistry (which was surprising to me). But, nonetheless it really does improve learning for students and that, in the final analysis, is what we’re trying to do.

Rebecca: It sounds like some really interesting projects. We’ll be looking forward to finding out what you find out.

Dominick: Thank you.

John: Well, thank you. this has been fascinating.

Rebecca: Yeah, thanks so much for spending some time with us today.

Dominick: Thanks, I’m gonna have another sip of tea. Thank you.

Rebecca: Thank you
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.