58. Role-play

Do your students sometimes settle for a superficial understanding of your course course content? Role-playing activities can provide an opportunity for students to become more fully immersed in the academic dialog of your discipline. In this episode, Jill Peterfeso joins us discuss a variety of role-playing activities that can be implemented into a single class session or over a more extended period of time. Jill is an Assistant Professor in and the chair of the Department of Religious Studies at Guilford College.

Show Notes

Transcript

John: Do your students sometimes settle for a superficial understanding of your course course content? Role-playing activities can provide an opportunity for students to become more fully immersed in the academic dialog of your discipline. In this episode, we’ll discuss a variety of role-playing activities that can be implemented into a single class session or over a more extended period of time.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Jill Peterfeso, assistant professor in and the chair of the Department of Religious Studies at Guilford College. Welcome, Jill.

John: Welcome, Jill.

Jill: Hi, nice to be here.

John: Our teas today are…

Jill: I am drinking candy cane tea; it’s a black tea. If you like peppermint tea, this takes it up a notch with even more sweetness. It’s really delicious.

John: What brand is that?

Jill: Adagio.

Rebecca: That sounds right up John’s alley, actually.

Jill: Oh really? For Christmas a couple years ago I asked for some and my parents are like, “Oh, what size?” And I was like, “You know what? I don’t know. A pound.” Well, you know how much a pound of tea is? [LAUGHTER] So I have enough to last me like a decade.

John: I had a mix of tea where it was peppermint, spearmint and tarragon and I got a pound of peppermint, a pound of spearmint and a pound of tarragon.

Jill: My gosh. [LAUGHTER] Yeah.

Rebecca: Speaking of lifetime supplies. I have English afternoon.

John: Again?

Rebecca: Yeah.

John: And I have blueberry green tea. We invited you here to talk a bit about how you’ve been using role-play in your classes. Could you give us some examples of what you’ve been doing with this and in what context?

Jill: Yeah, absolutely. This idea of working with role-play comes from my own interest as a theatre person in high school and college and even into my adult years and also just this memory I have of doing theatre where stepping into the role of another person opens up your mind in really different ways. I’ve devised a number of different things that I do in the class, sometimes borrowing from others, sometimes doing completely experimental assignments. So, I think that it’s sort of three different levels of immersion into role-play. A level one thing, for instance, might be I use dialog tests where I have students imagine dialogues. For instance, historical figures John Winthrop and Anne Hutchinson are having a conversation about conversion… what do they say to one another? And so instead of writing an essay on the thesis of conversion in Puritan New England, I have students imagine a conversation between these two historical actors. So, that would be an example of something that’s level one. Something that’s more level two… I often invite students to take on the voices and the ideas of authors, theologians and theorists that we’re studying. For instance, I teach an upper-level Holocaust class where we often read a lot of excerpts from very dense critical theory, for instance. I will assign students to different authors we’ve read… someone will get Habermas, someone will get Adorno, et cetera, et cetera, and then we come together in a colloquium setting and they need to speak in the discussion as that “author” or that character… and then sort of the deeper level of immersion would be something like reacting to the past, which is a very well established pedagogical role-play method in historical game that comes out of Barnard College and about 20 years old now—is developed by history professor named Mark Carnes—and in that students literally are assigned historical characters and then they play out some event from the past. Games that I have used in my classes include the Narrative Life of Frederick Douglass, Anne Hutchinson (who I mentioned earlier), the Council of Nicaea; I’ve done those in my classes, but they have them for all sorts of disciplines in all different time periods. So, that one, it’s several days. Students give speeches and form teams and do some politicking behind the scenes to come together and play their characters in order to see, like, literally playing with history.

John: In that second level of role-play, when they’re in the role of characters do you have students discuss contemporary issues or issues of the historical period?

Jill: When they are speaking as theorists or authors then I have them sort of in the secondary source mode so that they are speaking as contemporaries, even if it’s an Adorno or Habermas —who weren’t so much contemporary for us anymore—they’re still able to speak about contemporary issues. If we’re talking about Holocaust Studies, for instance, they are able to bring some of that to bear on whatever is happening here and now. I do something similar in a feminist theologies class where we read various feminist theologians over the course of the semester and one student is assigned a theologian each semester, one student per one theologian, and when we discuss that theologian the student speaks not as a student but as the theologian. So, it’s this extra meta-level that they that one student wears in this one moment or I should say in this one class period and what that lets them do is have this dexterity where they are connecting the text that they have a certain intimacy with as the “author” but then they’re also able to connect with their classmates who might be bringing up some of these different issues, like how this reading in feminist theology might connect to some of the reproductive issues that are going on now in politics or issues around concern for the planet, et cetera, et cetera. Really to your question, John, the way I see it, especially in that second level, is a hinging where they’re able to sort of pivot between the creation of the text and the application of the text and that’s one of the nice things about it because it keeps them again hinged where they’re connected to both parts and they’re aware of the fact that they’re swiveling, if that makes some sense.

John: It does. It sounds like they’re making some really deep connections.

Jill: Hopefully.

John: It’s a form in a sense of peer instruction.

Jill: Absolutely, yeah, thank you. That’s one of the things that I try to get them to do with role-play… with other activities I do in the class… but the role-play specifically… is to get students to realize that they can be instructors of their peers and just as successfully as I could be in some instances. It lets them feel that they are experts in what they are speaking on. That is ideally very empowering, but it also gives them—and I found this constantly with role-play, and this is something your audience might find interesting—is that when students are wearing a “mask” of someone else’s ideas or someone else’s character, they are much more willing to be directive with their peers and sort of challenge their peers if their peers are not thinking very critically or very clearly… and I’ve heard this from my students for the theologian activity in feminist theology… they get more annoyed if their classmates are not really stepping up and not really engaging “their ideas.” So, they’re able to say, “Well, wait a second, that’s not what I wrote. Look at the bottom of page 36…” and yet it works because no one really feels attacked by someone playing the persona of Elisabeth Schüssler Fiorenza. So, it works really nicely because that framing… that mask… however you want to think about it… creates the opportunity to step into a liminal space where it’s a little safer to push those boundaries, and students tend to do that and that allows them to do that peer instruction even more so than they would otherwise. I don’t think they think of it that way until after when I have them reflect, then they’re like, “Oh, yeah, I felt more defensive of these ideas and I also felt like I knew where they were coming from.” There was a material, historical context that gave rise to the need for me to write this theology or me to write this theory and they felt that attachment to it. To hear them say that I’m like, “Yes, that’s exactly what I would hope would happen.” As somebody who’s done acting and done theater, that’s the best part for me, immersing yourself empathetically in another experience and so it seems to work for students intellectually in a scholarly way.

Rebecca: …seems really powerful, but I can imagine that telling your students that they’re gonna role-play could be really intimidating. So, how do you prepare students for that experience?

Jill: Yeah, this is one of these things that I’m constantly trying to get better about. I tell them early and often. Whoever signs up for early on—so, again, right now I’m thinking about the feminist theology class where they have to step into and embody these ideas one at a time over the course of the semester—what I do is try to get some of my stronger students who might know me or have done this before in other classes to go first and I make sure to give them a special amount of direction and leeway and then after one or two students will go I will do a reflection like a stop, okay, what’s working. Students who are not doing this but were in the classroom discussion with the theologian, what are you noticing? And students who did this, what advice do you have for others? So, then again, the peers become the instructor. When it comes to other things, I mentioned earlier in my Holocaust class, and we do this sometimes in feminist theology, we do this in my Jesus in Film and Pop Culture class, where we really will be in a circle discussion and I mostly teach seminar. Disclaimer: most of my classes are 10 to 25 students, so this works really nicely. We’ll be in a circle and we’ll be looking at each other and channeling historians and scholars of the historical Jesus or Holocaust theory around memory studies. We might get into it and I’ll need to stop and say, “Okay, I want to pause. I noticed some of you are not speaking in the first person. Remember, I want you to be speaking as you’re scholar. Some of you are doing a really nice job with this, but I don’t hear you using quotes from the text, so remember the text is your foundation. The text is what gives you a platform. You don’t have to make up everything. You are using the text as a springboard to merge with your own ideas.” Constantly of doing that, modeling some of it for students as well and then affirming them. I think this all ultimately plays to where the majority of students get it at some point. But, Rebecca, I will tell you, I mean, you’re right. Some students never really get into this. They think it’s too strange or it’s too uncomfortable, or they’re really good students in the traditional way of doing things and they don’t think that this is something that they need or is helpful. That’s fine; not everything is going to work for everybody. What I love about some of these different liminal activities is that they will reach students who otherwise would feel that they can’t step into discussion in the traditional way because they don’t think they’re good at it, but giving them this additional costume of intellectual ideas to wear is liberating for some students, and that’s enough for me to do it once or twice a semester in some classes because it’s gonna invite in people who might not feel invited another way.

John: How long do these activities run? Is it a one day thing or multiple days?

Jill: It all depends again on which activities. When I do the symposia type models where we’re all together that’s usually at the lowest… it would be our 75 minute classes; sometimes I do it in our three hour classes, then it’s more about two and a half hours with a break in between. I’m getting ready to do one of these symposia in my Catholicism course and we’re gonna do it over two 75-minute classes, totaled about two and a half hours. What that does… and this ties to your question, John… what that does is it allows me to not be anxious that, “Oh my gosh, we have so much to cover and we’re not doing it,” and it really pushes me to the side, which is another key issue with this role-play is I as the professor in an ideal world create the settings, create the condition, give the instruction and then get the heck out of their way and let them stumble a little bit, let them struggle with some silence, let them look awkwardly at each other, let them look pleadingly at me but then turn to each other and realize, “okay, this has got to be us.” I encourage folks who want to try these sorts of things to give time because just investing in time means you’re gonna let the silence happen. Some things are much longer, so reacting to the past, for instance, which I’ve been playing with for the past year or more, that’s several weeks. We did the Anne Hutchinson game in my Religion in the U.S. class just last month and that was five 75-minute days and we’re gearing up for the Frederick Douglass game that starts next week. That’s gonna be six days. So, six days of game playing and then prep on the beginning and prep at the end. Doing that role-play meant completely redoing my syllabus for that course. There were reasons that that made sense given my teaching condition for that class, which I can get into if you’re interested, but that was a real total revamping. Everything from little bits to larger bits, depending on what you’re willing to invest in and what you’re looking to do with your students, what kinds of skills you’re trying to emphasize.

Rebecca: Jill, for someone who doesn’t have a background in theater, but…

Jill: Yeah.

Rebecca: …might find this to be a really interesting idea, what would you advise them to look at or how to start or an activity that they might do the first time out to just get their feet wet?

Jill: Reacting to the Past is a premade pedagogy. There are so many games I would recommend anyone who’s listening to this and thinking that sounds interesting to go to their website ‘cause those folks who run the Reacting Consortium will help you and there are so many games. I go to Reacting as a theater person who’s like, “Oh, won’t it be great if we all just sort of immerse ourselves in these characters of this historical moment and then give speeches as these characters,” this is like Jill in high school who did murder mystery weekends with her friend, like it’s getting these characters in and improving dialogue and a relationship and it’s just so fun, but that’s what gets me stuck on Reacting. A lot of folks who do Reacting are more gamers, they like that there are victory conditions and points for winning, or they’re historians who like this different way of doing history. That’s just my hook, but that’s not everybody’s hook. There are plenty of people I’ve met in the Reacting world who would never have thought of themselves as “a theater person.” So, I think that’s a safe one. Reacting has games as short as a day, as long as ten days. It’s good because it’s pre-made and you can go to conferences where you get to play some of the games, so that’s a good place to start. As far as some of the smaller ones, I think a safe and fun place to begin if you’re intrigued by this idea would be the dialogue assignments or the dialogue tests, sort of like I alluded to earlier, inviting students to put authors in conversation with each other… maybe across historical moments… maybe across religious traditions, in my case… maybe inserting themselves as a student into the conversation. And why is this valuable? Well, because when we want academic writing to happen, ideally students are putting different ideas “in conversation with one another”—juxtaposing different ideas—and so with these dialogue tests they were like literally doing that in a dialogue format as opposed to just writing a traditional paper where they may not be so aware that that’s what they’re doing. So… something very meta about all of this role-play stuff where they are with me—with the professor—the students are aware that they are trying on a different voice and that often for students makes something click. This is a different way of engaging. By the end of going through the process they’re like, “Oh, yeah, like I’ve made these discoveries that I didn’t think I would have permission to make otherwise.” Again, it gives them a permission to see and do something differently.

John: Could you tell us a little bit about the two reacting to the past scenarios that you’re running… in terms of what the main issues are that the students will be addressing?

Jill: The first one is the Anne Hutchinson game, which Anne Hutchinson was a Puritan woman… 1630s… Massachusetts Bay Colony… shows up… John Winthrop is the governor and they’re there to be the city on a hill to show the world this is what a true God-dedicated colony should look like; they’re gonna turn the eyes of the world to them and everything’s going to flow smoothly, and yet things start happening and people start getting religious ideas that aren’t quite in line with the orthodox and Anne Hutchinson is one of them… and she’s this woman… she’s a midwife… she starts having prayer meetings in her own home and, come to find out, that she’s having these visions and hearing these voices coming from the Bible and she believes God is speaking to her. And so doing she’s shifting the theology of the colony and people are starting to listen to her—that makes her dangerous. So this whole trial happened in the 1630s in Puritan, New England, and she is ultimately kicked out of the colony. The game scenario that we play in Reacting to the Past—again, I did not write this; I borrow it, I adapt it—much credit to the authors of the game. This game’s been running for many years. What happens is it’s this counterfactual where we imagine that there’s a second trial; she’s been banished but we’re gonna give her another try. What happens is you have the faction that is against Anne Hutchinson and then you have the faction that is Pro-Anne Hutchinson and then you have these indeterminate—this is a pretty standard format for reacting: you get one side, another side, and these indeterminates. The indeterminates are the ones you need to persuade; basically we have about three or four days of debate among these different groups trying to figure out what did she do wrong and do we want to let her back in, and the indeterminates are these immigrants who are arriving from England who want to get in the church. Before they can vote on Anne Hutchinson, they have to get into the church. So, they have their own objective of getting in but once they’re in both sides want these new immigrants to vote with them. It becomes this very fun game and what happens—this happened in my class just last month—a lot of students find themselves making arguments that they don’t agree with. They will step outside of class and say to me like, “I do not agree with what I’m saying; I can’t believe I’m arguing that Anne Hutchinson shouldn’t be speaking because she’s a woman. I believe that women have rights and should be able to have religious ideas and speak to men” and I’m like, “Yes, but you’re in a different historical context, so you need to be able to separate yourself from that.” I love—and I don’t say to students—I love that you’re trying to hold in tension what you think and feel with what you say and isn’t that how we sometimes have to act in the world? They did a really lovely job with that this semester. So, that’s the Anne Hutchinson game. The one we’re launching next week is the Frederick Douglass game which takes place in 1845. Abolitionism is really coming to its own as the political force. Slave owners are getting even more anxious and holding on to their power and the country is really in turmoil, specifically around the publication of the new autobiography by a man named Frederick Douglass. In this game there are even more historical characters—we have students who are about to be assigned the roles of John C. Calhoun and Senator Henry Clay and William Lloyd Garrison and Sojourner Truth and Angelina Grimké and they’re gonna go at it around issues of slavery, what the Constitution does and doesn’t allow about slavery and there will be indeterminates as well who can see the wisdom in both sides. They may not like slavery but is it politically advantageous to go against it at this point? This is a controversial game in some context, as you can imagine, because there will be students playing slave owners, there will be students playing former slaves, so you have to tread really carefully. That’s another thing with role-play, depending on your institutional context and who your students are, you do have to be careful as you ask students to step into roles that are not their own identities, but, again, that takes a lot of prep upfront. I think it can be done, you just have to be delicate with it. So with Frederick Douglass, for instance—I mean, who’s going to be John C. Calhoun? He was one of the biggest, baddest, most racist (by our terms) slave owners in the 19th century. Well, I’ve sold him this way to students. They’re gonna get input on the characters and I have an African American male in the class who said, “I want to play that part; that’s a part that I want,” and in fact when I did this last spring I had an African American male who wanted that part. I talked to them and I say, “are you sure? Why do you want this? And that’s awesome.” They say “I want to understand where they’re coming from, because that’s the worst thinking I can imagine and I want to know more about it.” So my student who did it last spring came to class in costume every single day as John C. Calhoun. I have pictures of the students wearing this wig and invariably he also wore some symbol of Africa on his person, like a shirt or a medallion around his neck. It was great what he was doing… how he was showing resistance to these messages that he was speaking in class—I mean, it was actually deeply profound. It also liberated other students to argue in the voices of pro-slavery advocates to have students of color in the class be willing to do that work too, and frequently at the end of classes—again, Frederick Douglass was about six days—like we would stop maybe five minutes early; I don’t know that the Reacting people would approve of us, but we’d stop and say, “Okay, this is getting heavy—how can we support one another around some of these really difficult conversations? How can we continue to support the pro-slavery students in the class…” because what would happen is the indeterminates in the Frederick Douglass game would just be like, “Oh, well slavery is bad; I know it’s bad ‘cause it’s 2018, slavery’s bad”—we just kept having to note that’s not where you are, but the water in which you swim as a mid-19th century fish is one in which slavery’s just accepted. You can’t don your 21st century hat and argue from that way. So, that’s also part of the learning objective. I think it’s probably clear that student collaboration is a big part of this… students having to work together in teams, having to come together around strategies and how to make an argument and who to target on the other team to try to turn their mind around.

Rebecca: Can you talk a little bit about the prep work before any of these role-playing instances? Clearly there needs to be some groundwork laid before you have a whole class period dedicated to any of these activities that you’re talking about.

Jill: Yeah, definitely. Letting them know in advance it’s coming is a big one. With Reacting it’s on the syllabus. I send out the syllabus about two or three weeks before the semester starts and I’m always like, “Hey, this is coming, so please look at this two-page document I’m sending you about what this entails; if this sounds great to you, wonderful; if it sounds miserable, let’s talk, because I don’t want you to be immersed in something that’s really unpleasant.” With the symposia that we do that’s a heads up in advance on the syllabus and then a constant reminder going into it—I tend to tell them within the days before to start making notes as you’re reading—read with the intent of thinking about how you would talk as this theorist. How would you channel these ideas? …and I also always for the symposia start class, ‘cause usually I can’t just have like one student as one person because they’re too many students, so I usually have two teams, like so there’s the Adorno team, whatever. What then happens is I give them about ten minutes to start, just to work together, to come, to brainstorm some ideas and maybe pull some quotes from the reading, also to prepare questions—that’s always the other thing, so it’s not just what are you going to say, but what are you going to ask of one another to keep the conversation going. Part of being in discussion is knowing how to ask questions and when to ask questions. Also encouraging them to draw connections between what different groups are saying—I never want the role-play to be an opportunity for every group to grandstand and then pass the torch to another group of grandstanders who aren’t really making connection. How does that work with role-play? That… either I’ve been modeling that all semester or I haven’t, but something I consciously do in class, I try to tell students, you know, when we’re building on each other’s comments let’s not change the subject without trying to bring in what has come before; if you’re going to change the subject, announce it and explain why you’re changing the subject. That’s part of the modeling that I try to do really consciously throughout the semester so that students get in the groove of how to have a conversation and then these other things just sort of kick it up several notches where then I’m more out of the way and they are hopefully building on tools they’ve been given and are ready to run with it.

Rebecca: You mentioned reflections earlier; can you talk about what that reflection process looks like in a little more detail?

Jill: Yeah, for the Reacting games, a whole day of debrief is built in; that’s one of the things that the Reacting people are pretty insistent on and I do not disagree with them. That tends to be a “Hey student, we just did this whole several days of this historical event. We changed history a little bit because in history this would not have happened and this would not have happened, so here’s what really happened and here’s why.” The debrief is very important for connecting back to larger class theme because, again, when you are sort of stepping outside of normal classroom behavior for a while it’s good to remind students as they gently re-enter why you’ve done what you’ve done and how it connects to these larger class themes, and that’s what the debrief is able to do. So, what you called reflection, Rebecca, I think of also as a debriefing. I also always have students write about these experiences. With Reacting they write a couple paragraphs reflection; I give them some very directed questions. After we did Anne Hutchinson, for instance, in my 101 class, I said, “What did you do well and what are you going to do better next time? Because we have Frederick Douglass coming up in a month.” …and then I share those with the students, like, “Okay, your classmates are really proud of how you all did this and here are the requests that people have for the class going forward. Many of you would like your classmates to prepare better; many of you would like your classmates to show up on time so you’re ready to give your speech; many of you would like your classmates to put more effort into writing your speeches and then delivering them with more confidence and poise.” In this way I don’t become that naggy teacher saying, “Okay, remember, we watched videos on public speaking before you delivered your speeches, but you are still not standing with much confidence and you are still reading from your paper.” Then it becomes the students doing it. Again, it goes back to John’s point before about peer instruction—the self or peer critique—and students don’t want to look foolish in front of their peers, so that sort of ups the ante there. With, for instance, the be the theologian assignment, and even the dialogue test, I always either give a journal assignment or even at the end of a dialogue test on a test say, “Okay, in three or four sentences what did you learn writing this as a dialogue that’s different from writing it as an essay? Or you’ve just performed the role in class of Mary Daly in feminist theology. What did you learn from being Mary Daly that’s different from you as Caitlyn talking about Mary Daly.” So, I think reflection is always such an important part of putting the lid on the assignment, really making it a full, complete thing so that it’s not that weird thing we did once in class but something that “Oh, like giving them the opportunity to make their own connections.” That’s why creating conditions for them to make discoveries for themselves and the reflection is sort of the last chance to do that and I don’t squander that opportunity. So, I think asking those questions and giving them space to reflect is really key.

John: There’s a lot of research that certainly supports that. Sounds like a great collection of activities. You mentioned of some concerns that students have. But, in general, how have students responded to these activities?

Jill: Yeah, a range of things. Some students, I will admit, seem confused. I’m thinking about the last time I did the be the theologian activity—I would say like the first month of class students were like, “What are we doing? Why are we doing this?” And I was trying to be patient and then I’m like, “What have I not been clear about?” And at some point it clicked and it seems to happen with role-play: at some point it clicked and it usually comes with one or two students and then like a lightbulb goes off and they get it and then everyone starts to get it. So I will say for anybody who’s thinking about these things or any creative pedagogy really from my experience: do it more than once, because the first time might not work, but that doesn’t mean that the pedagogy is not right; it might just mean that students are gonna need a little more time. Some students really thrive in it; they feel—I’ve talked about this earlier—they feel free to do college in a way that they haven’t felt free before and that’s really awesome to see because some of these are students who don’t speak. With Reacting, for instance, sometimes I’ve been in class with these students for a month or two months and suddenly we do this different thing and they just come to life and it’s really exciting. You get to see a different part of their personality. What is also exciting is how they then carry some of what they learned and some of the collaborative work that they did into future things like, “Oh, we really work together on this one game that we did, like maybe we can do our group project together at the end.” They respond really interestingly in that way. What I love is when I see then in their written work going forward how they make allusions to the role-play, even if it’s indirect. They start using some of the language and some of the teaching tools and some of the terms, it’s like they actually got it. So, it’s a real range. I’ve had some of my very best students not love it, but, yeah, I think those are the students who you pull aside and you talk to them about why because you can usually show them why you’re doing this pedagogy and why you’re doing something so different and they tend to have some really interesting ideas too, ultimately, and then they can sometimes help you reframe things. One of the things with this role-play stuff that I’ve been working on the past few years is I try to be creative but also humble. I’m not afraid—I try to not be afraid when students have critiques and suggestions ‘cause often they have some of the best ideas. They’re the ones who are doing it and so I invite them to do it; I think that goes to the reflection part that Rebecca had asked about earlier. Sometimes reflection means how would we do this differently? How can we do this better? …and sometimes that’s not just about students and their peers but also about me and the way the assignment is written.

Rebecca: How have your colleagues responded to what you’ve been doing?

Jill: Oh, good question. I’m fortunate because this year at Guilford we have a Center for Principled Problem Solving and they have faculty fellowships for a year and I was lucky to get one for the 2018-2019 school year focused on this performance and pedagogy stuff and specifically around trying to bring some of these ideas to my faculty colleagues. I should say again, I’m never an evangelist for these kinds of ideas because I think everybody should do them at all; I’m really an evangelist for teachers doing things that they think are cool and might work for their students and, while I’m not trying to force anything on anybody, but I am trying to help some of my colleagues just as they’re helping me to come up with new and creative ways to engage students and engage material and make what we do exciting to us. We’re going through a pretty significant curriculum and schedule revision at Guilford that’s gonna kick off next fall; we’ve got a lot of faculty who are rethinking their courses and course designs and activities and there’s not a small amount of anxiety about this change. So, one of the things I’m saying is, “Hey, this is a good opportunity to do some things that are more experimental and even experiential.” One of the things I did was, with the help of faculty development, brought in a Reacting to the Past Consortium board member who came and did a workshop for faculty development in September, and it was awesome. He was really engaging, gave us a lot to think about, and hopefully he’ll be back in January to do a small Reacting game. Reacting has some micro games that last an hour and a half. I believe some of Guilford faculty are going to go to a regional Reacting workshop in March. So, I’m trying to just invite people in—nobody’s being forced to do anything. I don’t have that kind of power, nor do I want it. I’m just trying to give people some ideas that have worked for me that I think are fun and that students seem to respond to and it helps our students. So, Guilford student population… we have traditional age students. we have very diverse, like ethnically… racially… in terms of class… we have a lot of diversity. We also have an adult population and then we have some high school students that take college students at Guilford in one of the best high schools in the state of North Carolina, so we have so much diversity, so how can we reach everybody? How can we invite everybody to the conversation? And this is one way that’s gotten people to sort of break down their walls. I think my colleagues are—some of them are suspicious and they should be—nobody should listen to this and be like, “Oh, this is brilliant, perfect, like, no, it’s not perfect.” Reacting to the Past is well-established, it’s not perfect. Some of my ideas aren’t perfect, but it’s a starting point and we can keep honing and keep working together to fix some of these ideas and that’s certainly what I’m doing. A lot of this work started with a fellowship I had a couple summers ago with the Wabash Center for Teaching and Learning in Religion and Theology; I got to spend the summer researching performance and pedagogy and that’s where I started to develop a lot of these ideas and some of the folks that run the teaching journal in conjunction with the Wabash Center, it’s called Teaching Theology and Religion—TTR. They’re excited about this; they think it’s worth hearing about so I’m working on an article with them—I’ve already published a few things smaller with them and I’m gonna work on some bigger pieces. There seems to be enthusiasm because I think we’re at a place where we want students to be engaged. The population of students seems to be changing in terms of their preparation for college, what they find interesting, what they’re willing to sit through in class. So, this is just one of many ways to get students trying a new way of learning. I don’t think if everybody did it that it would be awesome; I think it’s fun because they’re gonna remember in five years, “oh, yeah, in Jill Peterfeso’s class we did that really weird thing. It was weird, but it was also really cool.” I’m alright with that, I’m totally okay with that. Yes, I just say also like “I’m not afraid to be a dork about these things” and I think that’s disarming and students respond to that, ‘cause I’m like, “You guys are gonna get to role-play and I don’t get to play, but I get to watch, and do you want invite your friends?” and there at first they’re like, “no” and then by the end of the class, like we did this in the spring with Anne Hutchinson, I said, “So you want to invite like your professors or some of your friends?” and they’re like, “no, no” and then with Frederick Douglass they’re like, “We could invite everybody, like let’s put a message in the college newsletter,” like they got so into it. So, that’s learning and that self confidence and that’s not being afraid of trying new things and that transition over the course of a semester… something’s going on. I haven’t measured it and assessed it yet—don’t tell the administration—but it’s doing something and they’re learning because I read their reflections and what they come up with is pretty profound.

Rebecca: Sounds pretty incredible.

John: It does. I know in my own class I went from having students write papers to have them do a poster session and I asked if they wanted to invite other people and they were thrilled to have people from the department come in and the Dean came over and visited them and they were so much more excited and engaged about it. Small changes can make a big difference.

Jill: Yeah, I think that’s my thing whether you’re doing my level one immersion, level two, level three—those are just my categories—those small changes can mean a lot ‘cause even a little bit of reframing get students’ brains working differently and gets their hearts engage in different ways, so I totally agree with you, John.

Rebecca: I’m just sitting here contemplating how I can add role-playing into my Three Little Pigs exercise.

Jill: Aren’t you already doing it? [LAUGHTER]

Rebecca: A little bit, but I’m thinking about how maybe the students can do it more. I usually have someone come in and be the client and role-play the client role in my design class but the students are still acting as designers as humans but maybe they need to be characters in the Three Little Pigs or something for my assignment.

John: Actually, I was thinking of that—our second most popular episode has a title “The Three Little Pigs” and I can imagine all these parents playing it for their kids and finding out that it was really an exercise for a design class. [LAUGHTER]

Rebecca: Yeah.”

John: Much of what you’re describing in terms of being in this third party role is exactly the same type of thing, where students are able to see things much more clearly and are able to address issues that they’d be really cautious to approach if they were doing it in their own persona and I can see that connection and the benefits of that.

Rebecca: Somehow it’s just okay to embody that…

Jill: Yeah.

Rebecca: …other that they don’t feel connected to and explore and develop empathy and those sorts of things which is pretty powerful but I think the actual acting it out or writing the dialogues would really strengthen some of the things that I was already doing.

Jill: For sure.

John: How did you prepare to introduce this activity?

Jill: When I had my summer fellowship about performance and pedagogy I spent a lot of time doing research, starting to see who’s doing this and where and I was frustrated by the lack of what I could find in humanities classes or in sort of your more traditional classes—you get a lot of great activities coming out of theater classes or some more of the arts classes, but like high school classes or elementary school, like most of the role-play books I was finding were not geared toward college students doing the material that I wanted to do. So, I think there is room still for exploration and creativity here, that’s what I like about this. Reacting to the Past is certainly a place—Mark Carnes who designed it has written this great book called Minds on Fire, which I would recommend to anybody interested in this. There are other books about reacting that really do some pedagogical analysis of student experience in what’s going on. But, I think then within our individual discipline there’s a lot we can still do and I’m trying to think about that for myself as a religious studies scholar. I think there’s got to be stuff with empathy there and belief—I mean it’s really hard for students to try to understand beliefs of religious groups that they don’t subscribe to. This seems to be a way where they can at least intellectually be trying on beliefs of others just as they would an idea and I think that also shifts the location of some of these ideas where students are I find, “okay, I can understand that someone else may have this idea but to think somebody else may have this belief is like not about the head but the heart.” They’re more uncomfortable with that. So, trying to push those ideas of the heart as they see it up into the head, I think, could be really rich and beneficial for them. I’m sort of just riffing as I’m discovering this year but when we were talking about Puritan Theology… this Anne Hutchinson game… I just kept reminding them Puritans were intellectuals. These are highly educated people, so their beliefs weren’t just of an experience of God—it was well researched and reasoned with their relationship with scripture. So, I think there’s got to be some of that too… to think that where our emotions and our motivations come from connects heart and head; there isn’t some bifurcation of the two. I think that might help us as a society moving forward as we think about where some of our ideas and inspirations come from. I hope that what they take from some of these role-plays they’re able to put in other parts of their lives, that’s really the idea, ‘cause it’s more authentic than a classroom environment, this kind of here, I have some ideas and now I’m gonna improv conversation and ask questions and try not to step on toes… that’s life.

Rebecca: Sounds like a lot of interesting research can come out of what you just said. Sounds like you’ve got lots of plans. [LAUGHTER]

Jill: That’s what I’m thinking through right now. I guess that’s hopefully the next paper for the TTR Journal.

John: We always close with the question, what are you going to do next?

Jill: Next… So, I’m currently designing a new class. I alluded earlier to the new schedule that we’re doing, so we’re going to have three-week classes—some three-week classes, some twelve—and one of my three-week classes is going to be a new class called Religion, Voice and Performance, where I’m gonna use one or two Reacting games… we’ll see… in the service of helping students think through some of the things I was just talking about with empathy, compassion, belief, reason, rationality and relief, discovering voice, whether it’s claiming your own voice while speaking for another through Reacting and role-play or whether it’s trying to figure out who you are. I think that’s another beautiful thing about theater and acting is it invites you to figure out who you are while you are dancing around in somebody else’s shoes—that’s one of the things I’m working on now, which hopefully I’ll get to teach next year. Working on this article for Teaching Theology and Religion and I’m getting ready to keep working on these assignments that I’ve designed from the past and keep making them better. There’s always room to improve them, so those are my three things right now… and helping my faculty colleagues, as they may or may not want to try some of this stuff, so four goals.

Rebecca: It’s really exciting work, I’m glad that you were able to share it with us today.

John: Yes, thank you.

Jill: Thank you; thank you for inviting me.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

54. SOTL

As faculty, we face a tradeoff between spending time on  teaching and on research activities. In this episode, Dr. Regan Gurung joins us to explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.  Regan is the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research.

Show Notes

Show Notes

John: As faculty, we face a tradeoff between spending time on teaching and on research activities. In this episode, we explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.

[MUSIC]

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Regan Gurung, the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research. Welcome.

John: Welcome.

Regan: Thanks a lot, Rebecca and John.

John: Our teas today are…

Rebecca: I’m drinking Prince of Wales today.

Regan: Alright.

John: I’m drinking ginger tea.

Regan: Ooh, now you’re making me want to. [LAUGHTER]

John: We’ve invited you here today to talk about research in the scholarship of teaching and learning, or SOTL. You’ve conducted a lot of research on teaching and learning as well as research within your discipline. In most disciplines there has been an increase in the journals devoted to teaching and learning and an increase in research in teaching and learning, but it hasn’t reached everywhere yet. SOTL research is often not discussed in graduate programs and is sometimes devalued by campus colleagues. Why does that occur?

Regan: So. I think there are multiple reasons why the—and I’m going to start with the devaluing. I think there’s a lot of uncertainty about what it exactly it is, so on one hand, when people say a scholarship of teaching and learning… very often if it’s somebody who hasn’t really read up on it recently the sense is, oh, you know, that’s research on teaching; that’s not as good as your regular research. Now, I think that’s a misperception and once upon a time, and here I mean maybe even 15 years ago, there was some scholarship of teaching and learning that wasn’t done very well and I think people have heard about that in the past and that’s why there’s that knee-jerk reaction. Far too often it’s seen as something where it’s not as rigorous, perhaps, or it’s not done in the same way and most of that is wrong. What I like to tell folks who see that is, if you think the scholarship of teaching and learning is not rigorous, well, you haven’t tried to submit something to a journal recently. I co-edit a journal on the scholarship of teaching and learning in psychology and I can actually see some people submit poor work and I send it right back; I do the classic desk rejection and I say, look, this is just not good enough. So my favorite tip for “How do you write for a scholarship of teaching journal?” is very simple: just like you write anything else. There’s a lot of baggage, but I think that as you alluded to, John, it has changed more recently and I think part of what you notice now or what I’ve been seeing is that this kind of work, this kind of examination is being called different things. For example, a term that I’m hearing more and more often is DBER: disciplinary based educational research. And I’m hearing this come out of medical schools and engineering schools and social work schools and many professional programs where they’re doing DBER, which is essentially what the scholarship of teaching and learning is. So, I think because of that baggage with the term, people are calling it different things but in general the work is getting much more rigorous.

John: Excellent, and if changing the name is sufficient to do that, it’s a valuable step.

Regan: I think that’s why, when I talk about it I like to talk about it as: “Do you want to know if your students are learning? Do you want to know if your teaching is effective?” Well, then you should do some research on it. You can call it what you want. I started really calling it pedagogical research because that’s what it was, but it’s truly a rose by any name.

John: And that’s something that Carl Wieman has emphasized.

Regan: Asolutely, yup.

John: In the sciences, you test hypotheses and there’s no reason we couldn’t do the same thing in our teaching.

Regan: Exactly.

John: And that’s starting to happen, or it’s happening more and more.

Rebecca: In some disciplines, the scholarship of teaching and learning is not accepted as being part of their tenure and promotion file, for example. What would you recommend faculty do in a department like that if they really want to get started in SOTL?

Regan: Well, so, Rebecca, let me take you a half step back.

Rebecca: Yeah.

Regan: When you say “in some disciplines it isn’t as accepted.” What has surprised me is that most disciplines have actually been doing the scholarship of teaching and learning and publishing it for the longest time. I mean, if you take a look at chemistry, it goes back, gosh, seventy years or so. Almost every discipline out there has a journal that publishes the scholarship of teaching and learning, but, and here’s the big but: most of us in our normal training never run into it. So, I’ll take my own case. In psychology, the Teaching of Psychology Journal has been around for 46 years, yet all through grad school, all through my post-doc I never even knew the journal existed. Why? Because the programs that I went through weren’t focused on teaching the individuals—wonderful as they may be—who I worked with didn’t do that kind of work, so they didn’t know about it. So I think that’s a really important fine-tune there: there is a journal in almost every discipline—almost every discipline—for the scholarship of teaching and learning. So, it’s just a question of discovering it… it’s a question of finding it. Now, that said, where can they start? I think I can answer your question from a conceptual level and from a practical level, so I’ll start with the practical. The easiest place to start, there are lots of compilations of how to do it. For example, I think both of you have my website. On my website I have a simple tab called SOTL. On that tab is a list of places to get going, and I’ve organized it so that there’s a brief introduction to SOTL, there are journals, there are resources, there are little handouts. So, if a faculty member has even ten minutes, go to my website, hit SOTL, scroll through. That’s the more practical, that’s the easiest way to get started. From a conceptual standpoint it really starts with the question, what aspect of your teaching or your student learning are you curious about? John, I know you do some work in large-class instruction in economics. Why is this assignment not working? Can I get my students to remember certain concepts better if I change how I present information? It starts with a question. And you don’t have to read anything, you don’t have to look at any manual. If you look at your class and you go, “Hmmm, why isn’t this working, or why isn’t that working?” That’s where it begins, and from there you follow the same route that we always do: go look at what’s been published in it, fine-tune your question, design, think about what do you want to change and so on and so forth. I think it’ll help if I give you my working definition of the scholarship of teaching and learning, and when I think about it I think of SOTL as encompassing those theoretical underpinnings of how we learn. And more specifically, I see it as the intentional and systematic modifications of pedagogy and here’s the important part: the assessment of the resulting changes in learning. So that’s the key: you intentionally, you systematically, modify what you’re doing and then you measure whether it worked or not. That’s it. I could say that nonchalantly. There’s a technique , there’s a robustness to it, but at the heart, where do you start? You start by asking the question.

Rebecca: I think one of the things that I hear you saying is not much different than someone has a really reflective teaching practice—they’re doing it but not in that systematic way?

Regan:Yeah, absolutely right. There’s a term called scholarly teaching and in this kind of literature there’s a distinction made between scholarly teaching and the scholarship of teaching and learning, and all the distinction is is that scholarly teacher is reflecting on their work and then you’re right, you’re absolutely right; making those intentional systemic changes. That’s scholarly teaching. When it becomes the scholarship of teaching and learning is when you present it or you publish it, preferably through peer-reviewed ways, but you’re absolutely right; at the heart of it it’s scholarly teaching. It’s reflective intentional systematic changes.

John: One of the barriers, that people who are considering doing research in the scholarship of teaching and learning, is going through IRB approval, and in many disciplines that’s something they haven’t experienced before. It’s common in psychology. It’s less common in economics and perhaps in art.

Rebecca: It doesn’t exist in design. [LAUGHTER]

John: Could you tell us a little bit about that process?

Regan: Sure. Every university has an institutional review board and essentially what that board does is it’s in place to make sure that any research that’s being done isn’t harmful. Now, normally when we think about harmful we think about a drug or a food substance being tested, but here it just means any research that’s being done, and so when you do the scholarship of teaching and learning or when you’re examining your classes, yes, you could just look at your exams and see if exam scores are changing, but, if you do want to publish that, if you do want to share that, you really should go through institutional review board review. Now, the key thing here: it does sound like this whole new world, and it is, but at the heart of it is a very simple process. Now, there are three levels of review and I think knowing about the levels helps. For example, the first level is called an exempt review. The next level is called an expedited review, and the third level is called a full board review. I don’t think I’ve run into scholarship of teaching and learning that has gone through a full board review, because we’re not doing things that are more than minimum level of stress. Now when you say, hey, hang on, I didn’t know they were stress involved. Well, anytime you ask anybody to fill out a survey, there’s a minimal level of stress. And when you’re asking your students to reflect on their learning, well that’s a minimum level of stress. Every university has its own procedure. SUNY Oswego probably has a forum online. It’s a short forum; you’re basically telling this board what you plan on doing, what you plan on doing with the information, and most importantly, in these kind of cases, you are letting the board know whether or not students will be put under duress. What the IRB is going to look for is are you the instructor in some way forcing your students to do things that normally wouldn’t be done in the normal course of the educational process. But at the heart of it, all you’re doing is you’re sharing with this board whether or not you can do it and most scholarship of teaching and learning is at that exempt level. That exempt level essentially translates to exempt from further review. It doesn’t mean exempt from being reviewed; it just means this is mundane and low stress enough that it’s exempt from further review. Now that second level, expedited. If you do want to measure or keep track of names, if you want to look at how certain names relate to scores down the line—and that’s actually some really key research—well that’s expedited review. Now, even there it’s reviewed by one person. Both the expedited and the exempt review are reviewed by one person, often the chair. It often takes no longer than a week, and by doing that you just know that all your t’s are crossed and your i’s are dotted and it’s the ethical thing to do. So, whenever people say: “Oh, this is really mundane and I’m not really doing much more than just measuring student learning,” I still sa y if there’s any chance you want to present it or publish it make sure you go through the IRB.

John: And many journals will require evidence of completion of the IRB process.

Regan: Oh, absolutely. The moment you want to publish it you have to sign off saying that you got IRB review..

John: We do use an expedited review process on our campus. I was going to say, though, that we’re recording this a bit early because we’ve recorded a few things in advance, so we’re recording this in late October, but just yesterday I read that Rice University has introduced a streamlined expedited review process or IRB and apparently that’s something that’s been happening at more and more campuses. Are you familiar with that?

Regan: You know, not as much, because right now there’s so much up in the air with the IRB because national guidelines are changing. They were supposed to have changed in January, then it was moved to July. The latest I heard is it’s moved to next January. So, for the most part actual regulations are changing. Even on our own campus we switch from one form of human subjects training to another form, but this so called short-form expedited process will definitely help. That said, even the regular expedited, it’s a very easy process and I think the neat thing about this—and I tell students this when I’m teaching research methods, too—as the instructor or the researcher, just going through that IRB form really reminds you of some key things that you may have otherwise forgotten about, so, yes.

Rebecca: Do you talk a little bit about your own research to give people an overview of what project might look like from the beginning to the end?

Regan: Sure. What really got me interested in this is I teach large introductory psychology classes, the class is 250 individuals and I was struck by how when publisher reps come into my office and try to convince me to adopt one book over the other they would talk about the pedagogical aids in the textbook; “oh, look, our book has this and our book has that.” And that really got me started studying textbooks and how students use textbooks. So the umbrella under which I do research is student studying: What’s the optimal way for students to study? …and I use both a social psychology and a cognitive psychology lens or approach to it and it really started with looking at how they use textbook pedagogical aids. So, for example, in one of my really first studies I measured which of the different aids in a textbook the student uses and then I used their usage to predict their exam scores. Now, what I found, and this is what really surprised me and got me doing this even more, is that even those students were using and focusing on key terms a lot. Now, mind you, I’ll take a half step back—you may not be surprised to know that students use bold terms, they use italics, that’s what they focus a lot on. But students in my study also said that they use key terms a lot. Now if you’re studying key terms that should be good. If you’re making flashcards and studying those key terms that should be good, but what I found is that the more students use key terms the worse their exam scores. There was this negative correlation and that’s completely counterintuitive. Why would they go the opposite direction? So, I dug into it some more and I realized that students spend so much time on key terms or so much time on flashcards that they’re not studying in any other way. So even though they’re using flashcards, they’re so intent on memorizing and surface-level processing that they’re not doing deeper level processing. So, that was some years ago and I’ve been building on that, trying to unpack how students study. My most recent study… that’s actually under review right now… a colleague, Kate Burns, and I took two of the most recommended cognitive psychology study techniques, which is repeated practice or testing yourself frequently and spacing out your practice or spacing out your studying, and we took both of these and across nine different campuses divided up classes such that the students in those classes were either using high or low levels of each of these. So, in one study across multiple campuses we tested is there a main effect of one of these types of studying or is there an interaction? And what we found is that there is an interaction and the critical component seems to be spacing out your studying. Not so much even repeating your studying, but really spacing out your studying, and I think what’s interesting here is the reason this is happening is the students who said that they were testing themselves repeatedly, that sounds great, and if you’re a cognitive psychologist you say, hey, the lab says repeat testing is great; the problem is in the classroom a lot of students who were repeatedly testing themselves were repeatedly testing themselves during a really short period of time.

John: Right, I’ve seen that myself.

Regan: And I think that’s the issue, but because we had both these factors in the study, we could actually tease that out. So that’s the kind of work that I do… is take a look at what the cognitive lab says is important; let’s see how it works in the actual classroom.

John: Now was this a controlled experiment? Or was this based on the students’ behavior?

Regan: So, yes and no, okay. [LAUGHTER] I love this study because of a number of reasons. Number one, we tested two different techniques in the same thing. Number two, we did it at multiple institutions, so it’s not just my classroom. A lot of SOTL is one class. So, here we went beyond to try and generalize. But, to get to your question, we actually used a true experimental design. So we recruited these different campuses and we assigned a classroom. So, for example, I’d say, “Hey John, thanks for taking part. If you can have your students do high repetition and high spacing?” “Hey Rebecca, thanks for taking part. Could you have your students do high repetition and low spacing?” And that’s how we spread it out. We had about two campuses in each of these cells. That’s the true experiment on paper. To get to the other part of what you said… in reality, that’s not exactly what students always did. And you know students; we can tell them to do something but a whole bunch of things gets in the way. Fortunately, of course, we measured self reports of what students said they actually did and it was relatively close to the study cells, but even though it varied a little bit we could still control for it. So, yes, it was close to a controlled study as much as you could control something in the real world across nine campuses.

John: That brings us to the general question of how you construct controls. Suppose that you make a change in your class; how do you get the counterfactual?

Regan: Right.

John: What would be some examples for people designing an experiment?

Regan: The word control, especially in research, has the true connotation of the word control group and that’s controlling for factors as different from having a control group. Optimally we’d love a control group. The problem with the control group is that it means no treatment. So, very often a true control group means this group of students is not getting something. From a philosophical and an ethical standpoint, I don’t like the notion of one group not getting something. So, the word I like to use is comparison group. So, your question still holds, but what’s the comparison group? I think here’s where if you’re fortunate enough to teach multiple sections, well one of the sections can be the comparison group. If you’re not fortunate enough to have multiple sections, you compare the students this semester with the students the last semester when you weren’t doing that new, funky innovation. So, there are a bunch of different ways to gather the comparison group, but you’re absolutely right: having a comparison group is important. Most commonly in scholarship of teaching and learning, the comparison is the students before that intervention, so it’s a classic pre- and post- measure. I’ll give you this quiz before I’ve introduced the material, I give you an equivalent quiz after, let’s see if there are changes in learning. And that’s the most common comparison; you’re comparing them with them before but optimally again you want a different section, you want a group of students, a different semester, or so on, and so on.

John: And it’s best if you have some other controls…

Regan: Absolutely.

John: for student ability and characteristics.

Regan: You nailed one of the key—my two favorite are effort and ability. As much as possible, measure their GPA. If they’re first-year students, measure their high school ACT scores or their high school GPA and then you have to measure ability, and I think those two are probably the usual suspects for control. And again, a lot of SOTL doesn’t do that and it should.

Rebecca: I think one thing that comes up a lot for me (and maybe some others who are in disciplines maybe more similar to my own) is that the kind of research that we do is not this kind of research generally, but we’re really interested in what’s happening in our classrooms. So, for faculty who might be in the arts or some other area where we’re doing really different kinds of research, how would you recommend being able to partner or do this kind of work without that background?

Regan: And I think implicit in your question is the “Do I need to have a certain methodological tool bag?” and I remember I was at a conference once and somebody accosted me and said “Hey, is it true that you have to be a social scientist to do this work?” And the answer is no, and I wrote a pretty funky essay called “Get Foxy,” which is how social scientists can benefit from the methodologies of the humanists and vice versa. But, you’re right; you can collaborate if you need to do that kind of work, but there are a lot of questions even within your discipline… and when I think about SOTL I think about answering questions about teaching and learning with the tools of your discipline. Now, I’ll give you an example: a good friend of mine was an art and her project, or something that she wanted to dig into, was to improve student critiques in an art class. Here we have students learning how to do art (and I think it was drawing or jewelry making) and across the course of the semester everybody had to present their work and then critique each other’s work… and those critiques, they just didn’t have the teeth that she wanted them to, so she was giving them skills and how to do it. So here’s a case of how did she know whether or not the critiquing tools were increasing? Well, she came up with a simple rubric and to score them against and look at if the scores changed. Now, you may say, well, we very often in the arts and theater you don’t get skills to do that, which is true, but that’s where I think collaboration comes in and that’s why what’s really neat about scholarship of teaching and learning is very often there are class collaborations. I have a historian on my campus who wanted to change the quality of his essays and he and David Voelker changed how he was teaching and wanted to see it roll out and had students on their essays use teams in a different way. Well, he compared, and John this goes back to your point, he compared essays from before the change with essays from after the change, counted up the number of teams students had and then, Rebecca, to your point went over to my colleague in psychology and said, hey, can you tell me if this is statistically different. So, he didn’t even bother with doing the stats; he just said, “Hey look, I don’t need to do the stats.” But you can, in a click, and literally within minutes my colleague in psychology had done the stats for him. I think that’s the kind of stuff that can happen to truly get at those answers if you go, “You know, I don’t know how to do that.” But, you’d be surprised… the basic skills for SOTL can give you enough to test questions pretty well.

Rebecca: I think John and I have also found in the teaching center that it’s really exciting when faculty from different disciplines start talking about their research when they’re looking at learning because there’s things that we can learn from each other and the more that we’re talking across disciplines can be really valuable as well.

Regan: Right, and I think this is where reading the rich literature that exists in your discipline or even across disciplines on scholarship on teaching and learning really gives you the leg up, because I find now when I do workshops and somebody says, “You know, I’ve got this question; I don’t know how to start.” More often than not it’ll remind me of a study that I can say, hey, here’s what you can do. And it’s just because I read a lot and I’ve got all that in my head and I just matched to that question and it’s pretty easy. I mean, very rarely do we have to invent something from scratch. We go, “Hey, yeah, you know what? Here’s the study that’s pretty close to the question you have, let’s use that methodology.”

Rebecca: So, how do we build a culture of the scholarship of teaching and learning—the departments who might have faculty who are resistant to the idea of their colleagues spending their time doing that? How do we start changing minds and really building a culture that embraces the idea of the scholarship of teaching and learning?

Regan: Well, I think you’ve got to attack it from two different levels. You definitely want a champion in the administration who is educated enough about the scholarship of teaching and learning and how it can be done robustly. If you can convince somebody of it’s worth and then if you go “How do you do that?” …well that’s where you need to make sure you have at your fingertips, as a teaching and learning center, the exemplars of really robust work… and I think if you have that really robust work at your fingertips, that’s definitely a key place to start. One of my favorite examples along those lines of trying to convince (especially administrators) about the worth of scholarship of teaching and learning, I recommend a 2011 publication by Hutchings, Huber, and Ciccone, it’s called A Scholarship of Teaching and Learning Reconsidered and this 2011 publication is a great collection. It does your homework for you. That one book pulls together evidence for why scholarship of teaching and learning helps students, helps faculty, helps institutions. So that’s where the top down—get your administrators to check that book out and go, “Oh yeah, look, there is actually some good research.” Coming at it from the other angle—I know this for a fact—there are people on your campus doing some of that work, but often they may be isolated, they may be a small group. You want to strengthen them so that they can spread that to their circles, and that’s really how it starts. On my campus, when Scott was the Dean at Green Bay, we did a lot to develop scholarship of teaching and learning through the teaching center. There was one year where we had 14 faculty who got together every month and talked about their projects. Now you may say, well, that’s 14 and you had 160 faculty. You know what, you do 10 of working every year and colleagues see the value of the work those 10 or 14 are doing, pretty soon you’re gonna have a culture where people recognize it more and appreciate it more. So I think that’s how it goes… you put your efforts on those people who are already doing it to make them stronger and that’s gonna spill over and pretty soon you’re gonna win over folks.

John: We generally had support from the upper administration and there’s often been a lot of faculty who are new, interested in doing it; it’s usually the promotions and tenure committees that have served as a barrier in some departments, but we’ll work on that and we need to keep working on that.

Regan: Well, just along those lines on our campus we felt so strongly about the scholarship of teaching and learning that the Faculty Senate actually passed a resolution recognizing the importance of scholarship of teaching and learning. Now again, it still gave department chairs some leeway, but at least the faculty voted on it as something that the university values and that goes a really long way to having especially junior faculty say, you know, I can do this.

Rebecca: Certainly makes faculty, especially junior faculty, feel supported when the Senate is saying, “Yes, we believe in this” and it’s not just one person saying we don’t.

Regan: Absolutely. And they’ll be naysayers. We started off this conversation with “There are people out there who think it’s not good enough” and there are people out there but I’ve had conversations with such people on my campus where sharing some information, sharing things about how it’s done goes a long way towards changing minds.

John: In my department, it’s helped that I’ve been the chair of our search committee for a few decades now. We’ve generally hired people who are interested in this, but that’s not the case in all of our departments yet, but we’re hoping that’ll change. For those who have small classes or may not be interested in doing research in their own classes, one other option is meta-analysis. Could you talk a little bit about that?

Regan: So meta-analysis, where one study is taking a look at a lot of different studies, there is the mother of all meta analyses… is one that we should talk about because I think the interested person can run to it. John Hattie, now at the University of Melbourne, did a meta-analysis where actually he did a meta-meta-analysis; took 900 meta analyses and then synthesized the data from those 900 studies that had already synthesized data, and the reason I like talking about that is the sample size when you take all those 900 meta analyses is a quarter of a billion with a “b”; that’s a lot of data points, it’s a lot of students. And what’s neat about meta analyses is that instead of just being one study at one place it’s now multiple studies over multiple contexts, and if you can find an effect over multiple contexts, that’s really saying something because a lot of single studies are so geared into the local context of where that place is that if you run into a meta analysis, so even if anybody listening pulls up an educational journal or an SOTL journal and sees meta analysis in the title, I would spend more time reading that one because it’s gonna be more likely to generalize from that. So, I think it’s statistical and methodological advances now mean that there are more meta analyses around and more meta, meta analyses around as well.

Rebecca: As an advocate for the scholarship of teaching and learning, where do you hope the scholarship of teaching and learning goes in the next five years?

Regan: Honestly, I think it should be a part of every teacher’s repertoire. When I think about a model teacher, and it’s not just when I think about it—I’ve published on evidence-based college and university teaching and when my co-authors and I looked at all the evidence out there and what makes a successful university teacher… one of those components, and we found six… I mean, it wasn’t just student evaluations, no, it was your syllabi, it was your course design, but one big element was doing the scholarship of teaching and learning… and to answer your question, I think if in five years from now we can see it be part of teacher training to look at your class with that intentional systematic lens, I think that’s where the field needs to get to.

John: At the very least it would get people to start considering evidence-based teaching practices instead of just replicating whatever was done to them in graduate school.

Regan: Absolutely. People would be surprised at how much good SOTL there is out there, and I always like sending folks to the Kennesaw State Center for Teaching and Learning where they have a list of journals in SOTL in essentially every field. You will scroll through that list for ages and it is just mind-boggling to realize that, “Wow, SOTL has been going on for a very long time.” And Rebecca, you mentioned art and performance arts and theater and music… not as much, but even there there is a fair amount and I think it’s just a question of getting folks making those resources more available to individuals and that’s why whenever I interact with teaching and learning centers I have a short list of key resources to look at. And again, that’s on my SOTL link. But, even that small list is an eye-opener to most people who never knew this existed, and I think once they realize it’s there they will start seeing it everywhere and once you start doing it it really energizes you. For those of us who’ve been teaching for 20-plus years to look at our classes with that new eye of how can I change something, how can I make it better and then seeing the positive effects of those changes, that’s invigorating.

Rebecca: I’m energized after having this conversation.

Regan: It is good stuff.

Rebecca: Yeah.

Regan: I just got back from a three-day conference and all we did was sit around and talk about cool SOTL. And you’re right …came back and sitting on the plane I was texting people with study ideas to collaborate on. It was that exciting.

Rebecca: The more you talk… collaborate… the more it happens.

Regan: There you go.

Rebecca: So, we always wrap up by asking, what’s next?

Regan: You know, I think I like getting the bang for my buck and you mentioned this in the intro: right now I’m working on the American Psych Association’s Introductory Psychology Initiative and what’s next is basically two years of really focusing on the introductory psychology course. It’s taken by close to a 1.5 million students a year and I’d like to make sure we can make that course the best learning experience for our students as possible, so that’s where my energy is gonna be for the next little bit.

John: That’s a big task and a very useful one.

Rebecca: And definitely worthwhile. Well, thank you so much for spending some time with us this afternoon. it’s been eye-opening and exciting… energizing. I can’t wait to look through some of the resources.

Regan: You know, is there anything else that you’d like, get in touch and I welcome anybody listening to get in touch as well.

John: Thank you, and we’ll share links to the resources you mentioned in the show notes.

Regan: Sounds good.

[MUSIC]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.

[MUSIC]

49. Closing the performance gap

Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, Dr. Angela Bauer, the chair of the Biology Department at High Point University, joins us to discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

Show Notes

  • “Success for all Students: TOSS workshops” – Inside UW-Green Bay News (This includes a short video clip in which Dr. Bauer describes TOSS workshops)
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc.
  • Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative learning techniques: A handbook for college faculty. John Wiley & Sons.
  • Life Sciences Education
  • Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.
  • Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6), 613.
  • The Teaching Lab Podcast – Angela Bauer’s new podcast series. (Coming soon to iTunes and other podcast services)

Transcript

Coming Soon!

48. The Culture of EdTech

As faculty, we engage with education technology as it relates to our classes but rarely consider the larger EdTech ecosystem. Dr. Rolin Moe,  the director of Academic Innovation and an Assistant Professor at Seattle Pacific University, joins us to discuss the politics, economics, and culture of EdTech.

Show Notes

Transcript

John: As faculty, we engage with education technology as it relates to our classes, but rarely consider the larger EdTech ecosystem. In this episode we examine the politics, economics and culture of EdTech.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Our guest today is Dr. Rolin Moe, the Director of Academic Innovation and an Assistant Professor at Seattle Pacific University. Welcome, Rolin.

Rolin: Thanks for having me.

John: We’re glad to have you here. Our teas today are…Rolin, are you drinking tea?

Rolin: I am John.

Rebecca: Yes!

Rolin: I am having the Maui Up Country blend that I picked up on a on a vacation that I had brought for the office, and we ran out. So I am drinking the wonderful Keurig inspired Celestial green tea today. But I am joining you guys over there. [LAUGHTER] What are you guys having?

Rebecca: I think it’s a green tea day. I’m having black raspberry green tea.

John: …and I have a ginger peach green tea.

Rolin: Excellent.

Rebecca: We’re all in sync without planning, so that’s nice.

John: We invited you here to talk a little bit about your April 2017 EDUCAUSE Review article (which has created a little bit of a stir) where you were talking about the growth of educational technology in higher ed. What types of EdTech in particular were you talking about?

Rolin: So, John that’s a good question… and a little bit of preface on the article itself. I wrote that with George Veletsianos, who is Canada Research Chair in Innovation and a Professor of Education and Innovation at Royal Roads University in British Columbia. We started this project in 2013 at a time when MOOCs had just come into conceptualization. Laura Pappano noted that the year before had been the “Year of the MOOC.” John Hennessy at Stanford said that the MOOCs were going to be a “tsunami that was going to wash away higher education as we knew it.” Clay Shirky compared higher education to a rotting tree that was in need of a lightning strike and this was going to change it… and so, this very optimistic (to the point of Pollyanna) thought on educational technology. And George and I both, as people who are scholars and practitioners in educational technology, were a little taken aback by this. The promises that were being related to educational technology didn’t match the literature. The history of educational technology didn’t match the present and the future track of these innovations, based on their previous experiences (kind of Silicon Valley startups) was not a positive one. As I mentioned, we started writing this in 2013 and the landscape kept changing. Ownership would change, or business models would pivot, and we had to rethink what we were doing. So we kind of, instead, came back to this more systematic review of what is educational technology, or EdTech, and we thought of it in socio-cultural terms as a phenomenon. So, thinking about that, it’s not necessarily a product that we are providing critique for but it’s more of the idea that by bringing products in, whether they be cloud based softwares, learning management systems, apps, learning technologies interoperability, or LTI, or outsourcing it to a third-party vendor, whatever that vendor may be. That approach cannot be thought of as altruistic in and of itself, but it is built in society that is usually, at best, tangential to education, but often completely separate… being brought in for profit bearing reasons, whereas our institutions, by and large, are education-bearing institutions that are looking to gain enough profit to continue operation. So, what EdTech are we looking at? We really want to be creating a more critical consumer of all EdTech. And you can definitely see that today in privacy issues that are coming out with Facebook and algorithmic issues that are happening with Twitter, and discussions of what constitutes free speech or hate speech on these platforms. When we wrote, we were much more thinking about the technology that’s getting into schools, but even there, some of the things that are happening in K-12: the data from these students is not necessarily protected, whether it’s getting hacked and sold to other places or if the companies themselves have connections to other products and other vendors. So, it’s a really meta piece to be thinking about. I don’t necessarily have an axe to grind with any particular software. That’s why we were very software agnostic when we were writing the piece. We just really want to be much more conscious of how we’re using technology in our teaching practice and what is happening because of the technology we’re bringing into our classrooms.

Rebecca: Thanks for laying down that groundwork. I think that foundation is gonna give us a good ground for discussion today and will help our listeners know exactly where we’re starting.

John: A lot of these things, where people were really optimistic about the introduction of MOOCs and so forth, we’ve seen all this before. Television was going to do the same thing. Before that radio was, if we go back further, printed books were going to have this big impact. So, these are issues that have been around for a long time. But, you focus on several issues that, perhaps, are more pressing now. One of the things you talked a little bit about is how colleges have been pressured by economic circumstances, by rising tuition costs and pressure to keep costs lower, to rely more on these external vendors. Could you talk a little bit more about that aspect?

Rolin: Absolutely. I need to preface here again, John, I appreciate you bringing up television. Because there’s a time that a lot of institutions invested in broadcast studios, with the idea being that we were going to be able to amplify education and we’re going to be able to have closed-circuit educational opportunities at senior centers and satellite campuses. And so you have in many land-grant colleges these forgotten studios, that in some cases are now being turned into teaching and learning centers where you have a green screen and you can show what you’re doing in Canvas, or Desire to Learn, or Blackboard, or whatever the system is that you may use… Moodle, I don’t want to leave anybody out. But, to think about my experience as an educator, I have a connection to this particular podcast. I cut my teeth as an educator in my first career, which was in film, at Duke University’s Talent Identification Program where I got to know John Kane, who has been kind of very foundational in how I think about teaching and learning. So John, thank you for that, and it’s wonderful to be on your show. We’ve seen all of this before and we failed to learn our lessons in education. So, we didn’t get out of television what we thought… what we thought we’d get out of radio we didn’t get. It’s important to look back and see “Well, what didn’t happen that we expected to happen? What did we plan for? What was the consequence? What were the unintended benefits? and what were the unintended pitfalls?” The problem, or the big difference today, is a lot of the technology is being looked at from an efficiency standpoint. So, television and radio and even if you go back, like you mentioned with the printed books, you go back to correspondence courses and using the Penny Post in order to be able to give keyboarding instruction for secretarial jobs. So, those technologies were based on much more inclusivity in education. You had a technology that made education available for more, and you had an opportunity to get away from geographic distance as what was keeping people from school. With digital technology what we’re seeing now is almost an inverse relationship that “Yes, we have this opportunity and we sell it.” So, the MOOCs were sold as an opportunity to democratize education for everybody. But, this is really framed in a cost-cutting perspective. That we’re going to bring in technology to keep costs down. That’s very important, costs in education, and higher education especially have skyrocketed, and to think about how we can be looking at this. But it’s disingenuous to say that our digital technologies are going to democratize education for all when we want to use them to save money more so than grant access. We have to look at both critically. We have to put the same research behind both. Moreover, what’s happening when our use of technology is in the gaining of data analytics that could be used, at best, in our spaces, but at worst by third-party vendors that we’ve signed contracts with that we don’t truly understand where they’re going or where they’re taking these things. So, I started with your question and went in a lot of different directions I’m realizing. But, I think it’s important to do that historical review and think about all those places because there’s a desire there, with what education’s supposed to be, if you want to think about Enlightenment-based thinking on education. But, we are at a different point now than we were with what someone like Soren Nipper would have called generations of technology. The first generation being radio, the second television, the third digital. This fourth kind, of web 2.0, has a much greater economic impact, both on the institutions as well as the whole purpose of education. That’s something that we don’t see a lot of in the literature and something that compelled George and I to write this article.

Rebecca: I’m hearing you talk about the the desire for more access but then also these rising costs. If we’re using EdTech, are students actually just getting more access? or are we just making things more expensive at the cost of actual learning?

Rolin: Yes. [LAUGHTER] It’s difficult because in some cases there is an upcharge on taking the course online. And there’s good reason for that because in order to teach a course online, if I’m an administrator, I now have to think about a faculty member who’s going to be working through that course. I have to think about any licensing that I need for contents. So, making sure that my reserves in the library can be easily flown into my LMS and that I have the rights for reproduction in that space. I have to think about instructional design, I have to think about information technology. I have this much larger infrastructure that’s involved, depending on what I’m doing: if I’m going to be using an anti-plagiarism software; if I’m going to be using an online proctoring software, a special grader, a video library of contents. There are four or five different buckets of LTI and those are the general ones, not anything discipline-specific. So, that brings this cost up. At the same time, if you think about Moore’s law, and as technology is increasing and the capacity to do things continues to increase, traditionally we have seen costs go down in this model. That hasn’t happened with education. So you have a space where students are presented in media and, I would say in a lot of cases by schools themselves, that this online efficiency opportunity to engage is going to bring your cost down, but then your cost is becoming more, because the cost on the institution is more. All of that is to say, at some point, if you’re gonna be selling both cost savings and access, that’s not a recipe for success. In many cases, we have the access, but it’s not to people, it’s not to a high impact educational experience that you have come to think with a stereotypical higher education space. I think of the Sally Struthers ITT Tech, you know, where you can do the courses in your pajamas. So we’re giving access in real time to curriculum and to materials, are we necessarily giving it to really engaging learning activities? In some cases, yes… but I don’t think the literature would say that those brightest cases of access are meeting that romanticized version of what it means to be a student in higher education. In many cases the most successful institutions in creating access and bringing costs down are the ones where faculty have been replaced by kind of quasi-administrators who work as admissions support specialists, tutors, retention specialists, program developers, and fundraisers. Kind of doing all of that from an office space, and that looks remarkably different from what we see in cinema, as somebody who works in film studies… what we see in cinema as that college experience. So, we’re gonna have to rectify what it is we think college is supposed to be with what it is we’re selling it as.

John: Might some of that be that, with new technologies… giving an example from economics… when steam engines were first introduced, we didn’t see any real improvements in productivity for decades after that. When the internet was first introduced and people shifted businesses to that, it’s taken decades before we’ve seen much of an increase in productivity. Is part of it that we try to use the new tools in the same way that we traditionally taught and we haven’t learned how to use it more efficiently, or is it something inherent in the shift to more digital media that limits the interaction between the instructor and student and may limit learning somehow.

Rolin: John, thank you for bringing that point up. If you think about professional development technology, the stereotypical overhead projector that is used to present material is then replaced by the PowerPoint…and what was interesting is, in some cases, the first uses of PowerPoint in classrooms (because of bandwidth issues) were printouts of PowerPoint slides that were then put onto overhead transparency. So what we see in many cases today what constitutes online learning is the lecture based approach the “sage-on-the-stage” model of teaching where we’re using our learning management system to do what we’ve traditionally done, and it’s what I would call a mediocre middle. It both misses the point of improving education and also misses the point of utilizing the technology, but it’s what we do. My fear is that there has been a financial success in doing things in this way, or at least creating a media culture that equates formal education to the lecture. So you think about a TED talk, or you think about a Coursera lecture… this idea that it is a faculty members responsibility to share their wisdom as the person who’s speaking through it. A podcast is another space, we’re people who are talking in a space. Now that doesn’t mean there’s not a space for podcasts and there’s not a space for lecture, but it’s easy to package that content and put it into a learning space that you’re hoping to monetize. For learning to be effective online and bring down costs, probably requires a pretty seismic shift in how we think about business as normal. Some of the early critiques of online education were that it would turn us into a fordist space, where it was gonna be the assembly line production. That was gonna get away from a faculty member as kind of an auteur, somebody who has the course from its implementation to its full assessment. With online that’s almost impossible to do for the sanity of anybody. So, in some cases, that model is going to need to change in order to be successful. We haven’t figured out what that looks like yet and the human capital costs of doing it right so far outweigh the benefits that you get from allowing students to be able to take classes from a distance and increasing your enrollment, hopefully through online. We haven’t figured out how to weigh the human labor that goes into that. And I think some of it is also we haven’t changed… I’m gonna get radical here, the expectation of what it means to be a professor is still the same as it was 50, 60 years ago, but what we consider is knowledge has changed pretty significantly with Ernest Boyer’s thoughts on scholarship. What it expects to be a faculty member… so the expectations of teaching at even teaching heavy institutions have gone up but the expectations on scholarship or service have not changed. So instead of it being a triangle of scholarship teaching and service it’s this odd triangle that is morphed into a parallelogram with no extra time given to these spaces. So, we’re gonna have to think about our governance structures and our infrastructure if we’re going to be successful. There’s an article in The New York Times this week we’re recording this in mid-September talking about what the next financial bubble may be, and it points to student loans that the cost of education has gone up fourfold over the last 30 years, outpacing everything, including healthcare… and the student loan debt has over the last five years, overtaking credit card debt. It’s the largest amount of debt that exists in any industry. That cannot keep up. Y et costs continue to rise. So another thing; in the next 7 years, that traditional college age, students 18 to 22, is going to decrease in 2025 because of demographic shifts. So, there’s a lot that’s going on at this point, and John, you mentioned the steam engine and how it took decades… Well, we keep saying we’re the Wild West and we only have years until we get to the cliff, and many people would say we’re already past that point; we’re at the point of no return. I like to be a little more optimistic than that.

Rebecca: I’m gonna go back to a little bit of discussion about access. Some of the things that I hear you describing is that the technology is allowing us to have access to information or the distribution of information. Which is why the lectures, the podcasts, et cetera are easy to package and deliver the access to that information. But, what I’m not hearing is access to learning or the access to becoming a scholar, or a way of thinking or being in the world. And I wonder if some of the movements in OER or the open education resources are trying to push the envelope or push the technology and access more in that direction, or if it’s really still emphasizing the ability to just deliver information.

Rolin: Rebecca, you bring up a really great point. And I’ll touch on OER because it’s a fascinating case study in this space, but if you look at the history of distance education with technology, the focus was on bringing people together… that the content operability was not the key point… but it was being able to bring people from disparate geographies or cultures or climates together to learn. And so it is based in constructivism and constructionism and social learning theory and activity theory and all of the wonderful progressive learning theory that is moving teaching and learning today. And the technology that is predominantly used stands much more didactic, maybe behaviorist, in approach because it’s easier to measure that than it is to measure the much more engaging work that happens when you bring people together. So I had an opportunity (I’ll try and not give away any disclosing information on this), but I had an opportunity to work with a group on a MOOC in after the first wave of MOOCs—this was 2013–2014. They were on a major platform and they had created a course, and it was not a traditional STEM course; this was an arts-based course that they had created. And the platform came to them at the end and said, here’s what happened in your class and had this ream of analytics and they said “Well, wait a second. We had a Facebook group, we had meetups, we had a lot of people create artifacts. Where does that fit into this?” And the platform just kind of shrugged their shoulders and said, I don’t know. We can tell you how long someone watched the video and they were saying, “That’s not what’s important to us. What’s important is what were the conversations that were happening and how is that gonna relate to where they’re going further.” We’re in a time of measurement today, yet our measurement structures are much more basic than our capabilities with technology. And so we’re engineering the technology to perfect those measurement techniques. We can’t do much more with bringing people together and engaging more progressive emergent learning theory with technology. I think what George and I were arguing is the technology, as it stands today, doesn’t feed that because that’s not what’s getting the clicks, that’s not what’s moving the needle, whatever metaphor that you want to use in that space. MOOCs are a fascinating space to look at this because the MOOC acronym actually comes from an experiment in social connectivist learning from 2008 with George Siemens, Stephen Downes, Dave Cormier and the great Canadian contingent. And then Sebastian Thrun didn’t even talk about it when he became the father of the MOOC in 2011. He was looking at a bold experiment in distributed learning at Stanford. It was a New York Times reporter Tamar Lewin who made the link between what George Siemens had done and what Sebastian Thrun had done and called it a MOOC. And it kind of stuck and that’s where we went with that. So it’s very interesting to look at the hype versus the research and why the hype is what’s pushing the cart when in academia we like to say it’s the research that does. Now you mentioned OER. I want to focus on that because this is a really fascinating space that in the last couple years you’ve seen this remarkable push on open educational resources, open textbooks, and I am a longtime advocate of open education… been attending the open ed conferences that David Wiley has been putting on since 2013. I ran the unconference there last year. So I’m advocate for what they’re doing. But it is interesting to think about their success and what their advertising is. Their paramount success is really focused in textbooks. So while you have the opportunity to edit a textbook and you have the opportunity for a faculty member to build artifacts of knowledge with students and cross collaboration, that’s not what’s moving the conversation today. What’s moving the conversation are these static textbooks that bring costs down for students. And I like to be the voice that’s saying, don’t forget about these places, because I worry that we’ll see something, and you can even see a little bit of it happening now with publishers who are wanting to open wash or green wash or astroturf what open is and say, “Oh, you know, here we are over here at Pearson or McGraw Hill and this is our contribution to the space.” When you look under the hood it looks remarkably different, but if the focus remains on this static text book in that adoption, it’s easy for that to co-opt. So, to answer the question in a more broad sense, I think in general we have research that’s telling us one thing and we have marketing and public relations and cultural ideology that’s saying something else. I don’t want to say we’ve done a poor job, but the two are very incongruent right now and usually it’s that media PR machine that’s pushing things and we’re playing catch-up and it’s easy to lose track of the research in that.

Rebecca: As a public institution like we are, obviously access as in all people should have access to the information is really important, but I always get concerned about the people who are generating the technology pushing it in the wrong direction and people who value everybody having education and learning not being able to push the envelope or push the technology in the direction that we want to push it in. They’re kind of butting heads in some ways.

Rolin: I would absolutely agree with that. And accessibility, it’s really wonderful to see accessibility being brought forward in terms not only of contents but also of learners, and so the stigmatism of having learning disability or an emotional or physical or some need to engage with content, that now is going to be supplemented by an institution. And that we are designing with that in mind. We’re designing a universal access and UDL that we’re engaging in this space, and that’s a really wonderful change that has happened in higher education. When people talk about the cost of higher education, it’s important to note that things like that are bringing the cost up, and I don’t think any of us would want to get rid of any of those pieces. The problem, of course, becomes “What is the historical understanding of this place?” and “What is our institutional objective and our institutional memory versus these changes that are happening in how we think about teaching and learning?” And I’ve done as much as I can locally at Seattle Pacific University to start conversations and meet people where they are and I think we’ve had some some pretty remarkable success in rethinking some of our structures, but we’re a private liberal arts institution not dealing with the state bureaucracies, not dealing with a state system, not dealing with tens of thousands of students, and it becomes difficult to navigate all of that. Bureaucracy is the least worst tool that we have in order to work with that. But it’s also a great straw man or easy fall guy for any problems that come up, and too often problems continue to exist rather than being tackled because it’s tough to think about what the benefit would be going forward.

John: In your article, you talked a bit about the increasing reliance on private vendors, outsourcing tasks from institutions to vendors on the grounds that that opens things up to the free market in some way, but when we look at the provision of most of these platforms, it’s a fairly unstable market. We’re seeing so much concentration in the market where many small publishers have disappeared, and many of the innovative educational technology providers have been bought up by other large firms. We’ve seen many providers disappear.

Rolin: I’m glad you bring that up, John, because if you think back… and John, you and I have a background in K-12 and it’s really fascinating to think about this from a K-12 perspective because in the 60s and 70s, the heyday of educational film, it was the job of the media resource specialists at a K-12 library to work with faculty to be able to understand how these pieces fit together, and so they were working with Encyclopedia Britannica and World Book and Disney and ABC and NBC and the different content providers of the time, who were making educational titles. It’s a fascinating, fascinating time. The computing craze in the 1980s came at the same time as a recession. And the idea being for people to think about how this was all going to fit together. When this was created the idea was that that role was going to be vital, and the change that happened was we got rid of the media resource specialist and believed it will be up to institutions and collaborations to grow this, to make this go further. Educational film died because it became less expensive to make it and the belief was more and more people would make it. What we instead make are lectures and YouTube videos, and there’s value to both but the great expectation that we had on what these contents could be is gone and we’ve lost that. And so there’s an opportunity… I think that if you think about learning as this contextualized and locally defined space… there’s an opportunity to be able to create these contents. But there’s a lot of risk that goes into that. There’s a lot of quality control that we we didn’t necessarily expect. And there are a lot of other costs that came in and so we output to these third-party vendors, hoping that we hit pay dirt with somebody. In many cases those companies are folding regularly or they’re being absorbed into others, the learning management system ANGEL, which was a rather popular system in the early 2000s and early part of this decade got bought out by Blackboard, and a lot of the people who liked ANGEL liked it for the reasons that it wasn’t Blackboard. But to think about, in that perspective, it’s almost impossible today for institutions to take this on their own. There’s just not a return on investment that works for that, so that means you either have to create these partnerships across institutions that historically have been at war with one another, or you invest in the promises of third-party vendor, either a small one that’s telling you what you want but may not be around in a month, or a large one that you have a lot of trust issues… best-case scenario, trust issues on the kind of service you’re going to receive; worst-case scenario, what does it look like what’s happening to your data, what’s happening to your analytics, what’s happening to the ownership of what you’re producing.

John: Going back to ANGEL a little bit, we used to use ANGEL here and in many ways I loved it; it had some really nice features that Blackboard is a ways away from getting. It had automated agents and so forth, but ANGEL was actually created at Indiana University. And one of the problems they had was that in the 2007 recession, state support for Indiana University was cut significantly and they owned this big cash cow that they could sell off… and so we lost a fairly viable provider in large part because we see in general a decline in federal and state support for higher ed and it puts institutions in a difficult bind where they often outsource more and more.

Rolin: Absolutely and ANGEL is a good example of that. You can go into the 60s with Plato. It was a Midwest State school that was doing Plato. I think about Quest Atlantis was another great thing that gets mentioned in all sorts of progressive educational research that was funded by grants and the funding dried up and there was no way to sustain it. The MEK Corporation, the people who created Oregon Trail and super munchers and that educational software, where is that today? And I work in educational film, I think about it from that perspective. How have we lost those film providers and now we just think that content will fit in for what was historically this really rich and vibrant place to engage, but we’ve lost it on the software side and the teaching and learning side, and we’re outsourcing so much of what we already do to the free market. Certainly there is benefit to that, but at what cost? And I don’t think there’s been enough analysis of what that cost has been.

Rebecca: So, you’re really bringing up the idea that EdTech is not neutral and that there’s competing goals. So, technology companies are obviously trying to make money and then we’re trying to have students learn, ideally. How do we help those things become more aligned? What needs to happen so that we’re not at odds but that we actually find alignment and essentially make the world better which, in theory or in PR, is what’s being said?

Rolin: I think for the first piece, Rebecca, is understanding that EdTech is not neutral, and once we have that foundation, that we understand what we’re using and what it relates to, we can be much more thoughtful about how we use it. So, I am a faculty member but I am primarily an administrator and I use our learning management system here on campus. I could go off the grid; I could try and do something completely different, but it’s important to show support of what we’re doing with an understanding of how that works, and so we have our LTI, whether it’s anti-plagiarism software or proctoring software and all these pieces, and as a scholar I can have criticism of that. So, as a practitioner, how do I help my students understand what they’re getting into with this and making informed decisions about that space. So, I think it really comes into this idea of understanding the learning environment and what my job is: to control… to create pathways for students to be able to learn and to scaffold that and to fill knowledge gaps and help people expand their zones of proximal development, to go Vygotsky on us. I need to cede some of the “management control” that goes into: “Well, we use this, and this is what’s going on.” But, let people make thoughtful decisions about what’s happening with the technology that they’re using. My son in K-12 can opt out of state standardized testing and that’s a decision that’s made as a family. Dealing with college students, we don’t give them the same rights to opt out of some of the technologies that are being used. So, I think about the proctoring technology that was out of Rutgers that was running in the background on computers using retinal scans to engage people and that’s just what you get when you sign up and there’s no informed decision or consent. There’s not even a Terms of Service that you have to read through and then click a button that you don’t actually end up reading. Can we have more of these conversations? Can we be more informed? Because, if we have that information, we’ll be much more thoughtful in the decisions we make on what vendors we choose. The vendors will then have to respond to that market in making software that is more open or more transparent in its use and the application of its data. People have to make a profit. Education has to make a profit. We can marry those pieces together and have a somewhat vibrant marketplace that is serving the learning of students. I think the issue is, right now for EdTech, the student is the customer, not the buyer, and so there’s a gap there that if we have students much more involved in all aspects of that and involved in those conversations, that becomes part of learning experience. I think that that could see some more direct improvements than just generally saying, “well, we’re thinking about this and we’ll continue to think about this going forward.”

Rebecca: I think one of the things I’m sorry I was gonna pick up on the threat of audience but okay

John: You mentioned keeping students in the zone of proximal development. One concern with standard lecture based teaching is that students are pretty much forced to move along at one pace. What’s your reaction to adaptive learning? Is that something that could help, or are there some limitations that we should be concerned with there?

Rolin: I mentioned Plato earlier, the first personalized learning network—basically adaptive learning. I think that there’s a wonderful opportunity for adaptive learning platforms and for being able to bring in competency-based education into spaces. Thomas Edison University has an amazing program that is built on the idea of competency-based education. Alternative pathways and moving away from “seat time…” there’s definitely viability for that. It just has to be thoughtfully executed… and what is the purpose of the learning that is happening in that space? So, if I think about a School of Health Sciences, I think about nursing… if I’m going to get a degree in nursing, there are really specific things that I need to do. I need to pass very specific exams that are proctored in very specific ways that expect me to maneuver in very specific fashions. The seat time is important for that, and that space there needs to model what I’m going to be getting into in an industry. So, I can’t be an intrepid change agent saying, “No, this needs to be social learning theory,” it needs to be what takes off in nursing. No, nursing students need to be able to be successful in the expectations of their field. There are the places that adaptive learning can fit into that. You see it in foreign language in many cases and the supplements that are happening there. Keyboard instruction is another one where that comes in. So, how could we use the best of that to be getting into other spaces. I think some things that we could explore there, as we rethink disciplines and what works for economics or film studies or education. I think there’s some places with that critical thinking… that soft skills, 21st century learner stuff… where the adaptive learning could come in…. so, misinformation, media literacy, fake news… big hot topic and I wrote an article in 2017 that got a lot of attention (not all positive) saying that fake news wasn’t the problem; it’s not what’s ended up resulting in Brexit or the results of the 2016 election. But it was a small part of a landscape that had been neglected and was suffering from blight for a long time because of how we teach this stuff. And I wonder if thinking about digital literacy, which we’re all expected to incorporate into our classrooms, if that could be served by an adaptive learning platform that engaged content, theory, criticism and evocative video to be able to move somebody on a pathway. That’s a place where all of us could come together because there’s no discipline that owns information literacy. It’s built out of information literacy in libraries. But librarians often are the most flexible in thinking about how their craft is going to change. Places like that, critical thinking, the stuff that we’re all told needs to be imparted to our students, but it’s just kind of this hooray concept of “Oh yeah, let’s have this.” Maybe those are the places to really focus on the successes of that and then the research can help define how economics could engage adaptive learning or film studies or education or cell biology.

Rebecca: One of the things that you said earlier is that students aren’t the audience of or aren’t the buyers of the technology. And I wanted to shift that a little bit to thinking about audience and who things are designed to and I think you’re right in that tech companies are selling to administrators who are the ones that are doing the buying and the purchasing who are trying to facilitate certain things, keep cost down, et cetera. How do we shift that conversation so that tech companies start to see the end users who are really students and faculty as the audience of their marketing, of their conversations, and actually shift things so that they focus on the research around learning and improve learning rather than just facilitating something?

Rolin: The key part of what you said, Rebecca, you kept going back to learning, and I think that’s what’s missing in these vendor conversations. We have this idea of what learning is and if I’m a vendor and have mounds of data I can point to achievement and I can point to the things that I measure in my platform that lead to that achievement, and for most instructors that’s not evidence of learning. That might be a small part of it but there’s a much larger picture. And we do a poor job of amplifying that research. That research doesn’t play well in mainstream media, so how do we do a better job of sharing that research. What constitutes learning? What makes learning happen? I love going to YouTube and looking up “do-it-yourself how to fold a fitted sheet,” ‘cause I don’t do a good job of folding a fitted sheet. And I’ve tried numerous times and I still struggle, so that video isn’t the piece that I need to be able to move me there. Now, there are other pieces, potentially making something for dinner that I would be able to replicate in that space, but replication again is not learning. So, even an understanding of: What is learning? What does that mean? How do we define what it is to have learned something? What it is to be a learner? “Lifelong learner” is a commodified term at this point when it really should be a state of being for, I would say, pretty much anybody. How do we engage those conversations? That’s a really complex question. In terms of an institution, how do we bring more student voices into these spaces? and not in a placating fashion of, “Well, we now have a student sitting on this committee.” But to really understand how that student can canvass and caucus with their peers to be able to provide us information. In the same way that if I’m serving on a faculty committee so that I’m meeting my service requirements, but if I’m getting something out of that and I’m giving back that’s a wonderful experience. A student serving on a committee… how can we provide them what they need for their CV or for their graduation in a way that what we’re asking from them they can provide us? …and not just sitting there and saying we’re listening to what they’re providing but often not doing that. So, more student voices in those decision-making processes… more research that’s going to be shown to the vendors… and I think we need to be more thoughtful about those vendor conversations. One thing we do here at Seattle Pacific University, we actually have… with our faculty… we provide entry points for vendor assessment when we do test demos. What are some of the things that faculty who are very interested in being part of these conversations but are coming in the middle of it… what’s happened so far and what are questions they can ask we’ll be able to draw out their expertise and what we need from the vendor? The more of that that we do, the better. I know the California State University systems doing something similar on automating a great deal of the pre-production that goes into assessing vendors so that the stakeholders who are asked these questions have that information in a repository and can access it very easily to make an informed decision, rather than it being brought down from higher administrators… lots of information that’s tough to digest in a small period of time.

John: What do you see as some of the most promising areas where EdTech has some potential?

Rolin: Excellent. My wife loves to say it’s very easy to show why you’re against something, but you get into this business to be for something. Get in education to really share the diffusion of knowledge and help people rise to heights they didn’t know were possible. Fall in love with things they don’t yet know exist as Dr. Gary Stager would say. So, what are some of the positive things that are happening? I really think there’s a chance for a revolution in multimedia. Here’s this podcast that is happening in an interdisciplinary fashion in SUNY Oswego bringing in a faculty member from a completely different perspective who serves an administrator having this conversation. More and more of this is happening. Before we went on the air, John, you were talking about editing your two channels and making sure the sound was right and all of these skills that were picked up that don’t come when you get your PhD in economics. So, as these pieces are coming in how do we value that and so you see more administrations and more governance bodies that are providing value to that. We were talking, Rebecca, about open education. The University of British Columbia now will recognize the editing of OER materials as part of promotion, tenure, and review for their School of Education. That’s a phenomenal change that has happened in how we think about the role of the faculty member as a distributor and conveyor of knowledge. I think people are being more thoughtful at this in this day and age. But, you did ask specifically about technology, so I need to pivot back there for a second. I love some of the stuff that’s happening in virtual and augmented reality. Some of the really interesting research that’s happening there. I like the drop-in classes that are happening around special interest topics that often, in many cases, are informal or non-formal learning spaces. Museums putting on areas where you can come in and learn in a certain time. Kind of a gap between a human experience and the MOOC but you’re kind of doing both at the same time. I think that the opportunities that we have with free and ubiquitous devices… and I don’t mean free as in cost but I mean free as in access to… and especially in the West with broadband capabilities, what’s going on with video and how we can better engage that and as more people learn about nonlinear editing and cinematography and camera and sound. What are some of the resources we’re gonna build there? Opportunities for students to share their knowledge is the main thing that comes forward for me. WordPress, which runs, what, a quarter of websites in the world is getting incorporated more and more into courses. You think about the WordPress camps. There’s a great thing happening in New York City coming up on managing the web and how you can work with students to be able to be creators and owners of the knowledge that they’ve created and what the implications are in that space. It is kind of a tough time to be bullish on technology if you think about Facebook and Google and Apple and Amazon and antitrust that’s going on in all of those spaces. And so a lot of the stuff that I’ve mentioned here is somewhat renegade, somewhat guerrilla even. So where are those opportunities to engage with environments through online? It comes back to community in that space. How do you find and foster that in your networked identity. There are opportunities and more and more that’s going to be happening. I think that we’re in this storm and after this there will be, not a calm, but there will be an opportunity to look at what’s been broken and how can we build and improve going forward, and I think that we’re getting to that point sooner rather than later.

Rebecca: We generally wrap up by asking what’s next. You talked a little bit about what’s next in EdTech, but what’s next for you?

Rolin: What we’re doing at Seattle Pacific University around academic innovation; we have been offering seed grants to faculty for the innovations that they see as necessary, whether that’s in a classroom, in a department, in a college across the entire campus working out in the community. We provided 45 of those over a two-year period, so almost a third of our faculty directly affected by those and it was very powerful, so we’re taking that a step further and engaging at a school or college level and finding innovations that we can then potentially put into day-to-day operations. So, one of the things we’re thinking about actually are adaptive courses. What would it look like for a course in nonlinear video editing to be almost entirely online. And you think about that with lynda.com. I can go to lynda.com and take a tutorial in using Final Cut or Adobe Premiere. What am I getting out of being in a higher education institution that I can’t get off of Lynda? That’s what we’re exploring: what does it look like to have that scaffolding and support that’s directed toward a greater understanding of knowledge? Other things are definitely around social justice. We are seeing at Seattle Pacific an increase in first-generation and historically underrepresented students who are coming in with the same scores as their peers but, once they get here, we’re seeing a discrepancy between where we would expect them to score and where they are scoring. And we have statistically significant research showing that that is the first-generation student demographic. So, what are some pieces we can put into play to be able to help them with their success? Because it’s not a matter of not being able to do it; it’s a matter of the structure and the culture is not befitting them. So, we have a program called the Bio Core Scholars where we are working with tutoring and mentorship on research, community, and knowledge gaps to be able to move these students. We’re in our fifth year of this program, we’re looking at expanding it. But we have brought the students up a full standard deviation in their scores, and we had an 86 percent success rate in graduating people to pre-professional health programs, which is just a remarkable number. Personally, I’m really big on what we can do with educational video. What are some of the things instead of it just being a lecture? I love Skunk Bear on NPR, taking a topic and in three minutes doing an entertaining, evocative dive into that topic, but again, that’s Oliver Gaycken would call “decontextualized curiosity.” How do we take that and actually put it towards learning? So, I’m looking at what does it look like to have lecture mixed with a very product based assessment mixed with more evocative filmmaking to move people into learning? How does that all go forward? It’s a very exciting time to be in higher education, even with all of the things that are looming on the horizon.

Rebecca: Certainly doesn’t sound like you’re gonna be bored any time soon.

Rolin: Not at all.

John: Thank you for joining us. We look forward to hearing more about this.

Rolin: John, Rebecca, thank you guys for having me.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

45. Opening the STEM Pipeline

Preschool through high school experiences have a direct impact on the majors and disciplines that students want to study and engage with in college. Designing these experiences to invite underrepresented groups into the discipline early can help to inspire and motivate a new generation of professionals. In this episode, Dr. Stacy Klein-Gardner joins us to discuss how engineers are attempting to diversify the field.

Show Notes

Related publications:

  • Parry, EA, PS Lottero-Perdue, SS Klein-Gardner.  Engineering Professional Societies and Pre-university Engineering Education.  In M. deVries, L. Gumaelius, and I.-B Skogh (Eds.) Pre-university Engineering Education.  Rotterdam, Netherlands: Sense Publishers. 2016.
  • Reimers, J. E., Farmer, C. L., & Klein-Gardner, S. S. (2015). An introduction to the standards for preparation and professional development for teachers of engineering. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), 5.
  • Klein-Gardner, S. S., Johnston, M. E., & Benson, L. (2012). Impact of RET teacher-developed curriculum units on classroom experiences for teachers and students. Journal of Pre-College Engineering Education Research (J-PEER), 2(2), 4.
  • Klein-Gardner, SS, ME Johnston, L Benson. Impact of the RET Teacher-Developed Curriculum on their teaching strategies and student motivation.  Journal of Pre-College Engineering Education Research. 2(2):21-35. 2012.
  • Faber, C., Hardin, E., Klein-Gardner, S., & Benson, L. (2014). Development of teachers as scientists in research experiences for teachers programs. Journal of Science Teacher Education, 25(7), 785-806.
  • Mckay, M., Klein-Gardner, S. S., Zook, K. A., Yoder, M., Moskal, B. M., Hacker, M., … & Houchens, B. C. (2011). Best Practices in K-12 and University Partnerships Panel Winners ASEE K-12 and Pre-College Engineering Division. In American Society for Engineering Education. American Society for Engineering Education.

Transcript

Rebecca: Preschool through high school experiences have a direct impact on the majors and disciplines that students want to study and engage with in college. Designing these experiences to invite underrepresented groups into the discipline early can help to inspire and motivate a new generation of professionals. In this episode, we explore how engineers are attempting to diversify the field.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Dr. Stacy Klein-Gardner, the founding director of the Center for STEM Education for Girls, and currently an Adoint Associate Professor of Biomedical Engineering at Vanderbilt University, and a Senior Professional Development Provider with Engineering is Elementary, at the Museum of Science in Boston. She recently was appointed as a Fellow of the American Society for Engineering Education.
Welcome, Stacy.

Stacy: Thank you. I’m delighted to be here.

Rebecca: Welcome. Today our teas are…

Stacy: Well, I have to confess that I don’t care for tea. So, I had some lemonade with lunch and I’m good to go now. [LAUGHTER]

John: I’m drinking Tea Forte black currant tea.

Rebecca: …and I’m having Lady Grey today.

John: We’ve invited you to join us because of your very extensive work in improving educational, P to 12 STEM and STEAM education pathways in many ways. First, though, could you talk a little bit about your own pathway to a career in engineering and engineering education?

Stacy: Sure. I’d be happy to. I grew up in the American South…actually went to junior high and high school in Oxford, Mississippi. I wasn’t always satisfied with my educational opportunities there, so I spent every summer possible at the Duke University talent identification program, or Duke TIP. Which is where I made some wonderful lifelong friends that have influenced my personal life and career since then. I did go to Duke University, where I double majored in biomedical and electrical engineering. I spent my summers working at Duke TIP, really falling in love with education and realizing my passion for that. I did a masters and a PhD in biomedical engineering from Drexel and Vanderbilt University, respectively. Then, I always thought I would retire to teach high school one day and realized that was stupid, and if that’s what I really wanted to do, I should go do it. So, in the same Fall, I defended my dissertation, I started teaching high school full time and fell in love with being in the classroom and working with teachers. Since then, I’ve been a high school teacher, full or part-time, for over 20 years now and I’ve been on the Vanderbilt University faculty since 1999…and I’ve done everything from being Associate Dean to research track professor to adjoint professor now…but really have enjoyed creating my own career in engineering education.

Rebecca: You mentioned being the Associate Dean for Outreach at Vanderbilt School of Engineering. Can you describe what your role was like? I think it’s a little unusual, perhaps, to have an outreach dean so I think it’d be interesting to hear about that.

Stacy: Yeah, the title was definitely unusual at the time. You do find more positions now, often maybe at the assistant level. But, I had a really diverse group of things I was in charge of. I worked with our Career Center on setting up appropriate opportunities for the undergraduate and graduate engineers coming out. I managed a big sponsored lecture we had every year. My favorite part was definitely doing K-12 outreach for the School of Engineering and reaching out to local communities and schools and students. Another favorite part, one that maybe surprised me a bit that I ended up really loving, was study abroad for engineers and finding ways to help engineers figure out a way to get abroad. ‘Cause the rumor used to be that engineers couldn’t study abroad, but there’s so many more types of programs that you can go to and so mine was, finding the right kind of programs and aligning those with the degree requirements of Engineers and then helping the engineers know how to plan ahead to actually travel on them.

Rebecca: So, can you talk a little bit more about your work in K-12 and also the study abroad stuff because in fields where we might not usually think about these as being good matches, like engineering, we’re always looking for new strategies to find those relationships and what have you. So, can you talk a little bit about some of the strategies or things that you developed?

Stacy: Sure. In study abroad, a lot of it was doing the logistics, but some of it was also helping engineers realize that in order to come up with good engineering solutions, you have to really understand the client for whom you’re working; the person who’s found the problem that you’re looking to solve. So you need to not just understand the straight up science, technology, engineering, and math, but you also need to understand the culture of the person, perhaps the language…What is it about their environment that makes different design constraints? So, I think, having engineers study abroad, in such an international world that we live in, is crucial now. I’ve really seen it grow in popularity which has been really fun, even though I’m not in charge of it anymore. We have a very high percentage of students at Vanderbilt who now study abroad as engineers. The second half of that question, or maybe I took him in out of order, was around K-12. You know, at the time I was doing a lot of funded work by the National Science Foundation. My favorite project was a Research Experiences for Teachers program (RET). This is a program where you bring, typically, high school faculty (although that’s broadened some since then) onto your university campus, for six weeks during the summer. Then I would place those teachers into different labs that I had picked very carefully and they would have an assigned project that they worked on full-time, for most of those six weeks…and then at the end of that time, I would work with them on designing curriculum that would be both standards-based (so they would be allowed to teach it in their classroom) as well as based in the research of their labs. So that they were bringing in real-world, current research that was going on, and often the people from that lab would come to the high school as well. Then we would publish those units through a wonderful national digital library called TeachEngineering.org. So that was definitely my favorite piece. I did some other work. I designed some high school level medical imaging curriculum units, and getting to where people have a better grasp of “what is ultrasound? or MRI? and how do those things work?” and actually motivate you to want to study high school physics or math or something like that.

Rebecca: That sounds really exciting and a great way to get people involved in fields they might not know that much about.

Stacy: It’s definitely important, especially when you’re thinking about subjects that sometimes get a bad rap for being particularly challenging. It’s good to let people see why it is they’re learning those and to put that, when am I ever going to use that, upfront so they know exactly when and how they’re going to use that.

John: Has there been any follow up in terms of following students to see how many of them did go into careers in STEM fields?

Stacy: It’s a little hard to get some of that data because I often work at the teacher level and it’s a whole other level of IRB [LAUGHTER] to get at student level data.

John: That’s true.

Stacy: You know, I think it’s somewhat depressing in that the numbers for engineering percentage-wise aren’t increasing rapidly at all, even though a lot of people are putting a lot of time and effort into it. So, not always, I mean I definitely have a lot more confident teachers in the Middle Tennessee area who are integrating what is going on in engineering into their classrooms. Of course it helps now that the next generation science standards have engineering embedded into them and just recently in my state the Tennessee state science standards do as well.

John: In 2010 to 11, you established the Metropolitan Nashville Public Schools K to 12 Engineering Pathways. What problems did this address? How has it worked and are you still working with the Nashville public schools?

Stacy: I think one of the biggest problems it was created to address is a misalignment between what different careers and companies are looking for in their high school graduates, as well as probably colleges too, with what the high schools were producing. There’s such an emphasis now on STEM, and problem solving, and computational thinking that really wasn’t being addressed by the schools and so, with Race to the Top money, Metro Nashville Public Schools set out to form this engineering pathway. I was heavily involved with it for that particular year. I did a lot of professional development. I did all of the professional development for the elementary school that was part of this K through 12 pathway, using the Engineering As Elementary curricula and integrating that. Then, at the high school level, I actually co-taught a ninth grade engineering course at this particular school. So, I was helping another teacher who was an engineer by training but didn’t have as much of pedagogy, and that sort of thing. Trying to help her build up her skills and left her rolling. That high school, Stratford High School, is still clicking along and doing really well with STEM education. It’s growing in reputation and now has a middle school that’s been integrated into the campus as well. So, I think it’s it’s been a success and will continue to be. One of the teachers with whom I worked with the most there is now the STEM director for the entire district. It’s been nice to watch her come from being one of my RET teachers to that position at Stratford, now to leading our entire district. My involvement with the district kind of waxes and wanes over the years. You know, I’ll get really involved for a while, and then I’ll be less involved for a while. I’m not working with Metro Nashville Public Schools right now although I’m always available if they call on me for anything in particular. I’ve actually just joined an advisory board for the Williamson County Public Schools which is just south of here. So, I like to keep my finger in the pie in something locally, but then I often try to work more on a national level.

Rebecca: I wanted to follow up a little bit on what you were talking about elementary education and engineering. For many of us, perhaps, when we went to elementary school, engineering wasn’t a part of that curriculum. So, for those of us that aren’t in engineering can you talk a little bit about what that even looks like?

Stacy: I’d be delighted to. If you think about what the characteristics of an engineer really are…it’s around someone who’s creative, and who thinks outside of the box, and brings in different kinds of solutions, and doesn’t have a lot of preconceived notions. If anything, that’s exactly what a preschool to elementary age child does. They haven’t sort of been beaten down by the system to think in a particular way. They still have that inherent creativity. So, the ideal time to introduce the field of engineering is at the preschool through elementary levels…so that they learn what the field is about, can identify what an engineer does, and have positive feelings towards it, and that we’re creating them to be more STEM literate citizens. There are multiple programs out there. The one with which I’m most familiar, and have even liked so much I’ve joined their staff, is with Engineering is Elementary. But, with any of them you find an authentic but sort of compacted version of the engineering design process. I might look at what a college student would use, or even a high school student might use. and we might call out 12 different phases of the engineering design process. But, in elementary school we have five fingers, so we have five steps to the engineering design process, [LAUGHTER], and in preschool we have three steps. So, just kind of compacting it a little bit…always providing an accurate view of the field. Then giving the kids age-appropriate challenges, things that might happen to an elementary aged child, and then asking them to problem-solve.

Rebecca: Can you give an example?

Stacy: Oh sure. There’s one of the EIE units that comes to mind, where the kids are out there playing a sport and their team needs to be cheered on. They find this little turtle nearby, and they win the game, and so they decide that they’re gonna keep the turtle, and they have to bring the turtle back for the next round of the playoffs. Somebody’s got to keep the turtle in a place where the turtle can not die, because that would not be good for school spirit at all. So, the whole question becomes around, what do you need to design in order to have a habitat that this turtle can live in? They draw upon the appropriate science in this particular unit…and a lot of its around membranes and creating a habitat that has enough water but not too much water. So they draw upon things they’re already learning through the science standards for elementary age children, but they’re putting them to use, and they’re working to save the turtle. Of course they do. It’s an exciting unit, it’s based on a story book that sets the stage for it so you get a lot of your reading and ELA minutes and that sort of thing in it, but then really does bring in science and math as they use the engineering design process.

Rebecca: Sounds really fun.

John: It does.

Stacy: It is a lot of fun [LAUGHTER].

Rebecca: I mean I have to admit I asked that question just because I have a toddler and I was just really curious [LAUGHTER].

Stacy: Talking about the new Wee Engineer, WEE, it’s very cute its for preschool kids.

Rebecca: Yes, yes. Yeah, I want to hear about it. [LAUGHTER]

Stacy: Oh, you really do want to hear about it?

Rebecca: No, I really do.

John: She does [LAUGHTER].

Stacy: The new Wee Engineer units that are coming out are meant for the preschool setting where the teacher introduces the problem…and it’s actually not a teacher, it’s a puppet…and so the puppet comes and introduces and says something like “I want to throw a party for my friends, and I want to make this noise maker really loud, and what do you think of my noisemaker?” …and of course it makes no noise. The puppet then says, “Can you help me?“ …and so the students go through an explore stage, where they explore the materials that are available. A lot of the work at this age focuses on helping students think about how a material is made and how that affects its function. So, they explore different materials and then they get to create their own noisemaker in small groups… and they test it… and then they do it again…and they improve (which is a big part of the engineering design process), until they all have really loud noise makers which they then share with each other and they of course give back to the puppet so the puppet can help throw a good party.

Rebecca: I like that it’s given back to the puppet so that the teachers don’t go crazy. [LAUGHTER]

Stacy: Yes, that would be a critical part of not driving the poor preschool teacher insane.

Rebecca: That sounds like a lot of fun. Maybe I need to go back and teach preschool engineering instead of web design. [LAUGHTER]

John: But, so many students get turned off early on and reaching them early can be really effective in stimulating later interest.

Stacy: They do.

Rebecca: Can you talk a little bit about what you’ve done with STEM for girls and other underrepresented groups and how to get them interested and excited about STEM?

Stacy: Absolutely. My study of the literature shows that a lot of the things that motivate girls also motivates different underrepresented groups, particularly underrepresented minorities/ethnicities, and is often just generally in line with what is good pedagogy, if people actually stopped and thought about that in engineering. I’ll focus on the girls just because that’s been my wheelhouse for the last seven or eight years now. But, a lot of the research shows that girls are interested in helping people or the environment, or something like that. If we can help frame STEM as being something in which you can help people, we will inherently pull a lot more girls into that field. So, that’s kind of one basic way. Often, if I talk to a girl, she’ll say she wants to be a doctor or nurse or something like that in the medical field, because it’s so painfully obvious of how you help people. I try to turn her into thinking about engineering and what engineering really is. Majors like biomedical engineering and environmental engineering are often popular with women, because again, it’s obvious how you’re helping people or things. But if engineers are good, and there’s actually a whole book called Changing the Conversation published by the National Academy of Engineering, on how do we praise engineering appropriately, because it is all about solving human needs and want. If we can present the field more accurately and more fully that will help. I think also when I look at a lot of these things, I like to be very explicit about things like stereotype threat, or implicit bias, or imposter syndrome, and I try to be very overt about teaching students what these things are so that they can recognize it in themselves, know what it is. There’s something about identifying it. I even still have imposter syndrome at times, where I feel like somebody’s gonna figure me out…that I’m not actually that good at engineering education, despite having just been named a fellow of a prestigious Society, I feel like still somebody might figure that out. But I know what it is, I can call it out and say you’re just having a case of imposter syndrome and, somehow, it’s easier to move aside and move along if you know that it’s a real thing and you’re not the only person who has some of these…I call them issues, I’m not sure that’s the right word.

Rebecca: I agree with you. The ability to name it out and file it away allows you to move forward. When I finally learned what some of those things were as a designer, I too, was able to overcome some of those hurdles.

Stacy: I guess the other thing I’ll add, is Carol Dweck’s work with growth mindset, has really put a name to something…about having the ability to think of your brain as a muscle that you can flex and you can grow and it can get stronger. I think letting students know that that’s a thing. Or, at the school where I worked most recently, you were not allowed to say “I’m not good at whatever it was,” you were only allowed to say “I’m not good at _____ yet.” …and I really appreciated that word “yet” there, and the implication that you can and will be good with it, but it’s going to take some hard work, and things don’t always come easily…whether you’re gifted or not doesn’t really matter, you still have to work to accomplish anything good.

John: Besides stereotype threat, implicit bias, and imposter syndrome, what are some of the things that are being done in classes now that deter women and minorities from entering engineering and other STEM fields?

Stacy: The first thing that pops in my mind there is thinking about the examples that are used in a classroom. If there are examples that are supposed to illustrate some concept, yet they are completely unfamiliar to you because the situation in which you’re growing up provides you no context for experiencing that or understanding that, you’re immediately set at a disadvantage in the classroom, and that’s not going to encourage anyone to want to continue in that field. I think there’s also some cases of just downright bias. I had a professor in college that didn’t really seem to think women should be engineers, and well I do know that that is improved, that’s not gone. There are cases of bias that are still out there. I also think a lot about parents and the role of parents, and what they believe their children, their daughters especially, can do…and what’s appropriate for them. Because there are some cultures that have a lot of bias kind of built into them and so it’s about changing the way parents think. Because if a student…if her parents don’t think she should study engineering or science or something, she’s probably not gonna go study that in college. So, we need our parents to understand what these fields are about…educate them…and then get them as a part of our moving more and more diverse people into these STEM fields.

Rebecca: One of the things that I think a lot about is the relationship between design and engineering. As a visual designer, I know that I end up with a lot of students who seem to have a fear of math, or a belief that they just can’t do math, which the process of engineering in the process of visual design is, I don’t know, almost exactly the same. So it’s always interesting to me that they err on the side of the arts thinking that they’re somehow avoiding math, but then of course they discover that there is math there too. Are there things that we can do to help overcome this…I don’t know…. it’s like almost like a preemptive strike, that like “Oh, there’s math. so I obviously can’t participate in this.”

Stacy: I hope so. I feel like we’re making some strides in that area, because you’re right, it is often math that is the big hang-up on why people don’t stay in STEM. Some of that is from having one of your parents, especially the mom, saying “Oh, Honey, I wasn’t good at math, you don’t have to be either.” …which, of course, we would never in a million years say about reading or a lot of other areas. So, I’m not sure why we say that about math sometimes. I think we’ve got to figure out how to let the math come naturally; that, if it’s a part of some problem that you are actually interested in solving, you have empathy for your client, and you’re invested in it, the teachers picked a good problem…“Oh gosh, look we’re gonna have to do this math here” and suddenly it makes sense why you’re doing the math…and you have a reason to want to do it. I think those are critical things that we need to have in our math sequences from elementary on up, so that students don’t develop this hatred or fear of it that is somehow irrational. I also think that while there is math in engineering, not every engineer does mathematics all day long. So, there is some conceptual understandings you have to obtain in order to become an engineer, but it doesn’t mean you sit around and solve differential equations all day long, necessarily. Some can, but many don’t.

Rebecca: I think that’s an important thing. I think there’s a lot of fields where we just assume that people just do math all day and it’s just a misunderstanding or misconception about the field. I think it’s also, sometimes, we present some things in such an abstract way that it doesn’t seem relevant. So, I always like to share with my students the experience that I had around geometry. When I was learning geometry in high school, I could do geometry, I could answer the questions correctly, but I never really understood what the point was and like “I’m never gonna use this.” Then I started doing more programming stuff and made visual interfaces and then all of a sudden was like: “I understand why this is relevant” [LAUGHTER] and I had that breakthrough moment where I was using all kinds of different geometry equations and things to create visual interfaces, essentially.” So, I share that with students and that sometimes helps a little bit. I could put in the math, and then all of a sudden I saw a visual, and then it just clicked and made sense.

Stacy: …made sense…it had purpose to it.

Rebecca: Yeah.

Stacy: I think a lot of math traditionally has been taught like as this separate silo, never to be used…and sometimes I think it’s because the math teachers themselves don’t know when it’s used. They don’t know the science or whatever the other field is…or psychology…and there plenty of places with statistics that use math. But, I feel like we have to lead with those things. So, when I was a high school math teacher and I wanted to teach sinusoids, I would lead with “What’s the temperature gonna be on your graduation day?” …and so we would have to develop this whole model to predict what the temperature was going to be on their high school graduation day – which was not just in a few months, most of them are juniors taking the class. So, we would have to develop this whole mathematical model which involves sinusoids and all the different parameters of one, and then on the test I would give for that unit they had to answer that question for me. It was always fun on their actual graduation day to see how close we had come.
[LAUGHTER]

Rebecca: That sounds like a lot of fun and a great way to make things seem relevant.

Stacy: mm-hmm.

John: So, one thing that would help is if math instruction and science instruction was made more meaningful by using more meaningful examples, so that the math is motivated…so people can get past the fear, because they see that there’s a purpose to it it’s difficult to do that. But, you’re doing some of that with the Nashville schools. I hope we see that more nationwide.
Could you talk a little bit about your work with Engineering is Elementary in designing curriculum?

Stacy: Sure. I have been affiliated with the Engineering is Elementary program for about a decade now. I have followed their work…their research as it was designed and presented at the American Society for Engineering Education. I was super impressed along the way that they were actually doing real education research and they weren’t just developing some curriculum and going “Oh, look how many people use our curriculum.” They were actually looking to see if learning objectives were being met and things like that. So, I’ve had a lot of respect for them. About ten years ago, I affiliated myself with them and became one of their endorsed network providers, such that I could provide their workshops whenever I wanted to around the country, I always really loved and admired how they set up their professional development…how its implemented…that the PD itself is based in research. So, when I was looking for a new educational intellectual pursuit to take on in my career, I approached them and asked if I could come work for them, and to my delight they said “Yes.” So, I started working there part time in January and have enjoyed that. There are separate teams within EIE and I serve on the professional development team right now. So, I’m enjoying working with our extended network of partners, so the people I used to be one of, I now work with and help to take the best of what we’re doing in house in Boston and get that material and the best practices for engineering education out to all these people across the country who can help spread the word and get more kids into engineering. So, I’m really enjoying that piece. I’m gonna be developing some of my first, I’ll call it online PD, but it’ll have some hands-on components to it also, for the adult learners. I think that’s a fun new pursuit for me. In house at EIE, they’ve just created (as I was mentioning earlier) this new Wee Engineer program for preschool and pre-K and there’s also a new EIE for Kindergarten, which I’m thrilled to see, because those age groups desperately need some authentic well-designed, well-researched curriculum. That’s kind of been my role right now. We’ve got some middle school projects as well, but I’m really enjoying starting young and going all the way up through eighth grade and looking at how do you do that best.

Rebecca: As a college faculty member, I’m interested in how you got involved in more of the P-12 things. How might you encourage other faculty, no matter what their discipline is, to get involved at those earlier levels?

Stacy: Great question. I think it’s kind of fun to see, once it clicks to faculty members that what goes on in P-12 definitely affects what can go on at the university level. Some of it can even be slightly self-serving in that they want more students or more diverse students to enter into the university process. So, I think that’s part of it. I think that it’s fun to help a university faculty member see how they can take their passion and enthusiasm that they have for whatever their research field is and take that down to younger kids and distill it to the basics, but increase people’s understanding of their field overall. So, I like that piece of it. There’s the other motivating factor, if you’re gonna apply for like a National Science Foundation grant you have to have an education and outreach component. So, that there’s that external motivator as well to think about how could I be involved in this process. I also worry a lot about we have these new next generation science standards. which I think are quite good but they have a lot of engineering in them. So, who exactly do we think is going to help the K-12 teachers know how to do that, and do it authentically, if we ourselves are not out there helping them and teaching them.

Rebecca: What would you encourage a faculty to do as their first step to get involved in P-12
STACY. I’m trying to think of one single first step…probably, reach out to your kids’ local school and listen…ask the teachers and the administrators…but especially the teachers…ask them what they need. Because the teacher will know. He or she may not know how to go about getting it, but they will know what they need. Don’t go in and be like “I know everything” when in fact you really don’t know everything about what it’s like to be a K-12 teacher…but go in and ask how to be helpful. Listen to what they say and honor the fact that they have to be standards-based.

Rebecca: Sounds like really good advice, but at times, it’s just that little encouragement of what that step could be is helpful. So, thanks for that nugget.

Stacy: Yeah. I mean, go ask. People really want you to.

John: …and in many disciplines, I think, there are organizations that work with elementary and secondary schools. In economics, there’s the centers for economic education spread throughout the country, who do work with middle schools and high schools in providing some educational resources. I don’t know how common that is in other disciplines. Is there anything like that in art?

Rebecca: There is something more general for art, but not for design. Design stuff is kind of under the umbrella of art which, depending on what your position is, you might not think that they’re entirely related. They are, but they aren’t, you know. It’s kind of complicated. {LAUGHTER]

John: You’re working on a new advanced course in engineering for high school students. Could you talk a little bit about that?

Stacy: Yeah, this is a project that has been near and dear to my heart, and some of my wonderful colleagues, particularly from the University of Maryland (Dr. Leigh Abts, in particular). We have been trying to get an AP engineering course started for 14 years now, and when we started, really, the colleges weren’t ready for it. The schools of engineering were not interested in it. They weren’t interested in accepting credit for it. They didn’t really see the value of it. Thankfully, that has changed over time, which I’m really excited about. The College Board got interested in AP engineering and created a framework for the course. They had to put the brakes on that for a little while while they focused on the redesign of the SAT, but now they’re back to being interested in it. So, my team approached the National Science Foundation who said “Yes, we’re on board with this. This needs to happen.” So, we’re modeling our upcoming work off the very successful work of creating the new AP computer science principles course which has been a highly successful course for AP…and it’s also really successful in that a lot of women and underrepresented students are taking the course and taking the test, so that’s super exciting. So, we’re trying to draw some best practices from that and, knock on wood, I hope that the NSF will approve our final proposal to begin the work later this fall. The basic idea around it is we’re going to finish the framework…make sure it’s right…and we’re going to be developing a sample curricula and sample professional development for that. As you may know, with AP courses there isn’t any one set curricula that you’re expected to follow. You can do it however you see fit. You just have to make sure it fits the framework. So, we’ll be developing some sample ones and then we’ll be partnering with high schools. We’re hoping for about 70 high schools all over the country and a lot of diverse settings to train their teachers and have them work with our students and do engineering design at that level. We’re looking at having ultimately an assessment that is a bit like AP art studio actually in that we’re hoping for a portfolio process where you would submit engineering design work that you’ve done over the course of the semester or the year, and then that work would later be evaluated for possible engineering credit.

Rebecca: That sounds like an exciting advancement.

Stacy: I hope so. we’ve had over 110 Deans so far say that they’ll be interested in giving some sort of credit for it. I think it’ll be interesting to see how the universities handle it. I think some might give credit for their actual Intro to Engineering course while others might give it as more general science and engineering credit. I think that the universities now see it as a great way to get more STEM literacy in our population and I think they’ve started to see it as a great way to get more diverse students into their programs, because they will have done engineering at a younger age and done it in the more friendly confines of their local high school.

Rebecca: …and perhaps introduced populations who aren’t really familiar with the field at all to what the field is rather than expecting college students to just magically know what all of our disciplines are.

Stacy: Right, that’s true of a lot of disciplines, so it’s not just engineering.

Rebecca: Yeah, exactly.

Stacy: There aren’t tons of schools that have economics in them but you’re probably not gonna major in it if you don’t know what it is.

John: Actually, I think most schools do now have economics. It’s part of the New York State curriculum and I think most states do have at least one semester economics course. But, it took many many years before that was widespread.

Stacy: We’re trying to catch up with you, John.

John: It’s not always taught by people who know much about economics, unfortunately…. [LAUGHTER] …which is one of the reasons why the Centers do so much work.

Stacy: I would mention that if someone is interested in engineering education, I know that there are now actual programs in engineering education. You can get graduate degrees in it, and I would also steer them towards the American Society for Engineering Education. It’s a wonderful Society. It is the place to go to for what the best pedagogy is in engineering education, and to find your people there. They have lots of divisions, some are specific to your field of engineering and then we have a wonderful pre-college division there as well. So, it can be a great resource if anybody who’s listening wants to jump in jump in and join it.

Rebecca: So, we generally wrap up our podcast by asking what’s next?

Stacy: What’s next? Well, I feel like I’ve hit some of the “what’s next” because I’m in this great transition period in my career, which I’m excited about. So, I’m hoping that what’s next is that personally I’m able to view engineering education from preschool all the way through 12th grade and then into college and look for “How does that pathway work?” Are there things that are missing? Are there things we should be doing differently? So, I’m excited about taking that long view across engineering education and I’m always looking for new collaborators and people who are as excited about the field as I am.

Rebecca: Thank you so much for joining us. I hope that you’ve motivated a lot of others to think about their own disciplines plans from the elementary level all the way up through the university level. Sometimes, that longitudinal perspective can really help us have better perspective on what we’re teaching in higher ed.

Stacy: Definitely. Just to think about like “What matters? “What’s actually important?” It’s easy to get caught up in the minutiae of these little things you have to be sure you’ve mentioned to your students. Not really. Focus in on the big thing.

John: …and if you really want to do something about the gender imbalance in STEM fields you do have to reach out earlier because by the time you get to high school, people have already been sorted out. So, it’s really important to do that sort of work early.

Stacy: Very true. Most of the girls are getting sorted out late elementary to middle school, at the latest. So, you’re absolutely right there.

John: Well, thank you.

Stacy: Well, thanks for having me. This was fun. I appreciate you reaching out to me John, I was flattered.

John: Thank you, Stacy. We very much appreciate you joining us today.
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

42. Flipping the classroom

Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode, Dr. Dominick Casadonte, a Chemistry Professor at Texas Tech University, joins us to discuss research and best practices related to flipped classrooms.

Show Notes

  • Camtasia
  • Mediasite
  • Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement. International Society for Technology in Education.
  • Sams, A., & Bergmann, J. (2013). Flip your students’ learning. Educational leadership, 70(6), 16-20.
  • Bowen, J. A. (2011). Rethinking technology outside the classroom. Journal of Music History Pedagogy, 2(1), 43-59.
  • Bowen, J. A. (2014). The Teaching Naked Cycle: Technology Is a Tool, but Psychology Is the New Pedagogy. Liberal Education, 100(2), n2.
  • Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve student learning (ed.). Jossey-Bass
  • Belford, R. E., Stoltzfus, M., & Houseknecht, J. B. (2015). ConfChem Conference on Flipped Classroom: Spring 2014 ConfChem Virtual Poster Session. Journal of Chemical Education, 92(9), 1582-1583.
  • Stoltzfus, J. R., & Libarkin, J. (2016). Does the room matter? Active learning in traditional and enhanced lecture spaces. CBE—Life Sciences Education, 15(4), ar68.
  • Stoltzfus, Matthew (2016). Engaging Students in the Flipped Classroom (video)
  • Coats, H. J. (2016). A study on the effect of lecture length in the flipped classroom (Doctoral dissertation).
  • Casadonte, D. J. (2016). “The Effectiveness of Course Flipping in General Chemistry – Does It Work?” ACS Symposium Series, “The Flipped Classroom”, December 2016 (Book Chapter) The Flipped Classroom Volume 2: Results from Practice, Chapter 2, pp 19–37, Chapter DOI: 10.1021/bk-2016-1228.ch002, ACS Symposium Series, Vol. 1228, ISBN13:9780841231627eISBN:9780841231610,
  • POGIL.org

Transcript

Rebecca: Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode we discuss research and best practices.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is dr. Dominick Casadonte, a chemistry professor at Texas Tech University. Dr. Casadonte is recognized as a global leader in flipped learning by the Flipped Learning Global Initiative.
Welcome.

Dominick: Thank you very much.

Rebecca: Today our teas are:

Dominick: Well, my usual afternoon beverage is an iced green tea with three pumps of raspberry, but since I’m getting over a cold, today I’m drinking a hot green tea with vanilla, lavender, and honey.

Rebecca: That sounds really nice. Finally, a tea drinker.

[LAUGHTER]

John: I’m drinking Irish Breakfast tea.

Rebecca: …and I’m drinking chai today.

John: You’re recognized as a an expert on flipping the classroom, and you’ve been doing it for a while. Could you tell us a little bit about what a flipped classroom is?

Dominick: Sure. The usual paradigm, if you think about it with regard to teaching, is lecture, homework, lecture, homework, lecture, homework, tests, lecture, homework, lecture, homework, and so on… and the problem with that format is that it’s not conducive to deep learning… with the exception of maybe very bright students or very bright asynchronous learners. Trying to keep up with a lecturer is often difficult. A student, for example, may not have gotten down everything that they wanted to write down… they may not have gotten it down correctly… or it might be fragmented… or perhaps they just didn’t understand well enough to write it down in a meaningful way, or a way that’s meaningful for them. Then they try to go home and they try to do homework, but since they didn’t really understand the lecture in the first place, the homework is again difficult and they might only be able to work part of it. So then, the very next day, there’s a new lecture and a new topic. The normal sort of didactic approach to education often leads to what I’ll call fragmented learning, in the sense that there isn’t enough time in class to practice many of the active learning strategies that work so well because faculty are so concerned with getting the lecture done. In the flipped model, on the other hand, we flip the homework-lecture-homework paradigm so that the students’ homework is to watch a lecture online. Now it can either be PowerPoint or a video and different people use different techniques. It can be done on Camtasia, it can be done through Mediasite, it can be done through a variety of different platforms. It could be done on YouTube for example. So, they’ll watch usually a video or a lecture online before they come to class, and then they may or may not do online homework prior to coming to class as a part of the pre-class experience. Then, once in class, time can be spent, for example, checking the students knowledge… clearing up muddy points or any misconceptions they might have… working advanced problems. But now, in this particular case, class becomes more of a discussion and you can use a lot of the active learning strategies that really engage the students in learning the material. By flipping the classroom, we freed up time to really engage and interact with our students and it helps with their learning in significant ways which I can talk about later.

John: So, in a traditional class, you tell students what the content is in class, and they come in with very different backgrounds and some of them are able to pick it up, others get lost along the way, and then they’re sent out to do homework… where students seem to have the most difficulty on homework or tests… and we’re pretty much leaving them alone to do that. But, here in the classroom, you’re able to work with them and help them through some of the issues.

Dominick: Right… and so, for example, in the classroom with homework, now they have a mentor who can help them work through the more difficult problems to help guide them along. There are models of the flipped classroom where there are peer mentors in the class who can help and walk around and mill around as well. So, there are opportunities for group work. There are opportunities for group discussion. You can engage the topics in a much deeper way than you can by just simply lecturing at students. So, it’s a very very non-passive way of teaching.

Rebecca: What prompted you to get into a flipped classroom model?

Dominick: Desperation, I would say. [LAUGHTER] I first started teaching an online class in the fall of 2007 as part of a multidisciplinary science master’s program of which I was a part here at Texas Tech… and getting everyone on the same platform at the same time, my thinking was that lecturing to them live online was a waste of time, basically. The question that motivated me was the same one that motivated John Bergmann and Aaron Sams around the same time at the high school level up in Colorado. Namely, what could I do to use my classroom time more effectively? What could I do to use my online time more effectively? …and I thought, “Well, let’s take the lecture out of the classroom.” So, I pre-recorded my lectures. I had the students watch the lectures before we got together in the online environment, and then we spent the majority of our time having discussion and working problems…. and these were teachers at the time and they were very very enthusiastic about the ability to actually discuss the material that they were trying to learn… and so then I thought “Well, hmm, if that works really well on an online format why don’t I try it face to face?” …and so then, starting in 2008, I adopted this to my general chemistry class. The following summer, a high school teacher that I had as part of a workshop said “Oh, I see you’re flipping your class” and I said “Flipping, what’s that?” ‘Cause at the time, there were a lot of different terms for flipping. It was called: time-shifted instruction, reverse instruction, blended instruction, all sorts of things. Flipping is a term that’s really sort of stuck.

So, I started out of a sense again of “How can I engage my students more effectively in the classroom?” and once I realized that it worked swimmingly in the online environment and then I said “Well, okay, will this work in the face-to-face?” …and it worked even better there. By that point, there are pockets of people around the country who are doing this, and they used all sorts of interesting terms. Jose Bowen at SMU, who’s an art professor, used the term “naked teaching” because it can be unsettling when you walk into a classroom and not have the comfort of your lecture notes to be able to project or read to the students. So, very often I will walk into the classroom and say “What are we going to talk about today? or “What would you like to talk about today?” You can’t do that unless you feel comfortable about the subject material and you have some expertise, but it’s a great way, in the long run, to really, really impact students.

Rebecca: You mentioned both doing this in an online environment and also in the classroom. Is there a difference between your experience in both?

Dominick: Yes, it’s so much easier face-to-face. To be able to walk around to gauge what’s going on in the classroom, their level of understanding, and how learning is happening. The power of direct peer-to-peer contact should never be underestimated, I think. Now, in the online format, my focus was more on the development of a learning community working on a particular topic, rather than in peer-to-peer mentoring and things like that. So, they’re very different approaches, but you do develop a sense of community in both, I think.

John: Now, with your online classes are they synchronous or asynchronous?

Dominick: My online classes are synchronous, in the sense that everybody meets in one place at one time. We have various software programs that allow us all to be in the same place. Yeah, so they’re all together at one point, and that’s kind of interesting because I have people from all over the country who are taking these classes. They’ve never actually physically met, at least for the first class that I teach. It’s interesting trying to, in the discussions, get a sense of the personalities of the people who were providing discussion.

John: Are they participating in video formats or is it just audio or text?

Dominick: I’ve done both. I’ve done video and audio and depending on the bandwidth and the number of students in the class both can work well.

John: How large are your classes?

Dominick: Well, my online class had 24 students in it. So, a fairly large, I think, from an online perspective class. It wasn’t one of these very very large classes like you might see at MIT, for example, but 24 is a good class in terms of bandwidth… trying to get everybody in the same room at the same time… and my face-to-face classes… they’ve ranged anywhere from a low of 25 to a high of 150.

Rebecca: You mentioned two different techniques that you use both in online versus in person. So, online you mentioned community formation and then in person you mentioned peer to peer. Can you expand upon each of those?

Dominick: Yeah, my first adage for teaching is know your audience, and in the face-to-face environment I was working largely with in-service teachers who were trying to develop a more significant content knowledge of chemistry, and so there the idea was the development of a supportive community where these teachers could bring their ideas to the table in terms of not only the content that I was teaching them, but also how they could then apply that content in their own classroom settings… and in that regard they were able to help teach each other techniques that they could use in the classrooms based on what they were learning in terms of the content. There really was a community focus, sharing knowledge as opposed to just gaining knowledge. In the face-to-face classroom, it’s more a sense of the students trying to understand the content at a deep level. There I found that it is true that when you teach something you really hopefully really understand it and so peer-to-peer mentoring is much more effective there. So, we tend to work in groups… we tend to work with dyads… two people working next to each other… and then sometimes I will just have people go to the board… but once again it was in that context of community because I think it’s very important, if you’re going to do flipping, and do it well, that it’s an active, encouraging, engaging classroom experience. If my students go to the board, whether they get the problem right or wrong, the class gets into a habit very very quickly of applauding the student for their attempt… whether or not it’s right or wrong… and then we debrief. We talk about what works… what doesn’t work… So, it really is both peer to peer and community building there as well. But, the emphasis is more on the individual in the face-to-face classroom, I think.

Rebecca: It sounds to me a little bit like the choices that you’re making online and in-person aren’t necessarily because of the medium, but rather who’s in those particular classes. Am I hearing that correctly?

Dominick: That’s right, and one of the nice things about flipping is that it is such a rich environment in which to work. As I mentioned earlier, the pre-class videos can be video lectures, they can be audio lectures (if that’s appropriate), they can be PowerPoint presentations, they can be any number of things. In the in-class experience, it’s an active learning environment. so you tailor it to the people in the classroom. For example, if you’re trying to teach in a flipped environment of the class of 24, there you can do all sorts of things that promote individual learning in ways that it’s a little bit more difficult to do in a class of 300, for example. But, I have a good friend, Matt Stoltzfus at Ohio State, for example, who routinely flips like general chemistry class of 600 students, and he’s able to give them as close to a personal learning experience as one can, I think. The point that I’m trying to make is that the flipped environment is a very rich one and it allows you to tailor the learning experience to your class specifically.

John: Do you create your own videos or do you use ones created by other people?

Dominick: I actually create my own videos. I have my own recording studio in my office… I have a Mediasite setup…. I have a video camera… I have a document camera… and I have a wonderful microphone…. and so every couple of years or so I re-record all of my videos. Now, I do use a mixture of other formats. So, for example I do have post-video homework that the students have to do online before they show up to class… and that’s done using an online learning platform that we have here at Texas Tech through a national distributor. The advanced problems that we work in class come both from the textbook that we use and also from problems that I develop. So, it’s sort of a hybrid. But, the pre-lecture videos I actually produce…… and one of the things that studies have shown is that students develop a certain sense of identity with regard to the person teaching the class. Some people, when they’re starting to flip, might want to just use Khan Academy videos, for example, or things like that. But the studies show that the class wants to see folks on the videos (or your voiceovers if you’re using PowerPoint with voiceover)… they want to see the professor who’s teaching the class, because that’s their professor… and so they develop ownership, if you will. Plus if you’re using other media, for example a Khan Academy, they may not be teaching it exactly the same way that you want to teach it, and so then you have to either reteach or undo. There’s a quality control issue there. So, it’s just easier, if you’re gonna do flipping, to make your own videos… and there are so many different ways of doing that now that there’s really no excuse.

Rebecca: That had been my experience as well when I’ve done videos in my classes. The students really liked the quirkiness or knowing that it’s the same person that they had in their classroom and if you try to slip in something else occasionally they really didn’t like it.

Dominick: …and a lot of people have a perfectionist tendency and really want their videos to be really super perfect. Well, once again, studies have shown that that’s not really what students want. They want to see the foibles. They want to see you as you are. If they know that you’re going to say “um” or “uh” in the classroom then if you don’t say “um” or “uh” in your videos, then they’re gonna say “Is that a robot teaching the class that looks like my professor?” So, it’s okay to be human when you’re doing the pre-lecture videos. But, I think one of the things that often hangs up people when they’re starting to do flipping is this notion that it has to be perfect, and it really doesn’t.

Rebecca: I think they really appreciate when you make mistakes and things, too. I know that my students did when I do like a coding mistake, I’m like “Whoops, I made a mistake..” and go back and fix it and explain what I did and why it was wrong. We all make those kinds of slips and errors and things and we would do it live. So, it’s kind of nice to do it in videos, too.

John: It makes you seem more human by doing that.

Dominick: One of the nice things about them seeing you make mistakes is that it gives them permission to make mistakes. They don’t have to be perfect. I’ve had experience with students in classrooms where they come in and they’re intimidated… they’re shy… they’re just afraid that they can’t master the materials. So, seeing somebody make a mistake who is an expert gives them permission to make mistakes… and one of the things that that really does is it empowers them to learn, because at the end of the day when somebody’s trying to learn they’re going to make mistakes and I give my class permission to make mistakes. In fact, I tell them you have permission to make mistakes while you’re learning. After you’ve learned something, that’s a little different. If you’re an engineer, for example, and you’re building a bridge, I don’t want you making mistakes. But while you’re learning… absolutely, make mistakes. Part of education is this movement from novice to expert and in that process one makes mistakes. So, I told my class “You have the right to make mistakes” and I use an example. So, let’s say you’re a five-year-old and you’re trying to learn how to ice-skate you fall down, what does a five-year-old do? They laugh, they brush themselves off, they might giggle a little bit, they get up and then they just skate some more. Now, imagine you’re an eighteen year old and you’re learning to ice skate and you fall down. Most of the time, people stand up and say “Did anybody see me?” and they worry about what people are gonna think, instead of just getting up, laughing, and moving on. So, what often happens is the 18-year old never learns to ice skate, whereas the five year old, who’s willing to make mistakes, learns. So, I tell them it’s okay to make mistakes while they’re trying to learn… and also it’s about empowering students to be able to have confidence in themselves… and we talk about this a lot. We did a study of what motivates students in the flipped environment, and part of it is the confidence… the autonomy… that they develop in the flipped environment. When they really think they’ve got and they really understand, there’s a certain level of “Geez, I understand this. I can understand the next thing…” and so on and so forth. That sense of confidence really improves their educational experience.

John: How long are your videos? Do you tend to have very long ones or do you chunk them up into smaller chunks? and what would you recommend in terms of video length?

Dominick: We did a study a few years ago on the optimum length for flip videos, and it came about because our book dealers: McGraw-Hill, Prentice-Hall, Cengage… those are the top three that we have here at Texas Tech… were telling us that they were creating videos and they’re creating these five to seven-minute videos… and I said why are you making five to seven-minute videos?” and they said “Well, everybody knows that the Millennial generation has a five- to seven-minute attention span and so they want five to seven minute videos.” I said: “But, what is best for learning and improving learning outcomes? I don’t care what their preference is. What’s best for them to learn?” He said “Well, we don’t know.” I said “Well, do you have any data or evidence that shows the five- to seven-minute videos are really great for learning?” and they said “No, but we would be interested if someone would do a study and tell us.” So, I had a graduate student who embarked on a study of lecture video length. We set up essentially short video people… those are people to watch five to seven-minute videos… and we did this with master videos, so each video was probably 40 minutes to an hour long. We put stop signs in the video and when they hit a stop sign they would stop. They would work some online homework and then they could pick up again… or they could go off and do something else. but the short videos were five to seven minutes and then we had a long view group, as it were. We allowed them to decide which videos they wanted to do and not surprisingly, 62% decided to do short videos. So, I have no problem with the book dealers’ notion that Millennials prefer shorter videos. But, then we let the semester go on and we didn’t force them to stay into one group or the other. We let them move and we watched their video habits. We weren’t video stalking them, but we could watch their lecture habits using the Mediasite analytics down to the millisecond.

What we found was that 60% of that short video group switched to the long video length, which is very surprising to us, and we did also a variety of assessments. We looked at online homework grades. We looked at quiz grades. We looked at exam grades. We looked at final exam grades. We looked at the American Chemical Society standardized tests that we give as a pre-post and what we found was that there was a subgroup of the long-view group that watched the video as a long video, but then they stopped at specific points to either have a snack with a friend, go to the bathroom, whatever they needed to do… and we called them the long pause people. It turned out that in every assessment that required global understanding… so final exams, ACS exam individual exams… the long-pause viewers actually scored one standard deviation higher than the short viewers. In fact, the short viewers had the worst learning outcomes of all three groups. Then we gave them Likert scale questionnaires, and we also gave them open-ended questions, and we said “Why did you make the switch? What do you find?” and they said “Well, it just got too fragmented to look at these short videos and we couldn’t take what we saw in video A and connect it to video B and so on and so forth, especially if several hours had gone by, because we then had to go back and watch video A again to remember what we forgot. But, if we did the long videos we were able to just put it all together.” Also in that we found that the best optimum video time. So, what constitutes a short video versus a long video for Millennials is 20 minutes or less is short, 30 minutes or more is long. So, 20 to 30 minutes is the sweet spot for video length. That’s what the study showed us and we’ve just submitted that for publication.

Rebecca: Have you adjusted how you’re teaching based on that information?

Dominick: Yes, my videos are roughly in the range of 30 to 40 minutes, but I tell my students “Take the time you need. Take breaks. It’s going to help your learning outcomes.” I also share with them the results of our studies, because I think if you’re going to, in my case if you’re gonna be a scientist, you should be data-driven. So we want our students to know that we’re not just telling them this because it’s something anecdotal, but rather it comes from data that we’ve collected.

John: Have you thought about controlling for self selection and randomly putting students in groups? Because one concern I’d have with that is that it could be the case that those students who select the short videos might have done less well no matter which group they were in or vice versa… although you do have switchers in there.

Dominick: Yeah, I’ve actually been very lucky in my studies that I teach our honors general chemistry sections… and so I look at the SAT scores… I look at their previous class performance… and it’s a very very, as much one can have, a homogeneous group. So there’s really not much of a selection bias, I think, as far as the study goes.

Rebecca: You’ve also done some research on the flipped classroom approach in general, not just the video length. Can you share some of your findings?

Dominick: Sure. We found out a lot about the flipped environment over the past 10 years or so. As I mentioned a minute ago, I’ve been very lucky in that I tend to teach very bright students. I did a five-year longitudinal study on the effect of flipping which has been published in American Chemical Society monograph and we found that the average exam grade increased by 9.2 percent over that five-year period, and that the largest gains in learning came during summative assessments (for example during final exams and externally developed independently normed exams like our ACS exam). We also have done work on what motivates students to do well in the flipped classroom and we just recently presented at the biennial conference on chemistry education regarding the effectiveness of peer mentoring during the flipped classroom. The results there were very astounding to me. It shouldn’t have been because peer-led team learning has been around for more than 15 years, but nonetheless, in trying to tweak the classroom, the addition of the peer mentor took what was already much better than had been before and improved it dramatically. I’ll give you an example. On the American Chemical Society end-of-term exam, without the peer mentor present… So, I give my students incentives if they score above the 95th percentile on that exam, and it’s a challenging exam, they get an automatic “A” in my class. They don’t have to take the final because this exam is normed against thousands of students around the country in a variety of different university settings. Pre-flipping, I had zero to one student scoring above the 95th percentile; post-flipping the average was roughly nine. So, it increased tremendously. With the presence of the peer mentor, the number went from 9 to 34 and so almost a six-fold increase in the number of students scoring above the 95th percentile. The only difference being, since I used to run the discussion sections that I subsequently allowed the peer mentor to do, that was really the only difference in the environment as far as I could see in terms of controlling for the different factors, except for the obvious one of different students. But she did this in the fall of 2016 and the fall of 2017 and there were 34 in one in the fall of 2017, 33 in the fall of 2016. The percentages of the class actually increased. This is kind of reproducible, if you will. So, we’ve looked at a variety of different things, and with regard to motivation we looked at that from self-determinacy theory and found essentially three things that really sort of motivate people: one is autonomy, second is pace, and the third is responsibility.

In the flipped environment, this sense of autonomy… the sense that “Oh yeah, I am really learning something” is very important to the students. The pace, the fact that they could watch the videos in their pajamas, for example, was very important to the students as a motivating factor… and responsibility, the fact that they had to take responsibility for their own education as opposed to being sort of spoon fed in a lecture format was something that motivated them as well. That student just graduated in December, so we’re now preparing that for publication. So, we looked at a variety of different aspects of flipping.

Rebecca: Can you clarify about your peer mentor model? The students are watching videos outside of class, and then they’re coming to a class with you, and then also recitation session with another student?

Dominick: They watch a pre-lecture video. They do online homework. We have a class discussion… work problems… and then there’s a separate recitation section… and historically I have done all of that. I had a very bright student who really wanted some teaching experience and she said “Would you let me run your recitation?” and I, like many faculty perhaps, don’t really want to give up control of my classroom environment. So, it took a lot of cajoling on her part to get me to do that…. and I said “Well, let’s look at it and see what happens.” She took over the recitations and… talk about a motivated young lady… she would provide review sheets for them… she would do all sorts of things that I would do, but the way she would do it, I think, spoke to the students so much more effectively than I was able to do. I think that’s one of the real reasons why their scores went up.

Rebecca: She knew how to meet them where they were at in a way that, as we become more of an expert in our field, we lose touch with that.

Dominick: Yeah, that’s exactly right. So, one of the things that I’ll be interested in following is that there was a recent study that Prentice-Hall published about the Millennials versus the Gen Z students… and one of the things they noticed is that, while there was about a sixty or so percent “like” amongst the Millennials for using computers and computerization and video and things like that, that number was almost half for the gen Z population. It’s almost as though it’s such a normal part of their life, the gen Z population, that there’s nothing special… there’s nothing unique about it… there’s no value-added to doing videos… it’s the sort of normal expectation. So, I’ll be very curious to see how this sort of flipped environment works over the next 10 to 15 years when the expectation is that they’ll be seeing videos at some point either prior or post classroom experience.

Rebecca: …definitely an interesting question.

John: You’ve organized several symposium on flipped classrooms. What are some of the biggest takeaways from those symposia when you bring people from many different disciplines together? Are the results pretty similar across disciplines or do they vary substantially?

Dominick: Well, I think the first thing that struck me with regard to these symposia is how diverse the flipped environment really can be. Since active learning occurs during the classroom time, there’s almost as many different active learning strategies as there are teachers, and so no two flipped classrooms are the same… and that’s the first thing that I learned. The other thing that I learned is that you have to be committed to flip. You can’t do it half-baked. If you try to do flipping, there was one example of a professor who said “Well, all this is is what we’ve always done. You tell the students to read the material before they come to class and then we have a discussion.” So, he said “Okay, I’m going to ‘flip my classroom’ and just have them read the textbook before they come to class and then we’ll have a discussion.” He found very quickly that the students weren’t reading the textbook, so there was very little discussion going on, which frustrated the professor and frustrated the classroom and set up more of an adversarial relationship.. and it was the worst teaching experience he ever had and he said “I’m never going to flip my classroom again.” So, one of the takeaway messages, and this was reported at one of the symposia, is that if you’re really going to flip your classroom, you’re in for a dime, you’re in for a dollar. You do it as well as you can and be very concerned about what you’re putting in and what you’re expecting to get out or it can be a very very bad experience. Now, I will say this, that almost everybody when they get up on board the flipped bandwagon, especially if you’re using technology prior to classroom, it’s hard at first. That’s the other real take-home message. It takes a lot of time to flip your class the first time. But, once you do it, it actually is much more enjoyable. It’s actually easier, I think, than the regular didactic approach. Those are some of the take-home messages.

Rebecca: Can you talk a little bit more about some of the other challenges faculty might face if they’re doing it for the first time?

Dominick: Sure, yeah. The first and biggest challenge that people have is time. Don’t decide you’re gonna flip your classroom two weeks before the start of a semester. It can be the most horrible experience you’ve ever had as a teacher, because students have an expectation that each lecture is going to be there for them. As we all have various things come up or university requirements… meetings… things like that… and you may find yourself at 3 o’clock in the morning trying to record a video for dispersal at 8 o’clock in the morning. So, don’t wait till the last minute. That’s the biggest sort of thing. That’s one of the reasons why people are sometimes a little bit risk-averse with regard to flipping their classroom.

Other things that have come up: saying “Oh, I can just use Khan Academy videos or videos that are on the web” …and they haven’t previewed them, for example, and then they find that the students have a very different concept of the material than the professor has and then you end up spending a lot of time: “Well, which one do we believe? Do we believe the video that you showed us? or do we believe what you’re telling us?” and so it can create an environment which doesn’t propagate trust in the learning experience. Other issues are: online homework versus homework that you put together for the students… what are you gonna do during class time? how are you gonna fill that time. If you’re not familiar with active learning strategies, that can be very daunting. For example, you walk into a classroom and the students have already watched a video… they’ve done some homework… So, what’s your value added? If you’re not used to active learning strategies, that can be very difficult the first time that you’re doing it…. and then do I need to change my assessment strategy based on the fact that now I’m using a different kind of pedagogy? So, there are a lot of different moving parts and I think putting all those moving parts together can somewhat be inhibitory for people who are trying to flip for the first time.

John: What types of active learning strategies do you use? You’ve mentioned some group work on problems, but could you give a few examples of types of activities you use during the class sections?

Dominick: Sure, there’s group work first of all. I send students to the board. I’ll pass out file cards, for example, and ask the students to put down one thing that was unclear in the video or something that they really would like to learn about in addition to the video. So, it’s not just muddy points, but it’s also how do we expand and extend. Because in my case it’s an honors class and I want to give them a little bit more than the normal amount of material and experience, and so those are some of the things that we do. We have to make molecules and structures and do all sorts of things, and so I use human atoms. I have volunteers come up to the front of the class and they have to then make molecules. They have to develop particular structures. They have to show how they bond, how they vibrate, how they move. What they do. So, they have to actually sort of insert themselves into the molecular dynamic, if you will. They’re trying to understand that, as an atom won’t understand its environment. The first couple of times, students aren’t used to that level of kinesthetic learning, but once they get it, then I usually have a fair number of students who are willing to volunteer and come up. Because, while they’re doing that, we’re also discussing, and I’m asking the class: “Okay, so why is it that this particular atom won’t bond here? What’s wrong with this?” and so it’s a real discussion… but now using human beings as the models rather than just making stick figures or things like that.

So, I try to move students through a variety of different learning environments that engage them not only visually, auditorily, but also tactilely. and kinesthetically, and that’s somewhat unusual, I think, in chemistry classes. Because, once again, most people don’t think of chemistry as a visceral activity in many cases.

Rebecca: Actually, it sounds like a lot of fun.

Dominick: Well, and it is, and I’ll tell you I walk into my class the very first day and they’re all very respectful, because Texas students typically are very respectful and they’re honors students. So, they really want to make a good impression. I usually start my class laughing, telling them this is the last time this class will be quiet. A noisy class is a learning class, a quiet class can be a sleeping class, I don’t know. So, yeah, my class is always very, very engaging, I hope.

Rebecca: Have you ever had the moment when you’ve asked your students “What do you want to talk about today?” and then not have anything they want to talk about?

Dominick: Yes, I think every faculty member has that “Oh, my gosh” moment. I always come prepared with questions, so if they don’t have anything to talk about, then I’ll ask them “Well, what did you think about this particular part of the video?” or “Did you really understand this?” or “Let’s take this concept and move it farther” because one of the things I never do in my classroom is just do a rehash of my video. I figure they’ve watched the video. So, I might say “Well, okay, you saw this but how could we apply this in this other setting?” …and then if they really didn’t understand it, then I’ll be able to tell in a heartbeat whether or not there are real questions because they really don’t have a good sense of understanding… and then I can go back and say “Okay, at what point is this breaking down for you? How did this not work?” Silence kind of tells me that there is usually a breakdown somewhere… and so I try to address what that breakdown is and try to correct it. So, that’s what I meant earlier when I say one of the things I do during my class is try to clear up misconceptions… try to address muddy points… and just make sure that they really understand the lecture part as well. That can take anywhere from five minutes to 45 minutes. My classes are an hour and a half long. Depending on how difficult a lecture was… on what their level of understanding is… So, I want them to have deep understanding of the content and so if they’re coming in silent then I worry that that depth is not there.

John: When they see the videos, you have them take tests. Are the tests the same for each student? or do you vary the questions? Is there some randomization there?

Dominick: Well, no. So, we give departmental exams, so the exams are the same for every student, basically. I know many universities have a format and they can vary the numbers that are put in, for example, but no, my students really… they’re fairly separated when it comes to the exam, so I don’t really feel the need to give each one a separate numerical set of calculations.

John: …and it sounds like many of your students are honor students where that might be less of an issue.

Dominick: Right, and we have an honors code through the Honors College, and I tell them the first lecture if you’re cheating you’re out of here. If you’re cheating you’re out of the University… and I’ve had the unfortunate occasion to have students suspended from universities… not at Texas Tech, but at other places I’ve been… and so they understand that I’m very serious about that.

John: On those tests, do they have multiple attempts or just a single take on the test?

Dominick: Well, my tests are usually set up so that they’re half multiple choice, half free response. Because at the end of the day, I tell my students “Qe don’t live in a multiple-choice world… A. agree, B disagree… you know, we don’t… and so I need to know how you’re thinking. While multiple choice exams are expedient in terms of grading, they don’t let me know what you know. So, I give them both because some students like multiple choice exams. They think they’re really good at them. Some students really want the free response. So, it’s a mixture of both. I try to give them a rich assessment environment as much as possible.

Rebecca: So, to follow up on what John was asking, are your homework assignments kind of a multiple attempt to help learning? or is it a one attempt kind of thing?

Dominick: It’s a one attempt. Yeah. Now, what I do allow them to do after they’ve provided their answer, is that I’ll allow the question to be open so they can go back and review it, especially if the answer is wrong…. and I allow them to do that to help them review for the exams as well.

John: So, they can go back and retake it, but only the first attempt counts towards their grade?

Dominick: Right. So, they have to think about what they’re putting in there before they put it in. Because in some cases, I’ve heard stories of students who put a wrong answer in purposefully and then the online learning environment gives them hints or tells them how to work a problem just like that… and then they go back and they have another problem with just different numbers but they’ve already been coached essentially in terms of how to answer the problem. Once again, perhaps it’s because I think you have to do things right eventually at the end of the day, I really want them to get it right… and these are… once again… they’re relatively straightforward pre-class questions. Their designed as just-in-time or warm-up questions. They’re not multifaceted. The questions we’re getting in the class are really challenging problems. They’re challenging problems using an honors book. So, hopefully they differentiate between those.

John: Are they graded on the problems they do in class as well?

Dominick: They’re not. Because part of that environment, once again, is to have groups… have community… have a mentoring process… and so, the ultimate goal in that whole process is the solution of the problem. So, I’ve already tested them in the online learning environment. I’ll test them on the exams. I’ll test them on quizzes. But, in the class I want the process of how to solve the problem come forth and not the grade be the most important thing.

Rebecca: So, we always end or wrap up our podcast with the question: what are you gonna work on next?

Dominick: Well, I have a pretty active and diverse chemical education research group and with regard to flipping, specifically, I mentioned that I had a recent PhD who looked at motivation in the classroom and what we found there were there were basically three reasons that motivates students to want to do well: autonomy, pace, and responsibility. Through the flipped environment they learn how to develop confidence in their ability to learn, and secondly they liked that the class is largely self-paced and they get to watch the videos in their pajamas if they want to, for example… and finally they really appreciate the fact that they are responsible for their own learning. So, we’re going to be looking at the role of metacognitive intervention as an autonomy motivator in the flipped classroom. That is to say, if we help them think about how they’re thinking during the early parts of the flipped classroom, do they proceed to confidence in their ability to learn that much faster… and we’ll also be looking at how the flipped classroom, especially with community building activities and community building learning strategies can improve the learning outcomes among historically underrepresented communities in the sciences… though especially communities where the notion of family and community is so important in their lives… that are not necessarily in the classroom. So, those are the two areas that we’re going to be moving into with regard to flipping… and I have a number of other projects that are not related to flipping as well. So, it’s a very diverse group of questions that we’re trying to answer, but once again I think that the flipped environment is a very value-added environment for both the students and the faculty… and so I think it’s a mature pedagogy in the sense that we talk about process oriented guided inquiry learning (POGIL) being mature and peer led team learning activities (PLTL) as mature pedagogy. Service learning is another mature pedagogy that has matured over the last 20 years or so… and I think it’s now safe to say that flipping is a mature pedagogy. In fact, at the biennial conference on chemistry education, there was a wonderful paper doing a meta-analysis on flipping and the presenter showed that in terms of looking.. I think he looked at 18 or 19 different papers on the flipped environment… and he found that, in general, there’s about a 30% improvement in student learning outcomes and it’s even better in organic chemistry than general chemistry (which was surprising to me). But, nonetheless it really does improve learning for students and that, in the final analysis, is what we’re trying to do.

Rebecca: It sounds like some really interesting projects. We’ll be looking forward to finding out what you find out.

Dominick: Thank you.

John: Well, thank you. this has been fascinating.

Rebecca: Yeah, thanks so much for spending some time with us today.

Dominick: Thanks, I’m gonna have another sip of tea. Thank you.

Rebecca: Thank you
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

37. Evidence is Trending

Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode, Michelle Miller joins us to discuss recent research that will likely shape the future of higher education.

Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.

Show Notes

Rebecca: Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode we talk to a cognitive psychologist about recent research that will likely shape the future of higher education.
[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Michelle Miller. Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.
Welcome, Michelle!

Michelle: Hi, I’m so glad to be here.

Rebecca: Thanks for joining us.
Today’s teas are:

Michelle: I’m drinking a fresh peppermint infused tea, and it’s my favorite afternoon pick-me-up.

Rebecca: …and it looks like it’s in a really wonderfully designed teapot.

Michelle: Well, thank you… and this is a thrift store find… one of my favorite things to do. Yeah, so I’m enjoying it.

John: I have Twinings Blackcurrant Breeze.

Rebecca: …and I’m drinking chai today.

Michelle: Pretty rough.

John: We invited you here to talk a little bit about things that you’ve been observing in terms of what’s catching on in higher education in terms of new and interesting innovations in teaching.

Michelle: Right, that’s one of things that I really had the luxury of being able to step back and look at over this last semester and over this last spring when I was on sabbatical… One of the really neat things about my book Minds Online, especially now that it’s been out for a few years, is that it does open up all these opportunities to speak with really engaged faculty and others, such as: instructional designers, librarians, academic leadership, educational technology coordinators… all these individuals around the country who are really, really involved in these issues. It’s a great opportunity to see how these trends, how these ideas, how these innovations are rolling out, and these can be some things that have been around for quite some time and just continue to rock along and even pickup steam, and some newer things that are on the horizon.

John: You’ve been doing quite a bit of traveling. You just got back from China recently, I believe.

Michelle: I sure did. It was a short visit and I do hope to go back, both to keep getting involved in educational innovations there and, hopefully, as a tourist as well. So, I was not there for very long but I had the opportunity to speak at Tsinghua University in Beijing, which is a really dynamic institution that’s been around for about a hundred years. For a while in its history it specialized in things like engineering education polytechnic, but now it’s really a selective comprehensive university with very vibrant graduate and undergraduate programs that are really very relatable for those of us in the United States working in similar contexts. My invitation was to be one of the featured speakers at the Future Education, Future Learning Conference, which was a very interdisciplinary gathering of doctoral students, faculty, even others from the community, who were all interested in the intersection of things like technology, online learning, MOOCs even, and educational research (including research into the brain and cognitive psychology), and bringing all of those together… and it was a multilingual conference. I do not speak Chinese but much of the conference was in both English and Chinese and so I was also able to really absorb a lot of these new ideas. So yes, that was a real highlight of my sabbatical semester and one that I’m going to be thinking about for quite some time.

I should say that part of what tied in there as well is that Minds Online, I’ve just learned, is going to be translated into Chinese and that’s going to come out in May 2019. So, I also got to meet with some of the people who were involved in the translation… start to put together some promotional materials such as videos and things like that.

Rebecca: Cool.

John: Excellent.

Rebecca: So, you’ve had a good opportunity, as you’ve been traveling, to almost do a scavenger hunt of what faculty are doing with evidence-based practices related to your book. Can you share some of what you’ve found or heard?

Michelle: This theme of evidence-based practice, and really tying into the findings that have been coming out of cognitive psychology for quite some time, that really is one of the exciting trends and things that I was really excited to see and hear for so many different quarters I visited in different institutions… and so I would say definitely, this is a trend that is continuing and is increasing. There really does continue to be a lot of wonderful interest and wonderful activity around these real cognitively informed approaches to teaching, and what I think we could call scientifically based and evidence-based strategies. One form this has taken is Josh Eyler’s new book, called How Humans Learn: The Science and Stories behind Effective College Teaching. This is a brand new book by a faculty development professional, and a person coming out of the humanities, actually, who’s weaving together even from his humanities background everything from evolutionary biology to classical research in early childhood education to the latest brain-based research. He’s weaving this together into this new book for faculty. So, that’s one of the things that I’ve noticed and then there’s the issue which i think is another great illustration of best-known practice which is the testing effect and retrieval practice.

John: One of the nice things is how so many branches of research are converging… testing in the classroom, brain-based research, and so forth, are all finding those same basic effects. It’s nice to see such robust results, which we don’t always see in all research in all disciplines.

Rebecca: …and just breaking down the silos in general. The things are all related and finding out what those relationships are… exploring those relationships… is really important and it’s nice to see that it’s starting to open up.

John: We should also note that when you visited here, we had a reading group and we had faculty working on trying to apply some of these concepts, and they’re still doing that… and they still keep making references back to your visit. So, it’s had quite a big impact on our campus.

Michelle: This wasn’t true, I don’t think, when I first entered the teaching profession… and even to the extent when I first started getting interested in applied work in course redesign and in faculty professional development. you would get kind of this pushback or just strange looks when you said “Oh, how about we bring in something from cognitive psychology” and now that is just highly normalized and something that people are really speaking across the curriculum… and taking it and running with it in a lasting ongoing way, not just as a “Oh, well that was an interesting idea. I’m going to keep doing what I’m doing” but really people making some deep changes as you mentioned. This theme of breaking down silos… I mean I think if there’s kind of one umbrella trend that all of these things fits under it’s that breakdown of boundaries. So, that’s one that I keep coming back to, I know, in my work.

So, the idea of retrieval practice, drilling down on that one key finding which goes back a very long ways in cognitive psychology. I think of that as such a good example of what we’re talking about here… about how this very detailed effect in cognition and yet it does have these applications across disciplinary silos. Now when I go to conferences and I say “Okay, raise your hand. How many people have ever heard of retrieval practice? How many people have ever heard of the testing effect? How many people have heard of the book Make it Stick (which really places this phenomena at its center)?” and I’m seeing more hands raising.

With retrieval practice, by the way, we’re talking about that principle that taking a test on something, that retrieving something from memory actively, has this huge impact on future memorability of that information. As its proponents like to say, tests are not neutral from a memory or from a learning standpoint… and while some of the research has focused on very kind of stripped-down laboratory style tasks like memorizing words pairs, there are also some other research projects showing that it does flow out to more realistic learning situations.

So, more people simply know about this, and that’s really the first hurdle, oftentimes, with getting this involved disciplinary sometimes jargon riddled research out there to practitioners and getting it into their hands. So, people heard of it and they’re starting to build this into their teaching. As I’ve traveled around I love to hear some of the specific examples and to see it as well crop up in scholarship of teaching and learning.

Just recently, for example, I ran across and really got into the work of Bruce Kirchhoff who is at University of North Carolina – Greensboro and his area is botany and plant identification. He has actually put together some different really technology-based apps and tools that students and teachers can use in something like a botany course to rehearse and review plant identification. He says in one of his articles, for example, that there just isn’t time in class to really adequately master plant identification. It’s just too complex of a perceptual and cognitive and memory test to do that. So, he really built in from the get-go very specific principles drawn from cognitive psychology… so, the testing effect is in there… there’s different varieties of quizzing and it all is about just getting students to retrieve and identify example after example. It brings in also principles such as interleaving, which we could return to in a little bit, but has to do with the sequencing of different examples… their spacing… So, that’s even planned out exactly how and when students encounter different things that they’re studying. It’s really wonderful. So, for example he and his colleagues put out a scholarship of teaching and learning article talking about how this approach was used effectively in veterinary medicine students who have to learn to identify poisonous plants that they’ll see around their practice. This is something that can be time-consuming and very tough, but they have some good data showing that this technology enhanced cognitively based approach really does work. That’s one example. Coincidentally, I’ve seen some other work in the literature, also on plant identification, where the instructors tagged plants in an arboretum… they went around and tagged them with QR codes… that students can walk up to a plant in the real environment with an iPad… hold the iPad over it… and it would immediately start producing quiz questions that were are specific to exactly the plants they were looking at.
So, those are some of the exciting things that people are taking and running with now that this principle is out there.

Rebecca: What I really love about the two stories that you just shared was the faculty are really designing their curriculum and designing the learning experiences with the students in mind… and what students need and when they need it. So, not only is it employing these cognitive science principles, but it’s actually applying design principles as well. It’s really designing for a user experience and thinking about the idea that if I need to identify a plant, being able to identify it in this situation in which I would need to identify it in makes it much more dynamic I think for a student… but also really meets them where they’re at and where they need it.

John: …and there’s so many apps out there now that will do the plant identification just from imagery without the QR code, that I can see it taking it one step further where they can do it in the wild without having that… so they can build it in for plants that are in the region without needing to encode that specifically for the application.

Michelle: I think you’re absolutely right once we put the technology in the hands of faculties who, as I said, they’re the one to know: “Where are my students at? Where are the weak points? Where are the gaps that they really need to bridge?” and that’s where their creativity is giving rise to all these new applications… and sometimes these can be low-tech as well… or also things that we can put in a face-to-face environment… and I’d like to to share just some experiences that I’ve had with this over the last few semesters.

In addition to trying to teach online with a lot of technology, I also have in my teaching rotation a small required course in research methods in psychology which can be a real stumbling block… the big challenge course… it’s kind of a gateway course to continued progress in our major. So, in this research methods course, some of the things that I’ve done around assessment and testing to really try again to stretch that retrieval practice idea… to make assessments really a more dynamic part of the course and more central part of the course… to move away from that idea that tests are just this kind of every now and again this panic mode opportunity for me to kind of measure in sorts of students and judge them… to make good on that idea that tests are part of learning. So, here’s some of the things that I try to do. For one thing, I took time out of the class almost every single class meeting as part of the routine to have students first of all generate quiz questions out of their textbook. So, we do have a certain amount of foundational material in that course as well as a project and a whole lot of other stuff is going on. So they need to get that foundational stuff.

Every Tuesday they would come in and they knew their routine: you get index cards and you crack your textbook and you generate for me three quiz questions. Everybody does it. I’m not policing whether you read the chapter or not. It’s active… they’re generating it… and also that makes it something like frequent quizzing. That’s a great practical advantage for me since I’m not writing everything. They would turn those in and I would select some of my favorites I would turn those into a traditional looking paper quiz and hand that out on Thursday. I said “Hey, take this like a realistic quiz.” I had explained to them that quizzes can really boost their learning, so that was the justification for spending time on it and then I said: “You know what? I’m not going to grade it either. You take it home because this is a learning experience for you. It’s a learning activity.” so we did that every single week as those students got into that routine.

The second thing that I did to really re-envision how assessment testing and quizzing worked in this particular course, was something inspired by different kinds of group testing and exam wrapper activities I’ve seen, particularly coming out of the STEM field, where there’s been a lot of innovation in this area. What I would do is… we had these high stakes exams at a few points during the semester. But, the class day after the exam, we didn’t do the traditional “Let’s go over the exam.” [LAUGHTER] That’s kind of deadly dull, and it just tends to generate a lot of pushback from students… and as we know from the research, simply reviewing… passing your eyes over the information… is not going to do much to advance your learning. So, what I would do is… I would photocopy all those exams, so it has a secure copy. They were not graded. I would not look at this before we did this… and I would pass everybody’s exams back to them along with a blank copy of that same exam. I assigned them to small groups and I said “Okay, here’s your job. Go back over this exam, fill it out as perfectly as you can as a group, and to make it interesting I said I will grade that exam as well, the one you do with your group, and anything you get over 90% gets added to everybody’s grade. This time it was open book, it was open Google, it was everything except you can’t ask me questions. So, you have each other and that’s where these great conversations started to happen. The things that we always want students to say. So, I would eavesdrop and hear students say “Oh, well you know what, I think on this question she was really talking about validity because reliability is this other thing…” and they’d have a deep conversation about it. I’m still kind of going back through the numbers to see what are the impacts of learning? Are there any trends that I can identify? But, I will say this: in the semesters that I did this, I didn’t have a single question ever come back to me along the lines of “Well, this question was unclear. I didn’t understand it. I think I was graded unfairly.” it really did shut all that down and again extended the learning that I feel students got out of that. Now it meant a big sacrifice of class time, but I feel strongly enough about these principles that I’m always going to do this in one form or another anytime I can can in face-to-face classes.

Rebecca: This sounds really familiar, John.

John: I’ve just done the same, or something remarkably similar, this semester, in my econometrics class which is very similar to the psych research methods class. I actually picked it up following a discussion with Doug McKee. He actually was doing it this semester too. He had a podcast episode on it. It sounded so exciting, I did something… a little bit different. I actually graded it but I didn’t give it back to them because I wanted to see what they had the most trouble with, and then I was going to have them only answer the ones in a group that they struggled with… and it turned out that that was pretty much all them anyway. So, it’s very similar to what you did except I gave them a weighted average of their original grade and the group grade and all except one person improved and the one person’s score went down by two points because the group grade was just slightly lower… but he did extremely well and he wasn’t that confident. The benefits to them of that peer explanation and explaining was just tremendous and it was so much more fun for them and for me and, as you said, it just completely wiped out all those things like “Well, that was tricky” because when they hear their peers explaining it to them the students were much more likely to respond by saying “Oh yeah, I remember that now” and it was a wonderful experience and I’m gonna do that everywhere I can.

In fact. I was talking about it with my TA just this morning here at Duke and we’re planning to do something like that in our classes here at TIP this summer, which i think is somewhat familiar to you from earlier in your academic career.

Michelle: That is right we do have this connection. I was among, not the very first year, but I believe the second cohort of Talent Identification Program students who came in, I guess you would call it now, middle school (back then, it was called junior high) and what a life-transforming experience. We’ve had even more opportunities to talk about the development of all these educational ideas through that experience.

John: That two-stage exam is wonderful and it’s so much more positive… because it didn’t really take, in my class, much more time, because I would have spent most of that class period going over the exam and problems they had. But the students who did well would have been bored and not paying much attention to it; the students who did poorly would just be depressed and upset that they did so poorly… and here, they were actively processing the information and it was so positive.

Michelle: That’s a big shift. We really have to step back and acknowledge that, I think. that is a huge shift in how we look at assessment, and how we think about the use of class time… and it’s not just “Oh my gosh, I have to use every minute to put such content in front of the students.” Just the fact that more of us are making that leap, I think, really is evidence this progress is happening… and we see also a lot of raised consciousness around issues such as learning styles. That’s another one that, when I go out and speak to faculty audiences, 10 years ago you would get these shocked looks or even very indignant commentary when you say “Ok, this idea of learning styles, in the sense that say there are visual learners, auditory learners, what I call sensory learning styles (VAK is another name it sometimes goes by). The idea that that just holds no water from a cognitive point of view…” People were not good with that, and now when I mentioned that at a conference, I get the knowing nods and even a few groans… people like “Oh, yeah. we get that. Now, K-12, which I want to acknowledge it’s not my area, but I’m constantly reminded by people across the spectrum that it’s a very different story in K-12. So, setting that aside… but this is what I’m seeing… that faculty are realizing… they’re saying “Oh, this is what the evidence says…” and maybe they even take the time to look at some of the really great thinkers and writers who put together the facts on this. They say “You know what? I’m not going to take my limited time and resources and spend that on this matching to styles when the styles can’t even be accurately diagnosed and are of no use in a learning situation. So, that’s another area of real progress.

Rebecca: What I am hearing is not just progress here in terms of cognitive science, but a real shift towards really thinking about how students learn and designing for that rather than something that would sound more like a penalty for grade like “Oh, did you achieve? Yes or no…” but, rather here’s an opportunity if you didn’t achieve to now actually learn it… and recognize that you haven’t learned it, even though it might seem really familiar.

John: Going back to that point about learning styles. It is spreading in colleges. I wish it was true at all the departments at our institution, but it’s getting there gradually… and whenever people bring it up, we generally remind them that there’s a whole body of research on this and I’ll send them references but what’s really troubling is in my classes the last couple years now, I’ve been using this metacognitive cafe discussion forum to focus on student learning… and one of the week’s discussions is on learning styles and generally about 95 percent of the students who are freshmen or sophomores (typically) come in with a strong belief in learning styles… where they’ve been tested multiple times in elementary or middle school… they’ve been told what their learning styles are… they’ve been told they can only learn that way… It discourages them from trying to learn in other ways and it does a lot of damage… and I hope we eventually reach out further so that it just goes away throughout the educational system.

Rebecca: You’ve worked in your classes, Michelle, haven’t you to help students understand the science of learning and use that to help students understand the methods and things that you’re doing>

Michelle: Yes, I have. I’ve done this in a couple of different ways. Now, partly, I get a little bit of a free pass in some of my teaching because I’m teaching the introduction to psychology or I’m teaching research methods where I just happen to sneak in as the research example will be some work on say attention or distraction or the testing effect. So, I get to do it in those ways covertly. I’ve also had the chance, although it’s not on my current teaching rotation… I’ve had the chance to also take it on as in freestanding courses. As many institutions are doing these days… it’s another trend… and what Northern Arizona University, where I work, has different kinds of freshmen or first-year student offering for courses they can take, not in a specific disciplinary area, but that really crossed some different areas of the student success or even wellbeing. So, I taught a class for awhile called Maximizing Brain Power that was about a lot of these different topics. Not just the kind of very generic study skills tip… “get a good night’s sleep…” that kind of thing… but really some again more evidence-based things that we can tell students and you can really kind of market it… and I think that we do sometimes have to play marketers to say “Hey, I’m going to give you some inside information here. This is sort of gonna be your secret weapon. So, let me tell you what the research has found.”

So, those are some of the things that I share with students… as well as when the right moment arises, say after an exam or before their first round of small stakes assessments, where they’re taking a lot of quizzes… to really explain the difference between this and high stakes or standardized tests they may have taken in the past. So, I do it on a continuing basis. I try to weave it into the disciplinary aspect and I do it in these free-standing ways as well… and I think here’s another area where I’m seeing this take hold in some different places… which is to have these free-standing resources that also just live outside of a traditional class that people can even incorporate into their courses… if say cognitive psychology or learning science isn’t their area… that they can bring in, because faculty really do care about these things. We just don’t always have the means to bring them in in as many ways as we would like.

John: …and your Attention Matters project was an example of that wasn’t it? Could you tell us a little bit about that?

Michelle: Oh, I’d love to… and you know this connects to what it seems to be kind of an evergreen topic in the teaching and learning community these days, which is the role of distracted students… and I know this past year there just have been these one op-ed versus another. There’s been some really good blog posts by some people I really like to follow in the teaching and learning community such as Kevin Gannon talking about “Okay, do you have laptops in the classroom? and what happens when you do?” and so I don’t think that this is just a fad that’s going away. This is something that the people do continue to care about, and this is where the attention matters project comes in.

This was something that we conceptualized and put together a couple years ago at Northern Arizona University with myself, and primarily I collaborated with a wonderful instructional designer who also teaches a great deal… John Doherty. So, how this came about is I was seeing all the information on distraction… I’m really getting into this as a cognitive psychologist and going “Wow, students need to know that if they’re texting five friends and watching a video in their class. It’s not going to happen for them.” I was really concerned about “What can I actually do to change students minds?” So, my way of doing this was to go around giving guests presentations in every classes where people would let me burn an hour of their class time… and not a very scalable model… and John Doherty respectfully sat through one my presentations on this and then he approached me and said “Look, you know, we could make a module and put this online… and it could be an open access within the institution module, so that anybody at my school can just click in and they’re signed up. We could put this together. We could use some really great instructional design principles and we could just see what happens… and I bet more people would take that if it were done in that format. We did this with no resources. We just were passionate about the project and that’s what we did. We had no grant backing or anything. We got behind it. So, what this is is about a one- to two-hour module that, it’s a lot like a MOOC in that it there’s not a whole lot of interaction or feedback, but there are discussion forums and it’s very self-paced in that way… so one- to two-hour mini MOOCs that really puts at the forefront demonstrations and activities… so we don’t try to convince students about problems with distraction and multitasking… we don’t try to address that just by laying a bunch of research articles on them… I think that’s great if this were a psychology course, but it’s not. So, we come at it by linking them out to videos, for example, that we were able to choose, that we feel really demonstrate in some memorable ways what gets by us when we aren’t paying attention… and we also give students some research-based tips on how to set a behavioral plan and stick to it… because just like with so many areas of life, just knowing that something is bad for you is not enough to really change your behavior and get you not to do that thing. so we have students talking about their own plans and what they do when, say, they’re having a boring moment in class, or they’re really really tempted to go online while they’re doing homework at home. What kinds of resolutions can they set or what kind of conditions can make that that will help them accomplish that. Things like the software blockers… you set a timer on your computer and it can lock you out of problematic sites… or we learned about a great app called Pocket Points where you actually earn spendable coupon points for keeping your phone off during certain hours. This is students talking to students about things that really concern them and really concern us all because I think a lot of us struggle with that.

So, we try to do that… and the bigger frame for this as well is this is, I feel, a life skill for the 21st century… thinking about how technology is going to be an asset to you and not detract from what you accomplish in your life. What a great time to be reflecting on that, when you’re in this early college career. so that’s what we try to do with the project…and we’ve had over a thousand students come through. They oftentimes earn extra credit. Our faculty are great about offering small amounts of extra credit for completing this and we’re just starting to roll out some research showing some of the impacts… and showing it in a bigger way just how you can go about setting up something like this.

Rebecca: I like that the focus seems to be on helping students with a life skill rather than using technology is just a blame or an excuse. We’re in control of our own behaviors and taking ownership over our behaviors is important rather than just kind of object blaming.

Michelle: So, looking at future trends, I would like to see more faculty looking at it in the way that you just described, Rebecca, as this is a life skill and it’s something that we collaborate on with our students… not lay down the law… because, after all, students are in online environments where we’re not there policing that and they do need to go out into work environments and further study and things like that. So, that’s what I feel is the best value. For faculty who are looking at this, if they don’t want to do… or don’t have the means to do something really formal like our Attention Matters approach, just thinking about it ahead of time… I think nobody can afford to ignore this issue anymore and whether you go the route of “No tech in my classroom” or “We’re going to use the technology in my classroom“ or something in between… just reading over, in a very mindful way, not just the opinion pieces, but hopefully also a bit of the research, I think, can help faculty as they go in to deal with this… and really to look at it in another way, just to be honest, we also have to consider how much of this is driven by our egos as teachers and how much of it is driven by a real concern for student learning and those student life skills. I think that’s where we can really take this on effectively and make some progress when we are de-emphasizing that ego aspect and making sure that it really is about the students.

John: We should note there’s a really nice chapter in this book called Minds Online: Teaching Effectively with Technology that deals with these types of issues. It was one of the chapters that got our faculty particularly interested in these issues… on to what extent technology should be used in the classroom… and to what extent it serves as a distraction.

Michelle: I think that really speaks to another thing which I think is an enduring trend… which is the emphasis on really supporting the whole student in success and what we’ve come to call academic persistence… kind of a big umbrella term that has to do with, not just succeeding in a given class, but also being retained… coming back after the first year. As many leaders in higher education point out, this is as a financial issue. As someone pointed out, it does cost a lot less to hang on to the students you have instead of recruiting more students to replace ones who are lost. This is, of course, yet another really big shift in mindset of our own, because after all we did used to measure our success by “Hey, I flunked this many students out of this course” or” Look at how many people have to switch into different majors…our major is so challenging…”

So, we really have turned that thinking around and this does include faculty now. I think that we did used to see those silos. We had that very narrow view of “I’m here to convey content. I’m here to be an expert in this discipline, and that’s what I’m gonna do…” and sure, we want to think about things like do students have learning skills? Do they have metacognition? Are they happy and socially connected at the school? Are they likely to be retained so that we can have this robust university environment?

We had people for that, right? It used to be somebody else’s job… student services or upper administration. They were the ones who heard about that and now I think on both sides we really are changing our vision. More and more forward-thinking faculty are saying “You know what? Besides being a disciplinary expert, I want to become at least conversant with learning science. I want to become at least conversant with the science of academic persistence…” There is a robust early literature on this and that’s something that we’ve been working on at NAU over this past year as well… kind of an exciting newer project that I like very much. We’ve started to engage faculty in a new faculty development program called Persistence Scholars and this is there to really speak to people’s academic and evidence-based side, as well as get them to engage in some perspective-taking around things like the challenges that students face and what it is like to be a student at our institution. We do some really selected readings in the area we look at things like mindset… belongingness… these are really hot areas in that science of persistence… in that emerging field. But, we have to look at it in a really integrated way.

It’s easy for people to say just go to a workshop on mindset and that’s a nice concept, but we wanted to think about it in this bigger picture… really know what are some of the strengths of that and why? Where do these concepts come from? What’s the evidence? That’s something that I think is another real trend and I think as well we will see more academic leaders and people in staff and support roles all over universities needing to know more about learning science. There are still some misconceptions that persist, as we’ve talked about. We’re making progress in getting rid of some of these myths around learning, but I will say… I’m not gonna name any names… but, every now and again I will hear from somebody who says “Oh well, we need to match student learning styles” or “Digital natives think differently, don’t you know?” and I have to wonder whether that’s a great thing. I mean, these are oftentimes individuals that have the power to set the agenda for learning all over a campus. Faculty need to be in the retention arena and I think that leaders need to be in the learning science arena. The boundaries is breaking down and it’s about time.

Rebecca: One of the things that I thought was really exciting with the reading groups that we’ve been having on our campus… that we started with your book, but then we’ve read Make it Stick and Small Teaching since… is that a lot of administrators in a lot of different kinds of roles engaged with us in those reading groups, it wasn’t just faculty. There was a mix of faculty, staff, and some administrators, and I think that that was really exciting. For people who don’t have the luxury of being in your persistence scholar program, what would you recommend they read to get started to learn more about the science of persistence?

Michelle: I really, even after working with this for quite some time, I loved the core text that we have in that program, which is Completing College by Vincent Tinto. It’s just got a great combination of passionate and very direct writing style. So, there’s no ambiguity, there’s not a whole lot of “on the one hand this and on the other hand that.” It’s got an absolutely stellar research base, which faculty of course appreciate… and it has a great deal of concrete examples. So, in that book they talk about “okay, what does it mean to give really good support to first semester college students? What does that look like?” and they’ll go out and they’ll cite very specific “Here’s a school and here’s what they’re doing… here’s what their program looks like… here’s another example that looks very different but gets at the same thing.” So, that’s one of the things that really speak to our faculty… that they really appreciated and enjoyed.

I think that as well we tested good feedback about work that’s come out of the David Yeager and his research group on belongingness and lay theories, and lay theories is maybe a counterintuitive term for kind of a body of ideas about what students believe about academic success and why some people are successful and others are not and how those beliefs can be changed sometimes through relatively simple interventions and when it happens we see great effects such as the narrowing of achievement gaps among students who have more privilege or less privileged backgrounds… and that’s something that, philosophically, many faculty really really care about but they’ve never had the chance to really learn “Okay, how can I actually address something like that with what I’m doing in my classroom, and how can I really know that the things that I’m choosing do have that great evidence base…”

John: …and I think that whole issue is more important now and is very much a social justice issue because, with the rate of increase we’ve seen in college cost inflation, people who start college and don’t finish it are saddled with an awfully high burden of debt. The rate of return to a college degree is the highest that we’ve ever seen and college graduates end up not only getting paid a lot more but they end up with more comfortable jobs and so forth… and if we really want to move people out of poverty and try to reduce income inequality, getting more people into higher education and successfully completing higher education is a really important issue. I’m glad to see that your institution is doing this so heavily and I know a lot of SUNY schools have been hiring Student Success specialists. At our institution they’ve been very actively involved in the reading group, so that message is spreading and I think some of them started with your book and then moved to each of the others. So, they are working with students in trying to help the students who are struggling the most with evidence-based practices …and I think that’s becoming more and more common and it’s a wonderful thing.

Rebecca: So, I really liked Michelle that you were talking about faculty getting involved in retention and this idea of helping students develop persistence skills, and also administrators learning more about evidence-based practices. There’s these grassroots movements happening in both of these areas. Can you talk about some of the other grassroots movements that are working toward, or efforts that faculty are making to engage students and capture their attention and their excitement for education?

Michelle: Right, and here I think a neat thing to think about too is just it’s the big ambitious projects… the big textbook replacement projects or the artificial intelligence informed adaptive learning systems… those are the things that get a lot of the press and end up in The Chronicle of Higher Education that we read about… But, outside of that, there is this very vibrant community and grassroots led scene of developing different technologies and approaches. So, it really goes back for a while. I mean, the MERLOT database that I do talk about in Minds Online has been trove for years of well hidden gems that take on one thing in a discipline and come at it from a way that’s not just great from a subject-matter perspective but brings up the new creative approaches. In the MERLOT database, for example, there’s a great tutorial on statistical significance and the interrelationship between statistical significance and issues like simple sizes. You know, that’s a tough one for students, but it has a little animation involving a horse and a rider that really turns it into something that’s very visual… that’s very tangible… and it really actually tying into analogies, which is a well-known cognitive process that can support the advancement of learning something new. There is something on fluid pressures in the body that was treated for nursing students by nurses, and it’s got an analogy of a soaker hose that this is really fun and is actually interactive. So, those are the kinds of things. The PhET project, P-h-E-T which comes out of University of Colorado, that has been around for a while… again, faculty-led and a way to have these very useful interactive simulations for concepts in physics and chemistry. So, that’s one. CogLab, that’s an auxiliary product that I’ve used for some time in like hundred psychology courses that simulates very famous experimental paradigms which are notoriously difficult to describe on stage for cognitive psychology students. That started out many years ago as a project that very much has this flavor of “We have this need in our classroom. We need something interactive. There’s nothing out there. Let’s see what we can build.” It has since then picked up and turned into a commercial product, but that’s the type of thing that I’m seeing out there.

Another thing that you’ll definitely hear about if you’re circulating and hearing about the latest project is virtual reality for education. So, with this it seems like, unlike just a few years ago, almost everywhere you visit you’re going to hear that “Oh, we’ve just set up a facility. We’re trying out some new things.” This is something that I also heard about when I was talking to people when I was over in China. So, this is an international phenomenon. It’s going to pick up steam and definitely go some places.

What also strikes me about that is just how many different projects there are. Just when you’re worried that you’re going to be scooped because somebody else is going to get there first with their virtual reality project you realize you’re doing very very different things. So, I’ve seen, for example, it used in a medical application to increase empathy among medical students… and I took a six or seven minute demonstration that just was really heart-rending, simulating the patient experience with a particular set of sensory disorders… and at Northern Arizona University we have a lab that is just going full-steam in coming up with educational applications such as interactive organic chemistry tutorial that is is just fascinating. We actually completed a pilot project and are planning to gear up a much larger study next semester looking at the impacts of this. So, this is really taking off for sure.

But, I think there are some caveats here. We still really need some basic research on this… not just what should we be setting up and what the impacts are but how does this even work? In particular, what I would like to research in the future, or at least see some research on, is what kinds of students… what sort of student profile… really gets the most out of virtual reality for education. Because amidst all the very breathless press that’s going on about this now and all the excitement, we do have to remember this is a very, very labor intensive type of resource to set up. You’re not just going to go home and throw something together for the next week. It takes a team to build these things and to complete them as well. If you have, say, a 300 student chemistry course (which is not atypical at all… these large courses), you’re not going to just have all of them spend hours and hours and hours doing this even with a fairly large facility. It’s a very hands-on thing to guide them through this process, to provide the tech support, and everything else.

So, I think really knowing how we can best target our efforts in this area, so that we can build the absolute best, with the resources we have, and maybe even target and ask the students who are most likely to benefit. I think those are some of the things that we just need to know about this. So, it’s exciting for somebody like me who’s in the research area. I see this as a wonderful open opportunity… but those are some of the real crossroads we’re at with virtual reality right now.

Rebecca: I can imagine there’s a big weighing that would have to happen in terms of expense and time and resources needed to startup versus what that might be saving in the long run. I can imagine if it’s a safety thing that you want to do a virtual reality experience, like saving people’s lives and making sure that they’re not going to be in danger as they practice particular skills, could be a really good investment in these… spending the resources to make that investment… or if it’s a lot of travel that would just be way too expensive to bring a bunch of students to a particular location… but you could virtually… it seems like it would be worth the start-up costs and those are just two ideas off the top of my head where it would make sense to bend all of that resource and time.

John: …and equipment will get cheaper. Right now, it’s really expensive for computers that have sufficient speed and graphics processing capability and the headsets are expensive, but they will come down in price, but as you said, it’s still one person typically and one device… so it doesn’t scale quite as well as a lot of other tools or at least not at this stage.

Rebecca: From what I remember, Michelle, you wrote a blog post about [a] virtual reality experience that you had. Can you share that experience, and maybe what stuck with you from that experience?

Michelle: Right, so I had the opportunity, just as I was getting to collaborate with our incredible team at the immersive virtual reality lab at NAU… one of the things I was treated to was about an hour and a half in the virtual reality setup that they have to explore some of the things that they had… Giovanni Castillo, by the way, is creative director of the lab and he’s the one who was so patient with me through all this. We tried a couple of different things and of course there’s such a huge variety of different things that you can do.
There’s a few things out there like driving simulators that are kind of educational… they’re kind of an entertainment… but he was just trying to give me, first of all, just a view of those… and I had to reject a few of them… I will say, initially, because I am one of the individuals who tends to be prone to motion sickness. So, that limits what I can personally do in VR and that is yet another thing that we’re gonna have to figure out. At least informally, what we hear is that women in particular tend to experience more of this. So, I needed, first of all, to go to a very low motion VR. I wasn’t gonna be whizzing through these environments. That was not going to happen for me. So, we did something that probably sounds incredibly simplistic, but it just touched me to my core… which is getting to play with Google Earth. You can spin the globe and either just pick a place at random or what Giovanni told me is… “You know, I’ve observed that when people do this, when we have an opportunity to interact with Google Earth, they all either go to where they grew up or they’ll go to someplace that they have visited recently or they plan to visit. So, I went to a place that is very special to me and maybe it doesn’t fit into either one of those categories neatly, but it’s my daughter’s University… her school… and I should say that this is also a different thing for me because my daughter goes to school in Frankfort, Germany… an institute that is connected to a Museum. So, I had only been to part of the physical facility… the museum itself… and it was a long time ago… and part of it was closer to the holiday. So, this is my opportunity to go there and explore what it looks like all over… and so, that was an emotional experience for me. It was a sensory experience… it was a social one… because we were talking the whole time… and he’s asking me questions and what kinds of exhibits do they have here… and what’s this part of it. So, that was wonderful. it really did give me a feel for alright, what is it actually like to be in this sort of environment?

I’m not a gamer. I don’t have that same background that many of our students have. So, it got me up to speed on that… and it did show me how just exploring something that is relatively simple can really acquire a whole new dimension in this kind of immersive environment. Now the postscript that I talked about in that blog post was what happened when I actually visited there earlier in the year. So, I had this very strange experience that human beings have never had before… which is from this… I don’t know whether to call it deja vu or what… of going to the settings and walking around the same environment and seeing the same lighting and all that sort of stuff that was there in that virtual reality environment… but this time, of course, with real human beings in it and the changes… the little subtle changes that take place over time, and so forth.

So, how does it translate into learning? What’s it going to do for our students? I just think that time is going to tell. It won’t take too long, but I think that these are things we need to know. But, sometimes just getting in and being able to explore something like this can really put you back in touch with the things you love about educational technology.

Rebecca: I think one of the things that I’m hearing in your voice is the excitement of experimenting and trying something… and that’s, I think, encouragement for faculty in general… is to just put yourself out there and try something out even if you don’t have something specific in mind with what you might do with it. Experiencing it might give you some insight later on. it might take some time to have an idea of what you might do with it, but having that experience, you understand it better… it could be really useful.

John: …and that’s something that could be experienced on a fairly low budget with just your smartphone and a pair of Google cardboard or something similar. Basically, it’s a seven to twelve dollar addition to your phone and you can have that experience… because there’s a lot of 3D videos and 3D images out there on Google Earth as well as on YouTube. So, you can experience other parts of the world and cultures before visiting… and I could see that being useful in quite a few disciplines.

Rebecca: So, we always wrap up with asking what are you going to do next?

Michelle: I continue to be really excited about getting the word out about cognitive principles and how we can flow those in to teaching face-to-face with technology… everything else in between. So, that’s what I continue to be excited about… leveraging cognitive principles with technology and with just rethinking our teaching techniques. I’m going to be speaking at the Magna Teaching with Technology Conference in October, and so I’m continuing to develop some of these themes… and I’m very excited to be able to do that. I’m right now also… we’re in the early stages of another really exciting project that has to do with what we will call neuromyth… So, that may be a term that you’ve turn across in some of your reading. It’s something that we touched on a few times, I think, in our conversation today… the misconceptions that people have about teaching and learning and how those can potentially impact the choices we make in our teaching. So, I’ve had the opportunity to collaborate with this amazing international group of researchers who’s headed up by Dr. Kristen Betts of Drexel University… and I won’t say too much more about it other than we have a very robust crop of survey responses that have come in from, not just instructors, but also instructional designers and administrators from around the world. So, we’re going to be breaking those survey results down and coming up with some results to roll out probably early in the academic year and we’ll be speaking about that at the Accelerate conference, most likely in November. That’s put out by the Online Learning Consortium. So, we’re right in the midst of that project and it’s going to be so interesting to see what has the progress been? What neuromyths are still out there and how can they be addressed by different professional development experiences. We’re continuing to work on the Persistence Scholars Program on academic persistence. So, we’ll be recruiting another cohort of willing faculty to take that on in the fall at Northern Arizona University. I am going to be continuing to collaborate and really work with and hear from John and his research group with respect to the metacognitive material that they’re flowing into foundational coursework and ways to get students up to speed with a lot of critical metacognitive knowledge. So, we’re going to work on that too… and I like to keep up my blog and work on shall we say longer writing project but we’ll have to stay tuned for that.

Rebecca: Sounds like you need to plan some sleep in there too.

[LAUGHTER]

John: Well, it’s wonderful talking to you, and you’ve given us a lot of great things to reflect on and to share with people.

Rebecca: Yeah. Thank you for being so generous with your time.

John: Thank you.

Michelle: Oh, thank you. Thanks so much. It’s a pleasure, an absolute pleasure. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

36. Peer instruction

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

33. The Marmots of Finance

In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, Alex Butler, a Professor of Finance at Rice University, joins us to discuss how rich imagery can be used to help students make connections and deepen their understanding.

Alex received the Rice University Presidential Mentoring Award and the George R. Brown Award for Superior Teaching in 2018. He is also the recipient of the Jones School’s Award for Scholarship Excellence in 2011 and 2012.

Show Notes

  • Duke Talent Identification Program (TIP)
  • Medina, J. (2011). Brain rules: 12 principles for surviving and thriving at work, home, and school. ReadHowYouWant.com.
  • Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
  • Alex’s web site at Rice

Transcript

Rebecca: In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, we’ll discuss how rich imagery can be used to help students make connections and deepen their understanding.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Alex Butler, a professor of finance at Rice University. Alex received the Jones School’s Award for Scholarship Excellence in 2011 and 2012. Alex teaches financial markets and corporate finance in the undergraduate MBA, MBA, and PhD programs. Sometime in the latter part of the last century, Alex and I spent three summers teaching introductory economics to highly gifted middle school and high school students at the Talent Identification Program at Duke University.
Welcome, Alex.

Alex: Oh, thank you. Thank you for having me, John.

Rebecca: Today’s teas are:

Alex: I am not drinking tea…. just water at this point.

Rebecca: Another one… epidemic.

John: My tea is ginger peach white tea.

Rebecca: I’ve Prince of Wales today.

John: Could you tell us a little bit about the courses that you normally teach?

Alex: Sure. I have, over the years, taught almost every course that we have… either here or at other schools. Right now what I teach (and I’m glad to finally have settled into a group of classes that are my classes year in and year out) and those classes are the undergraduate business finance class and PhD courses – one in corporate finance, and one that’s a topics class on causal inference. I really enjoy teaching the undergrad business finance class in particular because I was an undergraduate student here at Rice many, many years ago, and so it’s fun to be back and be on the other side of the podium teaching the students. One of the things that I really like about teaching the undergraduates here is that they are able to appreciate my dorky sense of humor.

Rebecca: …which is indeed a very important thing.

Alex: Agreed. For years, I tried to suppress my dorky sense of humor in the classroom… ad tried to teach the course sort of straight up and dry, and it made everybody miserable… the students…. me… and so, eventually over time, as I grew more and more confident teaching, I started incorporating more and more jokes here and there… and then I allowed them to become more elaborate, and then before you know it, I’ve developed a full-blown dad sense of humor and full-on dork mode.

John: That’s actually why we invited you here. We read a little bit about that in a teaching award you just received. What do students expect the course to be about when they take an introductory finance course?

Alex: At Rice, the students have a good grapevine of information about what courses are about and so, at this point, the students come in with pretty solid expectations that line up with what the class actually is… and whereas some students are hoping that I’m going to teach them how to become millionaires in the stock market, what the course mostly is about is that’s very hard to become a millionaire investing in the stock market and how to make decisions in a corporate setting that will maximize firm value.

Rebecca: So you mentioned that some students come in with this misperception of becoming a millionaire. What are some of the strategies that you use to dispel that myth?

Alex: I should say this class is one that is a fairly standard course that’s taught in lots of different business schools all over the country, all over the world. One way that I do it differently is in the order of material that I cover… and so the very first week of class, I talk about market efficiency. That’s the notion that it’s very difficult to earn abnormal returns in the stock market. In other words, you can’t beat the stock market very easily, unless it just happens to be by luck… and so I come in the first week talking about the reasons why it’s very difficult to beat the stock market… and the reason why is because there are literally tens of thousands of people who have more money and more resources and who are faster and are doing this as a full-time job who are also trying to find the stocks that are mispriced… and so unless you are the investor who is faster, and smarter, and has more money to throw at the trading strategy that you think of, it’s very very difficult to beat the other 10,000 people… and so I spend the first week of class introducing this concept and then providing copious amounts of evidence… research that highlights how difficult it is for lay people to earn abnormal returns in the stock market. That sets the setting for the rest of the course, which is this notion of how competition affects prices and how that feeds through to other applications and the corporate domain as opposed to the financial markets domain.

Rebecca: What made you switch the order?

Alex: What I realized after teaching this class for a long time is that most of corporate finance, most of business finance, is about discounting cash flows back to the present at some appropriate discount rate, and I found I was having a hard time getting the students to understand the notion of what interest rate, what discount rate, should be used to make these cash flows that are spread through time to get a present value equivalent. I figured if I started with some aspect of that, where that rate comes from, where those prices come from. but that would make the rest of the course easier for them to understand as we go through. So, that’s why. Most people wait until after they’ve introduced things like: “oh, portfolio theory” and “capital asset pricing model” and other asset pricing concepts before they talk about market efficiency… and so I just sort of turned that around backwards… and I open the course with that. So, point number one is pedagogical, and point number two is that the lectures that I do on market efficiency are really fun, and so I really like starting the course off with something that’s really fun. So, we can talk about stories of insider trading, and we can talk about stories of surprise announcements and how that affects stock returns, and we can talk about “oh, so you think you’re going to beat the market, well let me explain to you how hard it is and the reasons why …” also were very fun. So, we can spend the first week talking about fun stuff. It’s a giant bait and switch.

[LAUGHTER]

I lure them in with fun stuff and then beat them to death for the next fourteen weeks with discounted cash flow analysis.

Rebecca: It also sounds like it’s a good way to motivate students. Not only is it fun, but it gets students motivated and interested and they buy into the class, which I wouldn’t discount that. I think that’s an important task.

Alex: Oh, absolutely.

John: …discounting in a different sense, but…

[LAUGHTER]

Rebecca: Yeah, sorry… I’m a designer

John: …but it also starts a class by dispelling that myth that they’re going to learn tools that will allow them to become really wealthy in financial investments and so forth. So, you’re setting it up by getting rid of that myth and they’re ready to start actually learning without having that at the background.

Alex: Correct.

Rebecca: I really like the idea of just meeting students where they’re at. If this is where some students are coming with, and maybe they’re super motivated in some ways but not in others, that you just tackle both of those in one week.

Alex: That’s right.

Rebecca: Cool.
You recently won a 2018 George R. Brown Award for superior teaching, and the Georgia R. Brown Awards are based on an interesting selection process. Can you tell us a little bit about that?

Alex: Yeah, so what the university does is they solicit feedback from recent alumni, people who graduated, I think, it’s two and five years ago. Now, they have graduated and they’re looking back what teachers would they want to see when these awards… and so, it’s really a neat honor, because once the students are two, three, four. five years out, they’re not responding to the short-term incentives that some professors used to gain teaching evaluations and things like that, they’re looking back and they’re actually remembering what the course was, what the professor was like, and whether it was meaningful to them. It’s really been just a phenomenal honor, and one that I honestly thought that I would never win. So, it’s been really special to have gotten that award.

John: It’s a really interesting idea to have an award given in that way, because it focuses on that long-term learning, that you’ve taught them something that’s going to benefit them later rather than, as you said, something that people do short term. I know some faculty, when we’re doing course evaluations, will give out cookies or other things just to boost their short-term course evaluations. But, that’s not going to pay off very much two to five years after graduation.

Alex: That’s correct. I went through the cookies and brownies phase myself years ago and decided that that just wasn’t who I wanted to be, so…. Now, we’re on to just just the pedagogy at this point.

Rebecca: I think there’s a couple of interesting things that I see about those awards, too, and one is that is more meaningful probably to win it because you can feel the impact but two it meshes really well with what we know about evidence-based practices in that students generally don’t like them while they’re happening, but they have longer-term effects… and that students tend to learn that material and transfer that material later on.

Alex: That’s absolutely correct, and I’m a big subscriber to that basic view that students often don’t like things that make them uncomfortable, and learning new and difficult things is uncomfortable. I think one could make students happier in the short run by giving them lots of little assignments where they feel like they’re making progress every day but they’re not actually being challenged, they’re not actually being pushed, and so instead holding them to very high standards for demonstrating their mastery of material while that is very uncomfortable for some of them, it is that that makes them better students and better scholars of that topic.

John: It reminds me of Bjork and Bjork, in their writings on “desirable difficulties.” that the most learning occurs when students are faced with feasible challenges… that if things are too easy, they get bored, and they may be happy with the course if they don’t have to struggle much… but they learn the most when they’re struggling but they see it’s possible.

We’ve heard that this award, though, based on some of the write-ups we’ve seen at your institution, may somehow be tied to marmots, wolves, and The Princess Bride. What do they all have to do with finance?

Alex: Several years ago, I was reading some books I thought would be…. some I thought it’d be helpful for my teaching, some that I just thought would be helpful for me, and one of the books was a book called Brain Rules by John Medina, and it’s basically a book that tries to take cognitive science, brain science, down to a level that lay people can understand and gives several rules of thumb of how the brain works and why the brain works the way it does… and as I’m reading this, reading it mostly for my own consumption so that I can be a better researcher and more thoughtful person, smarter, that sort of thing, I realized “Gosh, a lot of the rules here applied very directly to teaching, at least in the lecture format that I use in most of my courses…” and so one of the things that really stood out to me is how people learn better, remember better, I guess I should say, when they see images images that relate to whatever the topic at hand is. So, text maybe a PowerPoint slide with text, and you remember X percent but if you see an image, you remember much more of that material later on… and so this gave me just a license to, all of a sudden, start having fun on a completely new dimension. Reading this book and sort of embracing the notion that I could maybe help students remember the material better simply by infusing my lecture slides with some relevant images, was just eye opening for me… because now I could take my completely dry, boring slides with words and numbers and equations and now I can have fun with them…. and have this entire new dimension, a degree of freedom, to play around with what the slides are gonna look like and how the students are going to experience them. So, that’s sort of the extensive margin. The first part is: “Hey, I need images…” so the intrinsic margin is “what kind of images do I need?” “what will work best?” and Brain Rules comes to the rescue there again… and it says people respond to images that are faces, that are things that are scary, things that they can eat, things that might want to eat them, and things that they might want to mate with. Well, that last one’s kind of out for most of my lecture slides… [LAUGHTER] I can’t really incorporate that very directly. But it got me thinking “okay, what’s scary?” So, I started looking around for images of things that are scary to people… and I found this great image of this really, just terrifying, snarling wolf. I’m like “Ah, I’m gonna use that to get people’s attention…” But, as I started thinking about it, I wanted to lead into the wolf a little bit… and so the main prey of wolves in North America are marmots… yellow-bellied marmots… and so I found this great image of a little cute yellow-bellied marmot sitting on a rock somewhere in the mountain somewhere… and now when I come in to teach what could be the absolute driest lecture of the entire course which is time value of money and understanding how to discount cash flows (it’s the tool that everything else builds on, so it’s incredibly important to get it right, but it’s also potentially incredibly technical and boring)… and so I start that lecture not with an equation not with numbers but with a giant image of a marmot filling the entire computer’s projection screen…. and I just leave it up there… and the class is all looking at it… and I look back at them… and I pick someone at random. I cold call… and I say: “So, Charlotte, what do you think?”

“What do you mean what do I think?”

“What is it?”

… and we go through a series of guesses, and the guesses range from just ludicrous things: “it’s a gopher.”

“No, no, clearly it’s not a gopher. Gophers are a lowland creature. This is obviously in the mountains.”

“It’s an otter.”

“No, no, no. Otters prefer marine habitats, and this is clearly not there.”

… and go through this for a while until usually somebody recognizes it as a marmot. I say “Yes, very good. Alright..”

…and so then on to the next slide and the next slide is this picture of George Soros with no caption, no explanation, but again filling the entire screen, here’s this giant picture of George Soros… and so I go back to the first person: “Charlotte what do you think about this one? You didn’t get the marmot, how about this one?” …and so invariably somebody will eventually guess it’s a hedge fund manager.

I say: “Yes, very good. That’s George Soros, a famous hedge fund manager.” then the next slide is the wolf, the snarling wolf, and so at this point the captions on the slides read “This is a yellow-bellied marmot,” “This is a hedge fund manager” … and then it’s obvious what the wolf is. People get that right away. So, Charlotte gets to redeem herself at that point.

[LAUGHTER]

“This is a wolf, one of the main predators of the yellow-bellied marmot,” and then the next slide is another picture of a marmot but this one looking somewhat quizzical and the caption here is: “Why do wolves eat marmots, but not hedge fund managers?” …and the answer that I propose is because hedge fund managers understand the time value of money, but marmots do not. Now, that obviously doesn’t actually follow, but the correlation is there… that part is true… and so I just sort of lean into that and the ridiculousness behind that statement and from there on the marmot becomes our time value of money mascot… and so every time throughout the course I introduced a new application of time value of money then I bring in another image of a marmot to sort of tie all that together. iIt’s hard to see that the tie that binds everything together is time value of money and so the marmot, the mascot, is the visual cue, that “oh, this isn’t special, this isn’t different, this is just another application of the same technique we’ve been doing over and over again.

John: That notion of using visual cues goes back to the Greeks who use it to remember long stories before there was much printed word, and one of the arguments is that it’s because visual imagery developed much earlier than the use of language and the things you described in terms of things that might eat you may be tied back to our evolutionary adaptation, and so we’re tying into things that evolve fairly early in the evolution of human beings.

Rebecca: …and it also is just that emotional response.

John: Right, when you trigger an emotional response and fear is a major one…

Rebecca: Yeah. mm-hmm

John: Do the wolf’s chase the Princess Bride? I don’t remember that…

Alex: Well no…. So, the Princess Bride lecture is where I talk about decision rules using discounted cash flow techniques…. and so this sort of standard playbook for any finance professor anywhere as you talk about net present value and internal rate of return and payback period and maybe a few other rules as well, you just sort of go through the… whatever they have in the book in the order in the book, and you talk about each one in turn. But, it’s a lot more fun if you can personify those.. and so that’s what I use the characters from Princess Bride to do… and so the main character, which is net present value, is personified as Wesley / (there’s a spoiler alert here – so you haven’t seen the movie Wesley is the Dread Pirate Roberts). I always show my kids my lecture slides and when I showed them that one my older daughter was mortified that I would ruin the Princess Bride movie for students by revealing that Wesley and the Dread Pirate Roberts are one and the same… and so from there, I personify the different rules based on the different characters in Princess Bride… and so internal rate of return which is sort of flashy and useful but has some flaws as Inigo Montoya; the payback period which is sort of a blunt instrument is Fezzak the Giant, and so forth… modified internal rate of return is the Cliffs of Insanity.

[LAUGHTER]

Rebecca: So, the important thing to ask then is do you have the spoiler alert at the beginning of your lecture.
AEX: I do now.

[LAUGHTER]

John: How old was your daughter at the time when she reacted to that?

Alex: About ten years old.

John: Okay. So, by the time students are in college, they’ve probably either seen it or they may be less likely to, so it probably doesn’t do quite as much damage.

Alex: I hope so, and in fact I even now encourage them to watch the movie before that lecture without really telling them why. So some of them do. Some of them ask around and figure out why, but that makes it more fun.

Rebecca: That was gonna be my follow-up… is that homework?

Alex: It’s the best homework they’ll ever have.

John: So, how have students responded to this? Do they remember this later?

Alex: Yes and no. for the Princess Bride lecture I believe that the students sort of uniformly enjoy that…. but I don’t have a good sense of how much that actually impacts the depth of their learning. For the marmots, it’s a completely mixed bag… some of the students really love it, and I really do lean hard on the fact that this is the mascot… this is the thing that ties it all together… and this is the visual cue so you will now recognize that when we do bond pricing “oh, here’s a marmot, that means it’s just time value of money…” “oh, when we do net present value, here’s a marmot showing up alongside our Princess Bride character it’s just time value of money” and so forth. Many of the students really enjoy that and grab on to it and some students, they’re not having any of it. They want boring… they want dry… and I’m afraid they’ve come to the wrong place. Been there and done that. I don’t do that anymore.

John: But that use of imagery is really common. People who work on developing memory… the memory palace type things where you tie specific concepts and bundles of concepts with chunks of item to key images, helps people remember things long term. I can see how it would be really effective.

Rebecca: I also could see that one of the things that students often struggle with when information is new to them is making those connections. So providing that visual cue like “here’s time to make a connection…” it’s actually really helpful, because those are the kinds of things that might seem really abstract and very separate if you’re not making an explicit. So, I like that you don’t even have to say explicitly like “this is the thing” you have by putting the image up there, and you’re prompting the students to predict what that connection is or challenging them to think of it on their own before you reveal what that connection is.

Alex: That’s right, and so what started off as just a fun way to get attention: “Hey, class is starting let’s all key in… Here’s a scary wolf.” So, now we’re all paying attention to the scary wolf because that’s how our brains are hard-wired. It’s now grown into this entire thing where throughout the entire class is a continuing callback.

Rebecca: It sounds to me like you must have some pressure every semester to have to have something new that you introduce into some sort of lecture so that there’s some anticipation.

Alex: I’ll be the first one to say that I shamelessly recycle all of my jokes…

[LAUGHTER]

… and so I’m always sort of terrified when I have a student who started the class last year but had to drop at the 6 week mark or the 8 week mark because whatever was going on their life, and then they’re back the next year. Like “oh man, you’re gonna get the exact same jokes with the exact same timing, the exact same patter.” It mostly seems to work fine, and I do continually try to incorporate new things. One of the nice things about teaching finance, although some of it is very static… the basic concept of time value of money is going to be there for basically forever and I won’t need to change the actual examples in those slides really ever… but a lot of the other material changes very rapidly. So, the notion of what does market efficiency mean? who can beat the market and win? what is the evidence? But for topics like market efficiency, for topics like financial markets, for topics like investment banking, those areas transform rapidly… and so I’m continually changing those lectures year after year because one of the big topics that I cover in my course that is not really traditional for an undergrad business finance class is how firms raise external capital. The reason why is because it’s one of my main research interests, and so I have lots of ideas of what I want the students to know, and lots of research that I can tie into the lectures that I give. For the past five or ten years financial markets, the regulation of financial markets, firms’ ability to raise external capital, has changed tremendously as regulation has changed and so I’m continually revamping that portion of the course basically every time that I’m breaking it out.

Rebecca: One thing that I can’t help but think is that you and John have some similar backgrounds in terms of content, so the time value of money that you keep referencing probably makes sense to you but I feel like our listeners might not know exactly what that is so maybe we should just take a minute and give a quick cap of that so people know.

Alex: Yeah, absolutely. So, if you have $100 and you invest it earning an interest rate of 5% per year, in a year how much money will you have?

Rebecca: One hundred and five?

Alex: One hundred and five dollars.

[LAUGHTER]

Rebecca: It’s not a trick question, right?

Alex: No, No, it’s not…

[LAUGHTER]
… and so, as you invest money it earns some rate of return and so then money in the future you have more, because it earns some positive rate of return, and conversely, would you rather have $100 today or $100 in a year? Well, the answer is $100 today, because you could invest that for that year and have the hundred dollars plus some additional return. So, you’d have a hundred and five dollars one year from now instead of the: “oh, I’m going to give you a hundred dollars a year from now” and that’s all the time value of money is.. and then it’s application of that over multiple cash flows and multiple periods where people start to lose track that it’s just math.

John: That’s the fun part.

Rebecca: Yeah. Yeah.

John: I also ask my students have they ever burned their mouth on a slice of pizza, and why don’t they just wait? ..and it’s one way of introducing the notion that we prefer things now to later. It’s a fairly important concept in economics and finance and it’s at the basis of finance.

Alex: Oh, absolutely… and I like the way of framing it as how patient you are. Are you willing to wait or are you impatient? …and so the way that I sometimes describe that in class is if you have a low discount rate that means that the future is worth about the same as the present and so you are patient. If you have a very high discount rate well then the future and the present are very different, so high discount rates → impatient, low discount rates → patient.

Rebecca: You realize that the irony of this whole thing is that I have these conversations with my husband all the time because he’s in finance, right? Yeah….

[LAUGHTER]

John: Ok, well, does that help explain it?

Rebecca: Yeah, yeah. I’ve had it explained to me many times.

[LAUGHTER]

I figured our listeners would need it.

Alex: Well, since we’re talking about the things that our students do, I’ll just share one example that I use in class that I continue to use even though it completely does not work with undergraduate students. It’s when I’m trying to introduce the concept of sunk costs… and so a sunk cost is some amount that you have paid… a cash flow that has happened in the past perhaps… and so once that is paid, you can’t get it back …and so it’s like the notion of “should he throw good money after bad” is another way of phrasing it… and so, what I used to say is, “Well, you go to a movie, you pay your fee to go in, and you decide it’s a terrible movie, should you keep watching to get your money’s worth or should you leave? and then a student pointed out to me one year that well, actually, if you go and you complain you can get your money back from the management.

[LAUGHTER]

Okay, different example. This is the one that never works on the undergrads. You go to an all-you-can-eat buffet and you have a choice of do you buy one plate for this amount or do you pay a little more to get the all-you-can-eat buffet and you decide to go for the all-you-can-eat. You pay the $10 for the all-you-can-eat instead of the $7 for the single plate… and you go and you fill up your plate and you eat… but you decide you’re full after just your first plate… but you’ve already paid that all-you-can-eat price. Should you go back for more? …and the intellectual scholarly answer is “Well, no, because the marginal benefit of eating more food is negative at this point because you’re full. Even though the marginal cost is zero.” But, for the students, the undergraduates, the marginal benefit of eating more food is always positive.

[LAUGHTER]

They view it as a sport. Dinner’s not over until they kick me out.

Rebecca: I think that’s really important to consider your audience and what works. So, I think that’s a really good demonstration of knowing your audience and why something might not work for a particular audience.

Alex: Yes, nonetheless, I still use that example… but I tell them ahead of time “I know this is not going to make sense to you. You’re going to push back on me. Nonetheless, I want to talk it through with you so that we can think about who has marginal benefits and marginal costs of what amount.”

John: Just last semester I used exactly the same thing of asking them how many of them would leave a movie theater if it was clear that they weren’t enjoying the movie and this time I had some people argue “I do that all the time.” So, yeah…

Rebecca: Movie theaters? Don’t you just have Netflix?

[LAUGHTER]

John: Okay, that’s actually a relevant point. In terms of the Princess Bride, we’re moving into a generation where many of your students might not have seen that. Do you have any other movie references that you might substitute in the future?

Alex: You know, I haven’t found the right one yet and, honestly, even if I found a good substitute, I’d really want to keep the Princess Bride, just to give them the incentive to watch that movie.

[LAUGHTER]

It is such a classic and such a gem. I really would like for them to all see that movie.

Rebecca: It’s an investment you’ve made.

[LAUGHTER]

Alex: That’s right.

Rebecca: How have faculty responded to the methods that you’ve been using?

Alex: Faculty are not surprised that I do the things that I do.

[LAUGHTER]

John: They’ve known you for a while.

Alex: They’ve known me for a while and I really lean into the corny pun bad jokes big time in class…. and I really I’m quite shameless about it.

John: That’s not new… I seem to remember that back at Duke when you were still in college.

Alex: …and it’s just a question of do I try to suppress that innate desire or not?

[LAUGHTER]

…and I’m at the stage in my career where “nope, not anymore.” You’re just gonna let it all out. My colleagues are not really surprised, but interestingly, one of my colleagues has adapted her lectures quite a bit to embracing this notion of adding images into the slide decks… and that’s my wife. She teaches business law and regulation of business. She was an attorney by training and I described her as a born-again economist because when she started teaching regulation of business she had to teach herself basically all of public choice economics. So, she got a couple of high-level textbooks and worked through them all but she and I regularly discuss teaching techniques and so she now has gone down the rabbit hole of finding that perfect image to highlight the point that you want to make… to have that really stretched metaphor that you can then call back throughout your course. So, it’s been a lot of fun to have her as a sounding board to go back and forth with.

Rebecca: I appreciate that you’ve brought design into the process.

Alex: Yeah.

John: Excellent. It’s something we all should probably do more of and think about more and certainly much more effective than those PowerPoint bulleted lists that are so common.

Alex: Yeah, a little bit of both actually goes a long way.

Rebecca: You’ve won some awards for your research. How do you maintain a balance between teaching and research?

Alex: It’s difficult. There are some ways in which research can feed directly into the teaching… and so my research that touches on market efficiency, some of that can come into the classroom. My research that touches on how firms raise external capital, some of that can come into the classroom. Now when I teach my PhD courses, those are heavily flavored by my research interests and preferences, but when I teach the undergraduate core finance class there’s not a whole lot that can flow back from that teaching into the research. So, that’s one of the downsides of teaching that class as opposed to perhaps a class that is more specialized or an elective or something that’s a little bit further downstream from the core class.

Rebecca: But at the same time, if it’s something that you teach routinely then the prep isn’t as difficult.

Alex: Absolutely. That’s true. It is hard to overstate how useful it is to teach a class a second time, or a third, or a fourth, or twentieth. Nonetheless, every single year, even though it’s the same class and mostly the same topics, I go through every slide, every lecture from the beginning every year.

Rebecca: But, I remember teaching a bigger selection of classes and one of the things that I’ve liked about my position at Oswego is that that suite of classes has gotten smaller… and then there’s a little less I’ll keep on top of to make sure that you have all that fresh information and what-have-you for classes.

Alex: Absolutely, and I’ve done the same thing. I’ve taught a variety of courses over the years and it’s been nice… as you describe it that suite of classes narrows, so that you have the same core group of classes that you’re teaching over and over again and you can start to specialize. You can really invest the time to get over the fixed costs of finding all those right images for the slides… to going through and taking time to invest in the design aspects of the lectures. That if you had four different courses every term, it would be incredibly difficult to find the time to do that effectively.

John: While you’re working with PhD students, you must do quite a bit of mentoring of them. How do you see the role of a faculty member as a mentor for graduate and undergraduate students?

Alex: Faculty vary widely on their views of how much mentoring PhD students should have… and so you have one model where it’s sink or swim… the PhD students are some of the smartest people in the world… they’re good students… they’ll figure stuff out… just point them in the right direction and let them go, and they’ll get there. That’s not the view that I subscribe to. Because I think we frequently overestimate just how much the doctoral students know, particularly about how the profession works. There’s no book for that. They can’t just go down to the library and find a textbook on how to be a good assistant professor or… there are books on how to write a dissertation but that only gets you so far, because it really needs to be very field specific. So, I tend to go very much the other way, which is a lot of sort of high-touch mentoring. I write co-authored papers with many PhD students, one of whom is now your colleague there at Oswego in the Finance Department in the School of Business, and it’s enjoyable for me. It is a good learning experience for the students and I think it helps them to learn how the profession works much more efficiently, because when it comes time to write a paper and they might put together some tables and say I want to structure the introduction this way. Oh, no, no, no, no, we can’t do that… because if you do that then it’ll make people be concerned about this issue here. So, instead, we need to twist it around this way and start with this… start with the big picture, not what your paper does but what your paper’s about. That sort of thing. That’s hard to learn on your own.
PhD students are PhD students because they are extraordinarily good students and they’re really good at learning. Though, that’s not the job for academics. The job is not the learning, the job is creating knowledge… and the transition from being a consumer to a producer of knowledge is scary, and it is the road that has very few signs or roadmaps to help them get down. It’s a transition of going from a consumer to a producer of knowledge… is very profound for a lot of people.

Rebecca: What’s interesting about what you’re hearing is you know my field the terminal degrees in an MFA a Master of Fine Arts, and the undergraduate degrees are really professional degrees. But, it’s the creators of cultural content ,and so that struggle happens at the undergraduate level too, of going from being that consumer of culture to a producer of culture. It’s really not that much different… just what they’re creating is a bit different.

Alex: That’s right.

John: I remember when I was working on my PhD, one time, where up until that point I had been meeting with my advisor every month to talk about my research, and at first I was just asking him questions… and then he was asking me questions and I realized suddenly that I knew more about the topic than he did… and that’s I think that sort of transition that’s sometimes difficult… because when you’re working on your research you’re mostly going out and finding all these earlier studies and so forth but you get to some point where suddenly you become the expert in the field and that’s a tough transition to make. It’s scary, as you said.

Alex: It is. It’s quite the watershed moment when you realize… when you are presenting your research to a room of 30 presumably learned scholars that maybe collectively they know more about the topic than you, but you know more than any individual person in that room… and becoming that expert and then owning it, so that you can write confidently is, I agree, a very tough transition

Rebecca: I like the emphasis on the owning it part. I think that’s key.

Alex: Yeah, imposter syndrome is real.
[LAUGHTER]

John: Everywhere.

Rebecca: So, we usually wrap up by asking “what are you gonna do next?”

Alex: Well, that’s a good question. I really wanna keep doing what I’m doing, making my class better year on year, teaching PhD students and training them year on year and working on research, hopefully research that people will actually find interesting and useful.
I have a sabbatical coming up.

Rebecca: When’s your sabbatical? What’s the countdown?

Alex: Well, the sabbatical is a year from now and in between then, one of my colleagues with whom I co-teach a doctoral seminar, he’s on his sabbatical. So that PhD course, that normally I teach half of, I’m now teaching the entirety of next fall. So, I think the first thing I’m gonna do is prep the rest of that class.

John: Excellent.

Rebecca: Deadlines make a difference.

Alex: Yes, indeed.

John: That procrastination thing… and that time preference…

Well, thank you!

Rebecca: Thank you so much.

Alex: Thanks.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

29. Learning about learning

Learning is hard work. The most commonly used study techniques often provoke the illusion of knowing. David Parisian, a member of the Department of Curriculum and Instruction at SUNY-Oswego joins us in this episode to discuss how he helps students overcome their misperceptions by introducing them to the science of learning.

Show Notes

Transcript

Rebecca: Learning is hard work. The most commonly used study techniques often provoke the illusion of knowing. In this episode, we discuss one faculty member’s success in helping students correct misperceptions by introducing them to the science of learning.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Dr. David Parisian from the Department of Curriculum and Instruction at the State University of New York at Oswego.

Rebecca: Today our teas are…

John: Barry’s Irish tea.

David: I’m not drinking tea today.

Rebecca: No?

David: Nope, I’m trying to cut back. [LAUGHTER]

John: With some people, it’s an addiction.

Rebecca: I have a problem. I’m drinking Prince of Wales tea today. Could you tell us a little bit about what you teach and how you became interested in incorporating evidence-based practices in your classes?

David: Well, I teach a few different courses. When I’m on loan to the Computer Science Department for the CSC 103, which is “Computer Tools and Informational Literacy for Educators” and then in the teacher prep program, I do ADO 394, which is “Interdisciplinary Methods” and then EDU 303 and oversee the “Block One Practicum” students, and do the online EDU 430, which is taking concurrent with their student teaching.

Rebecca: What is the 303 and the last one you just mentioned?

David: 303 are the “Block One Practicums.” So, their first semester junior year, when they entered the the block sequence for education, they have to spend time in a classroom. So, Field Placement secures their placements and then they spend a semester observing and helping where they can, but they’re getting their feet wet of being in a classroom. And the EDU 430—the online course—is “Professionalism and Social Justice” and that’s an online course that they take concurrent with their student teaching. So, a lot of the modules that they’re based on actually helps them in preparation for the edTPA that they have to submit for certification.

Rebecca: So how does the evidence-based practices fit into these courses that you’ve been teaching?

David: Well, one of the things that happened is the CSC 103 is designed for freshmen or transfer students that come to us for teaching certifications. So, one of the things we look at ishellip; approach that class from how technology is integrated in education. And one of the thingshellip; there was the book-read we had a few years back by Peter Brown, Make It Stick, kind of reaffirmed my doctoral work as we went through it… and I created a little quiz… a little matched-pair quiz… so that if they had to learn effective means… “does repetition build memory or does retrieval build memory?” and taking the material from the book and organizing this quiz… and what I found out was that all my students failed. So when it came to the 12 Principles within the book that we were testing on, less than one percent passed… which meant that a lot of the students we see coming to us, even though they can come to college, don’t have the knowledge or foundation of what strategies are most effective to learning. And that, in a teacher preparation program, my thought was, “do I continue that propagation through the pipeline or do I try to stop this stop that flow of students who are not effective or aware of the most effective strategies as they go out to be teachers or do we want them to have those effective strategies so they can implement them once they start teaching?”

Rebecca: So you staged an intervention?

David: Yes, it really becomes an intervention at that point and it’s really interesting because they have so many misconceptions coming in. And what we try to do is… they change their relationship with how they view sleeping. We set it up within the course so they take Barbara Oakley’s MOOC Learning How to Learn. So, they’re taught how sleep flushes out the chemicals overnight that are built up in the brain. They talk about procrastination and the Pomodoro techniquehellip; the benefits of flashcards and how to integrate that in spaced practice… and that’s one of the interesting thingshellip; I was just talking students today because they’re preparing presentationshellip; but a hundred percent of the class knew about spaced practice but no one knew how to implement it, or what it might look like.

Rebecca: Right.

David: So we provide opportunities for them to explore those areas, whether it’s creating flashcards where they’re using a flashcard app or have them making flashcards to build in the spaced practice to know when to practice and how to interleave and all those components that research has shown to be most effective.

John: Many of our listeners are familiar with all of these, but could you go back and talk just a little bit about the Pomodoro Technique.

David: The Pomodoro Technique is really fascinating. What it is… is for those who procrastinate, you block out 25 minutes …and actually the Pomodoro Technique is based on a tomato kitchen timer… where you set it for 25 minutes and then at the end of 25 minutes, you take a five-minute break… and then you set the Pomodoro timer and you go through it again. And what happens is you can begin to measure your workload in terms of how many Pomodoros it’d take. For the students, what I notice is they start changing their perception of assignments from a product driven “what’s the end product?” to a process of “how to get to that end product?” So many have commented on that and you can get apps on your phone that are the Pomodoro Technique, that will set up a 25-minute block. There are apps that will stop any notifications for that time so you don’t have to be keep looking at their phone because that’s the distraction that I’ve noticed, is that people are attached to their devices and have to be aware of every beep and every notification and everything that comes as opposed to blocking time. So the Pomodoro Technique has been pretty effective and most students have implemented that in terms of working on their procrastination skills or adjusting… changing behaviors.

John: So why do students have these misperceptions?

David: Good question. I come from a K-12 environment. I was a secondary science teacher for 18 years. I was a district administrator for another 12. So, coming from the K-12 environment, what my message was to them is that you were taught by loving, caring educators who were passionate about what they did and did the best they could with what they had… based on the information. That might not have been entirely best practices of what we now know about neuroscience and how the brain learns and the effective study strategies on the materials from Make It Stick. So, I think they grew up with whatever progression they went through. One thing I did notice is that students who struggled… that worked hard in school but just got into collegehellip; still struggle with the workload. What surprised me was talking to some studentshellip; is the students who were the bright ones… the ones that got it quick in school… that went through high school with no challenges… come here and all of a sudden, they’re placed in a situation where now they have to study and do all those things and they don’t have the skill sets to study. Just coming up through, I think our assumption is we feel that students know how to study because they’re in college and really what’s taking place is students aren’t really taught how to study. I think the assumption that students know how to study probably backs all the way down into fifth, sixth grade. I think we had some earlier comments where we were talking about us being in school and teachers made us write flashcards. We didn’t understand why and they might not have understood why, but we made flashcards. Now, as that is an appropriate practice, or one that works that you know, that can be integrated into a spaced practice and it’s really just teaching them a little bit about neuroscience… how the brains learns… how that all we do is encode, consolidate, and retrieve… and how do you build those principles and practices using technology… using skillsets… managing their time and trying to put that into a package where they can begin to see it. Because once they see it, then they want to improve their studies so they may begin doing it… so they go to bed earlier, they get better sleep and they begin to change their behaviors. I didn’t expect that but it was an awesome sidebar from that.

John: Part of it is, as you said, they haven’t always had much practice or training in learning how to learn and partly that may be because many of the teachers didn’t have the same…

David: Exactly.

John: …people have been just doing the same things that seemed right for an awfully long time… and one of the problems, though, with some of the evidence-based practices is that it doesn’t feel quite as good… because when you try to work on retrieval practice, after you’ve been away from a topic for a while, it doesn’t feel as good as perhaps repeatedly rereading something until it looks so familiar that you think you understand it. So there is that fluency illusion that people get that feels really comfortable and when you do some low-stakes testing or when you do some attempts at retrieval, you realize you may not understand it quite as well and it doesn’t give you the same sort of reward immediately. So it takes some training, I would think, for students to be convinced that these methods really do work. It sounds like you’ve been able to achieve that in your classes.

David: One of the advantages is it’s a setup so that we use the class and the content within the class as a training mechanism. So, they’ll practice working on a flashcard app to learn the app and then create flashcards for the content. One of the things they do have to do because the modules I set up are in a worksheet form so the initial encoding part is our instructional part. The consolidate part is them now going into another class and applying it in another class and showing the evidence in mine. So that they’re forced to, not only just use it in mine, but now go into a Psych 100 or a LIT course or whatever they’re in to create flashcards, create mind maps, begin to incorporate that… and we map out, When are your tests,?” So, how far do you already have to back up. So it’s really just coaching them on how to utilize those strategies and to get them so they can begin to internalize that to meet with their success.

Rebecca: Do you find that the students are a little resistant at the beginning or do you think because you’re providing the evidence and the science behind… the reasons behind… why you might use a method, is that what’s helping?

David: Well I think it’s a combination of both. Part of it is they have to go through the MOOC… so it’s not me presenting the science. Then we have a series of videos. There’s a gentleman by the name of Thomas Frank, who has a series of videos on studying and using it, so he has a more modern twist to it, being late twenty-something or just through college so it’s a more animated video. And then I’m in class to give suggestions, those type of things, but once they saw that on the first day school, they take a quiz and fail it and then realize that they don’t know what they think they know, then they’re pretty open to “this is what we’re gonna do for the semester.”

Rebecca: Sounds pretty motivating.

John: It is a motivating technique, yeah.

David: Well, I had one student once say like, “first day in college and I’ve already failed a quiz.” I’m like, “rather fail it now then in 15 weeks.”

Rebecca: Yeah.

David: …that’s the poise. It’s really just redefining their tool sets and introducing new ones… and not everyone takes to everything. Like, mind mapping is a classic example. whether it’s Tony Buzan’s method, but I’m very mind map oriented, but a lot of students have a negative experience towards it and I said, “if you want to be a teacher you may never do a mind map but you will know how to do a mind map, because you may have a student that needs to know how to do it and you have to be able to teach it so even if it’s not for you and you don’t like it, you darn well better know how to be able to help someone else learn it.” It’s redefining a different role for the teacher as our candidates come through as be more knowledgeable base, more pragmatic, and more understanding on what effectively works because they can then share that information with their students.

Rebecca: So what do we do for all those students who aren’t in your class in their first year, right, who aren’t becoming teachers but they’re, you know, trying to exist in this system, who might also have the same exact struggles that your students have? What’s your recommendation?

David: Well again, Thomas Francas has great videos and whether it’s how to read a college textbook, how to take flashcards, how to do any of that. That’s one aspect– knowing that you need help is another aspect. We have the success tutors that that have those skill sets, are using the same information to help other students. So, I think there’s resources on campus to support that.

John: …and our Student Success specialists or I do are very familiar with all these techniques and they work with students that are struggling to help build that up.

David: Right, and I think looking at it from a instructor side of the house, just as people become familiar with the strategies… is making those known..making the strategies known. If you’re teaching an Economics course or teaching whatever, if you say there’s a set list of vocabulary or concepts that you need to know, you know, put them in flashcards, go to these links, it’s not like you have to prepare everything because you can’t make meaning for someone else… they have to make it on their own… but you have to provide those initial concepts or the information because the student has to be able to take it in, but telling them how to take it in or just pointing them to links can facilitate that process. I don’t think anyone really comes here with a desire to flunk out, you want to be successful, but some of the times, they get paralyzed by the paralysis of the whole situation.

John: Some of it can be done through course design too, so that they are designed to include some level of retrieval practice,

David: Absolutely.

John: … some amount of interleaved practice,

David: yeah

John: ….and some spaced practice.

David: Yup, yeah, absolutely. And those type of structures in the redesign isn’t that large of a jump to do for the professional side of the house. The biggest thing I can tell for any student, the simplest thing to do if you’re feeling that stressed is breathe. It’s kind of funny… whenever you’re upset, what do people tell you to do they tell you to breathe and just from a physiological standpoint, taking deep breaths… slow deep breaths… once you start breathing about six breaths a minute, there’s a branch of the vagus nerve of the parasympathetic system of our physiology that automatically slows the body down. So, when you’re taking a test, what do you do? Breathe slowly. If you get stressed out on it, draw a circle around every fifth question so that every fifth question, you’re just taking three deep breaths just to say, “this is all right, we can do this.” And you that will calm the physiology and reduce that stress response of seeing an exam… the fight, flight, or freeze… and we’ve all experienced those… with varying degrees of success. [LAUGHTER]

John: But we want to set up a system where students generally be successful.

David: Absolutely… Absolutely.

Rebecca: So you’ve talked about the Pomodoro Technique… you’ve talked about breathing… you’ve talked about flashcards. What are some other key things that instructors could help students think about as strategies to be successful?

John: Well, you mentioned sleep, too.

Rebecca: Sleeping’s good.

John: And that’s something students often have trouble with.

David: Well it was interesting because there was a presentation done once and one of my students… I commented that she would stay up till 2-3 in the morning texting her friends and then they got that part of the MOOC and talking about sleep and she’s the one that started to go to bed at a more reasonable hour to be able to sleep. I think if one hasn’t read Make It Stick… that’s a great book and I think for a lot of us on the professional side of the house… as instructors we’re like the great white shark of our content, we’re the rogue, we’re at the top of the food chain. But one of the things, I think, the book does is lays out for you this way in which people encode information, consolidate, and retrieve it and I think having that as a foundation and then, reaching out to the center here for like, “how would you design something? I’m struggling with this” and just say “I want to reach the students” because I think a lot of professors do want to make those connections with the students. I think there’s help and support on campus to help people in designing those strategies. Personally I start with trying to give the overview… try to give the gestalt of the course… “what’s the wholeness gonna look like” and then just lay out the clear objectives and then integrate and make the assignments, being those flashcards and studying and there’s just so many ways to communicate, I guess. Even taking Barbara Oakley’s MOOC Learning How to Learn and just learning some of the basis from the neuroscience, even from the students. Have students get together informally and go through it. It doesn’t really take a lot, all the information’s out there.

John: And if I recall, it’s offered every month isn’t it? It’s a four-week MOOC and it’s offered very regularly.

David: Well it’s a four week…, like every week now, I think. For those who aren’t aware of the MOOC through Coursera, a MOOC is a massively open online course, Coursera is a vendor for that and Barbara Oakley’s MOOC. I think last year i read in the New York Times where it was the most popular MOOC on the planet.

John: It is i believe still the most popular MOOC…

David: I know when I took it, I took it with two hundred and seventy-five thousand other people at the same time. That was a few years back. Integrating that type of information and layering it into a course just gives a different feel for the students. And as I tell them even though they’re freshmen and transfer, I said “if you’re going to be a teacher, down the road, some of your professional development is going to be delivered through these.” So, we’re just getting them ready orientating them. Well there’s just you know some really simple information that people can do to design, redesign, and be more effective, more engaging and have students be more successful.

Rebecca: I’ve had students respond very positively when you explain why you might do a particular technique or a method. At first it seems like “why are we doing this? why are we doing quizzes?” but if you explain why and how it helps they’re much less resistant and actually embrace the idea. I’ve had students ask for more quizzes because they’ve learned how it’s helpful…

David: That just reminds me of, in our lesson plan development, as the students are going through it… one of the parts of the lesson plan development I have after activating prior knowledge is setting the purpose… and setting the purpose is “so, why is this lesson important? why do you need to know it? How many courses have we ever been in that no one’s ever told us why we’re there and you’re absolutely correct. Once you say the why and give the explanation, it creates an association or connection with the student and that’s one of those underpinning things… as the more associations you can connect to them, then the more apt you’re gonna have their buy-in to engage you in the content.

Rebecca: I think a lot of times students see assessments like quizzes and tests as some sort of penalty…put you in the penalty box or whatever… and that’s a faculty member’s way of torture or something, but as soon as you say that the purpose of doing this is to help you recall information and to make sure that you have that foundational knowledge, you can continue building in these more advanced classes. They stop seeing it that way and it’s pretty amazing that it doesn’t take much of a conversation… it takes having the conversation.

John: And the more frequently you do that, the easier it is for them to get past that because high-stakes testing is stressful, but if you replace it with lower stakes testing and more regular testing, it’s easier for them to see that they’re learning from this experience and it doesn’t hurt them as much if they screw up… that it’s an opportunity for them to improve and continue…

Rebecca: However your students responded after your class like moving into these upper-level classes because you’ve been doing it long enough now that probably some of them are now in those advanced courses I would imagine.

David: Part of it is coming up through the pipeline and having viewed or patterned assessments or quizzes as the “gotcha” and getting hammered. Ken and Rita Dunne, one of the things they stressed that I always kind of stress is “high content, low threat.” So, whenever you can engage and have your material be of high content but a low threat, students are more apt to engage because they don’t have that fear reaction going on and I think it’s changing that patterning by having the conversations that these are beneficial for you. So, that’s my thought on the quizzes. Some of them continue that patterning and that’s one of the things where I try to keep it going for the semester in hopes that they’re gradually continuing to do those processes, those strategies to continue their learning. The one thing about teaching the freshmen… I’m probably one of the few faculty members that have this interesting purview because I see them as freshmen or transfers coming in, I’ll see them as their block one junior EDU 303 practicum. I’ll touch base with them again for the ADO students that take 394 as their block two, so I have them as a class and than in block 4 when they’re seniors going through the the end of the pipeline. So I can see their growth along the way from that and…

John: And ADO is adolescent

David: Yeah, the adolescent, yeah… and that one they’re constructing the lesson plans, they’re trying to see how this all fits together, trying to pull on multiple layers. We revisit Make it Stick because in all my courses that’s just part of it. So, they’ll read the book as a textbook in that section… so then they can begin to refresh like “Oh, I remember when we did this” and “how’s it going?” and then we have conversations on when they’ve been using it and how effective it’s been. I just keep trying and plugging away and stepping up to the plate…. every day you get to take a new swing

John: And the more people who do that in their classes and certainly the more people who are trained to do that at lower levels in elementary and secondary school, the better off students will be.

David: Yeah… especially nowadays when you you look at the various challenges in a k-12 environment. If you can begin to seed the ground with what works and just focus on that then we’ll be okay.

John: Do students go on and use these practices in other classes after they’ve been exposed to them in your intro level course?

David: Part of it, once they leave me is trying to continue that propagation based on the courses they’re in. Some of the strategies are more effective in some content areas than in others. Math is always a struggle in terms of looking at applying the flashcards while you can do color coding or dual coding where you’re including images or multicolored in the equations as you follow different variables through an equation sequence, those type of things. The other thing is that the strategies, and this might be one of the misconceptions students have, learning isn’t easy. Some people comment like “well if I just sit here I should get an A”…… but learning is messy… learning’s organic… learnings dynamic… and learning takes a lot of work and sometimes, depending on the student’s course load and what they’re taking… if you’re taking a 4-credit science course, you got three hours of lecture and a three hour lab somewhere in the week… having been a science major and having a lot of hours on a lab. So part of it is finding time to create the materials… the mind maps… the flashcards. Those students who have a better time management… work ethic, those are the underpinnings I think that makes this a successful student and that they put forth the time and schedule that to do all those things that are necessary, whether it’s creating a mind map or whether it’s creating flashcards or creating the time intervals for the spaced practice or when to do the spaced practice. I was talking to a friend who used flashcards and whenever she was grocery shopping for her family picked the longest line because it wasn’t about getting out of the grocery store fastest, it was about being in the line the longest as she pulled out her set of flashcards and reviewed them in the line at the grocery store. You can find intervals to do those type of things.

John: I always wondered why there were always more people in the longest lines. Maybe…Maybe they’ve been in your class [LAUGHTER].

Rebecca: Yeah… everybody’s doing flashcards.

David: Then again yeah, flashcards is just one aspect of that but you can integrate that from a quiz standpoint… from a retrieval…. and one thing that could be interesting is when you look at the research on flashcards, or how to create them, there isn’t the level of “how do you create” going down Bloom’s continuum of higher processing from a flashcard aspect. A lot of the information we see is low level…. vocabulary words, or those type of things… but how do you all of a sudden take two flashcards and compare them and say “compare and contrast these two concepts of something…” and so how do you get a bigger cognitive load going from using those… and the designing of flashcards… that’d be a great study for someone to do. I’lljust put that out for anyone who’d want to. [LAUGHTER]

John: You get to work with students a bit later in their academic careers that you worked with earlier. How do they respond when they come back in upper-level classes? What do they say about their experiences?

David: Usually, semesters later they forgot about me. [LAUGHTER]

Yeah, but one of the things we try and do at the end of the course is they have a five-minute presentation they have to do… and I give them the slides and so it’s like “how has this changed you?” and they go through and reflect on that. So, I give them a template of what the presentation is and it’s their five minutes of fame where they get to begin to find their teaching voice, and it’s the first time they’re in front of the class talking about it. So, you talk about how they how they did it. In terms of seeing them later on, the people who use flashcards and grew up continue using them and then you have various levels of people who took the buy-in to create those processes. The other thing is you try to encourage them to use it, so as they’re developing their mini unit you have them do the flashcards to go with the unit… you have them do the mind map. So, you have them go through it, and I think from that aspect, they recall fairly quickly what it is they needed to do to generate it… and then it reminds them like “oh, yeah.”

John: But, if we do this in more classes and we use them or we structure our classes so that students naturally adapt some of these practices, it’s going to help reinforce these things… and the more people are reminded of how important this is and how their usual practices may be really helpful in cramming for a test the next day but aren’t going to allow them to remember the things much past that day.

David: Right. In the real world you have to remember those things past that day.

Rebecca: A couple of the themes that you were mentioning most recently is about time management and the work ethic component fitting into this and so it seems like that’s the next discussion. How do you make sure that students know what is a good work ethic? Do they even know? Do these conversations happen? I’m not sure that they do.

David: You can see those students that have work ethic pretty quick …and I’m just trying to flashback through courses and images through my head of students… from a freshman those that are asking questions… those that are getting work done and turning things on time… those who are turning it in early… those who who show up to class earlier… and sitting there… they have a certain level of comfort within themselves… where they’ll ask a question. Truth be told, I didn’t feel comfortable asking a question in college till after seven years of college… not that it took me that long to get a bachelor’s degree… it wasn’t till after I did my student teaching a long time ago… and after I did my student teaching I learned how to study. I did the outlines before the chapter. The following semester I went for a neuroscience certification taking our geology courses. So, I outlined the chapters before I walked in there. I pre-taught myself the material. I laid out all my notes and stuff before the lecture… had the conversations because then it was internalized to me that this was important… and I think until someone gets that into their intrinsic fabric of themselves… where they want to take this as being important… and I still don’t know what I want to be when I grow up. I think that’s the big thing. People who are serious know to get the job done. They’ll do the work and they’ll do it to what they need to do… and if you give them criticism and feedback, they correct it.

Rebecca: I’m hearing a growth mindset described.

David: It is.

Rebecca: …you’re open to feedback… The the real challenge is how do you get students with the fixed mindset, who maybe don’t have that work ethic. or good time management skills. to get on board.

John: …and you mention that case about some students earlier who had done really well in high school and then suddenly struggle when they get here. Those, empirically, tend to be the students who have a fixed mindset… where they’ve been successful with the techniques they’ve used, but once they have to move into a new environment or they have to engage in more transfer than they had to earlier… suddenly face some struggle and it’s a shock to them and they often give up.

David: Well, and one of the interesting things, though, and this they’ll need to be reminded of, but within all of us we have growth and fixed mindsets.

JOHN. Yes.

David: So, I could have… I’m not picking on math but I’ll pick on that…. Most students… math is not a strong aspect, because really the only time you do math is in math class. No one asks you when you’re walking down a street to factor a quadratic equation, right?

John: Well, rarely… [LAUGHTER]

David: …rarely… but you consider a musician. They know with practice they get better. In sports, you know with practice you’ll get better. In math, if you practice you’ll get better. So, part of that is understanding the context of which it is. I may struggle in math but I might be a musician in the band, and so I know. So, you have to kind of transfer where they have been successful in showing them, in this part where they’re not being successful, how they can be. Because, if you can tap into a person’s growth mindset… and it could be in a K-12 environment. You have somebody I saw students riding their skateboards… phenomenal skateboarder… can do all these tricks… will spend hours learning a trick, right? That whole idea can be shifted to their studies as well. How long did it take you to learn this trick?

John: How many times did you fall you know along the way.

David: …and how many times did you fall? The culture is changing where, not only are we imparting the information to the students, but we’re also being their coach. We’re trying to nurture them. We look at them as adults coming here… without the parents for the first time. The baton we’re handed is actually trying to nurture them into the adult working life and understanding how we learn… how we process information… how we interact… the building of rapport. how do the rapport aspect is all part and parcel… I feel… what we need to do… or what we do… I know it’s what I do.

John: How did you integrate the MOOC into your class?

David: I build the class around modules that last two weeks. So, with the beginning of class, I took some of the material from the MOOC or what the topics were and then I created worksheets based around that. So, if it was procrastination then there’s articles that they were reading in, and picking up on, and getting their takeaways… So, that part was teaching the foundational aspects of of how these strategies work, and then giving them time to practice and doing them within the two weeks. The MOOC they can view offline. They can take the quizzes. The other thing I did, is with Barbara Oakley, she had Coursera set me up as an administrator for my course. So, then I could just upload my class list to it and then it would keep track of the quizzes…

John: Oh, nice.

David: …and then I could download the the grades or whatever. So, she was giving him the content on one side; we were building in related practiced and article support on the other side. Then the consolidation part… and I broke it down into it an encoding section of the worksheet, a consolidation, and a retrieval part basically patterning our learning process. So, watching videos and reading we’re encoding… applying the material was the consolidation… and then their reflection and the reflections based on making a video, responding to an interview question, or reflecting on their experience over the two weeks… and they were able to communicate those. So, part of it is just finding out what the MOOC is doing, getting materials that kind of pattern that (that’s where I brought in you know the Thomas Frank videos and other support materials).

John: The learning scientists also have some good ones.

David: Yeah, exactly, and I use a lot of the Learning Scientist’s material and McDaniel’s site deals with retrieval practice, so there’s a variety of things. We try to overlay the MOOC with Make it Stick and strategies there to create an environment that over the first four weeks they’re experiencing… they’re learning… and they’re beginning to apply… and then as we build out the other modules, we still keep repeating… For example this is module four we’re in and one of the things I’m training them in is advanced search strategies. What are the topics they’re going to be searching? Well, there’s eight setups within the lab so… elaboration… retrieval… spaced practice… those are the deep research things. So, each group has to now prepare a presentation, but they have to do the research. What’s the research that supports this? and what are strategies to help? So, now there’ll be eight presentations of the 8 strategies that they’ve learned. Trying to deepen and make touch points through the semester and keep reminding them… constantly reminding… constant… constant…[LAUGHTER]

John: So, what are you going to do next?

David: Well, next will be a continuation with the Computer Science…. Looking for more in-depth application across the content areas to help students… and then professionally, this summer, working with Educational Administration Department and their Project Blend Symposium. We’ll be doing the third installment of Resiliency and Leadership working with the Institute of HeartMath. So, outside of that, we’ll continue to work in those areas of heart-brain synchronicity and just working and having fun.

John: Thank you for joining us. It’s been a pleasure.

David: It’s been my pleasure. Thank you very much.

Rebecca: It’s always great to hear what you’re doing in your classes and the results and thanks for sharing that for everybody else.

David: Very good, thank you .

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.