74. Uncoverage

Introductory textbooks in most college disciplines tend to become thicker over time as new topics are steadily added while old topics remain. Classes designed to “cover” all of these topics necessarily sacrifice depth of coverage. In this episode, Dr. David Voelker joins us to examine how some faculty are changing their focus from “coverage” to providing students with an opportunity to actively engage in the discipline and uncover its power to help explain their world.

David is an Associate Professor of History at the University of Wisconsin Green Bay. He is also the Co-Director of the Wisconsin Teaching Fellows and Scholars Program and the co-author with Joel Sipress of “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” which was published in the Journal of American History in March 2011.

Show Notes

  • Voelker, D. J. (2008). Assessing student understanding in introductory courses: A sample strategy. The History Teacher, 41(4), 505-518
  • Sipress, J. M., & Voelker, D. J. (2009). From learning history to doing history. Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind, 19-35.
  • Sipress, J. M., & Voelker, D. J. (2011). The end of the history survey course: The rise and fall of the coverage model. The Journal of American History, 97(4), 1050-1066.
  • Voelker, D. J., & Armstrong, A. (2013). Designing a question-driven US history course. OAH Magazine of History, 27(3), 19-24.
  • Wiggins, G., Wiggins, G. P., & McTighe, J. (2005). Understanding by design. Ascd.
  • Gordon Wood – Author and History Professor
  • Gary Nash – American historian
  • Angela Bauer – Professor and Chair of Biology at High Point University
  • Ryan Martin – Psychology Professor and Associate Dean of Recruitment, Outreach, and Communications at University of Wisconsin at Green Bay
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • 61. A Motivational Syllabus. Tea for Teaching podcast (with Christine Harrington)
  • Christine Harrington – Associate professor in the Department of Educational Leadership at New Jersey City University and the author of Dynamic Lecturing
  • Lendol Calder – Professor of History at Augustana College
  • University of Wisconsin’s Teaching Fellows and Scholars Program
  • UW Faculty College
  • Spring Conference on Teaching and Learning
  • Regan Gurung – Professor of Human Development and Psychology at UW Green Bay
  • Scholarship of Teaching and Learning
  • Gurung, R. A., Chick, N. L., & Haynie, A. (2009). Exploring signature pedagogies: Approaches to teaching disciplinary habits of mind. Stylus Publishing, LLC.
  • Ciccone, A. A. (2012). Exploring more signature pedagogies: Approaches to teaching disciplinary habits of mind. Stylus Publishing, LLC..
  • Gurung, R. A., & Voelker, D. J. (Eds. Gurung, R. A., & Landrum, R. E. (2013). Assessment and the scholarship of teaching and learning..). (2017). Big Picture Pedagogy: Finding Interdisciplinary Solutions to Common Learning Problems: New Directions for Teaching and Learning, Number 151. John Wiley & Sons.
  • 54. SOTL. Tea for Teaching podcast
  • International Society for the Scholarship of Teaching and Learning
  • American Historical Association
  • Tuning Project

Transcript

John: Introductory textbooks in most college disciplines tend to become thicker over time as new topics are steadily added while old topics remain. Classes designed to “cover” all of these topics necessarily sacrifice depth of coverage. In this episode, we examine how some faculty are changing their focus from “coverage” to providing students with an opportunity to actively engage in the discipline and uncover its power to help explain their world.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Dr. David J. Voelker, an Associate Professor of History at the University of Wisconsin Green Bay. He is also the Co-Director of the Wisconsin Teaching Fellows and Scholars Program. He is the co-author with Joel Sipress of “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” which was published in the Journal of American History in March 2011. Welcome, David.

David: It’s nice to join you.

Rebecca: Today’s teas are…

John:…are you drinking tea?

David: Oh of course, always. Yes. I’m having a Moroccan Mint Green tea.

Rebecca: So you’re a tea drinker so you can come anytime. [LAUGHTER]

John: I’m drinking Black Raspberry Green tea.

David: Oh that sounds nice.

Rebecca: And I have Bombay Chai today.

John: We’ve invited you here primarily to talk about your work on redesigning the introductory history course. Could you tell us a little bit about the problems that you observed in the traditional approach to teaching the survey course?

David: I started teaching history at UW-Green Bay in 2003 and I was trying to create a class that would be very engaging for students. We would look at primary documents, I would really encourage them to think critically about the materials, and so forth. But I was still doing a lot of coverage, I went through the standard list of topics that you would follow to teach American History and I had a textbook. Well, after doing that a couple of years, I had two really wonderful students in my office. These were very engaged, hardworking, curious students, and they were talking to me about an upcoming exam and I found that they were just trying to memorize the content as I had taught it. Now, they were clearly capable of so much more, and they were doing more than that. But when it came time to the hard work they were putting in to study, they saw it as a memorization exercise. And I realized, “Wow, if these are the most engaged students in the class, then something’s not going the way I want it to go,” because that’s where their attention was…on memorization. That’s when I began a long journey to rebuild the way I was teaching—or reconceived how I was teaching—the intro history course. I ended up actually developing not only a new way of teaching the course, but entirely new ways of assessing student learning. So that when students were studying, when they were really putting in the effort and so forth, they would be focusing on the things that I really wanted them to learn, which was: how to interpret a primary source, how to find the argument made by another historian, how to put all that together into historical analysis. So that’s a quick summary of how I got on the path of reworking my introductory history course.

John: I don’t think this is unique to history. I see the same thing in economics. And in general, when students come to college, they often learn somewhere along the way that memorizing lists of things is what they’re expected to be able to do in college, and what you’re addressing is a problem that we see in all of our disciplines to some extent. So what would you recommend as an alternative? How have you approached this issue, or how have you tried to resolve this?

David: Well, first I should say I was really inspired by the book Understanding by Design. And I just looked at my copy and apparently I acquired that and started reading it in 2006. So it’s been a while ago, and it’s quite marked up. But the basic premise there is, instead of starting with, “Here’s a list of all the things I want to cover,” or “Here’s a list of all the things I want students to read,” you really start with “What is it that I want students to be able to do as a result of taking this class?” Now, students taking an intro history class, I’m not going to say, “Well, I want them to be able to build a house,” or something like that. What they’re going to be able to do is think in particular ways. You know, not necessarily the kind of skills that you would be able to observe in some product other than their thought. So I really tried to develop a class that would help students develop their ability to think historically. Now I realize they’re doing that for an intro course at a beginner kind of level, but the emphasis is still on historical thinking.

Rebecca: When you’re talking about the ways that we want students to think and work, we often get frustrated as faculty when students aren’t doing that but we don’t always take the time to articulate that, so I think it’s important to highlight that as a good starting point.

David: One thing I tried to do here is… well, just to be really intentional. What is it that I want them to learn? And then I think the crux of it is to make sure that I’m actually assessing them on the things that I really want them to learn. And I would guess that the overwhelming majority of college level history instructors really do want their students to come away with the ability to think historically, or at least to have some growth in that area with the ability to know what to do with a primary source, you know, historical documents, or perhaps an artifact or something like that. So I would say our hearts are in the right place but the question is, how do we actually get there? How do we align our highest learning outcomes with the actual assessments that we’re doing? In Understanding by Design—I don’t think I mentioned the authors earlier but Grant Wiggins and Jay McTighe—they really put a lot of emphasis on assessment. Now I know in higher education there are folks who are wary of assessment and they’re thinking of institutional assessment, which I would say is important, but here we’re thinking about how are you grading the students? How do you know what they’re learning? So I decided to rethink my assessment and make sure that my exams were really requiring students to do some historical thinking on the spot, also drawing on and demonstrating that they have knowledge about the history we’ve been studying. So rather than seeing if they memorize some content, and you know they can somehow show that on an objective exam, what I do is on exams, they encounter a historical claim. So a really simple one would be “Christopher Columbus discovered a new world in 1492,” and I ask them to think about that critically and to think about that the way a historian thinks about it. And I asked them to argue both for and against the statement like that. So they’re arguing, they’re not simply regurgitating information. That means they’re using evidence and, in order to use evidence, you have to have content knowledge, right? You have to have knowledge of historical context, you have to know some of the sources. And so I see all that stuff that we “cover,” so to speak, really is the raw materials that students are working with as they develop their ability to think historically.

John: You’ve mentioned using in general, a backwards design approach, and what you’ve just described is part of it, where you start with the ultimate learning objective and then you design assessments that would measure that. How do you prepare students to reach that level?

David: Well, I guess that’s another big question, right? One thing, I try to be very transparent about this. I mean it’s interesting how sometimes in higher education we’re not very transparent. And that means that I talk with students about what it means to think historically, what that looks like. I try to model that for them, of course, but I do it in a way that’s explicit. I mean, here again, I would say most history faculty across the nation model historical thinking all the time, but they may not be very explicit about what they’re doing. So I tr y to be explicit about what it means to think historically and then also to give students a chance to practice that in class. So it happens my classes are 80 minutes long and well, I’m very grateful that I’m not lecturing for 80 minutes straight. I would just find that very difficult to believe I’m engaging anybody… I’m covering stuff for 80 minutes. So, I really try to break the class session up and have students not only talking about primary documents—which is pretty common for history classes—but we also look at different perspectives from historians. Now, I don’t introduce them to the vast historiography on any topic that we’re looking at, but I do make sure that they know about at least two different perspectives on major issues of the colonization of North America, or the American Revolution, or the coming of the American Civil War. So they can see that there’s actually a debate out there, rather than assuming that, “Well, history is pretty cut and dry. Like, we know what happened, we’re just describing it.” So, I really try to introduce to them the ways in which the study of history is an ongoing dialogue among historians, that we’re always turning up new sources, and we have new ways of thinking about old sources. And so I give them examples of how historical thinking is working in the field and really ask them to wrestle with some big questions. Again, it’s at an introductory level, so they’re not going off and reading everything that Gordon Wood wrote about the American Revolution and then comparing that to Gary Nash’s interpretation or something like that, but they are getting little snippets of that sort of thing so they can really get a sense of the debate and they talk about that in class. I don’t actually hold formal debates but I’ll have small groups discussing a particular issue and then I make a sort of matrix on the whiteboard and they go up to the whiteboard and sort of vote for where they think this historian stands, and where they think a different historian stands on the issue, and then where their group is, and so they’re really playing around with those different positions. And then, of course, on the exams, they’re actually making their own individual arguments using the primary sources and the secondary sources.

Rebecca: One of the things that you said was “modeling, but making that explicit.” Could you talk a little bit about how you shift from just modeling modeling explicitly or being explicit about doing that?

David: Yeah, sure. Well, one thing I do is I have a kind of graphic that I show that I created that tries to describe in a basic way what historical thinking looks like because I do think a lot of students come in with a really simplistic understanding of history. Again, this notion that it’s pretty cut and dry, it’s a description of what happened in the past. And so I try to help them understand both, through my own comments but then through practice that, “Okay, yeah, we have descriptions about the past, but those are based on evidence that has to be interpreted.” And those descriptions, we assemble those into a narrative and built into those narratives is an actual explanation and analysis of what happened. So we’re really thinking about why things turned out the way they did, how different things are interconnected, and then finally—and maybe the most importantly—is significance. Historians are also wrestling with the significance of the past, different events, and developments. Not only the significance in a narrow historical sense like, “Okay, this event was significant because it led to another event,” but also a longer term significance, and then I think even a kind of moral significance. What can we learn from the past? How can we use the past to understand American national identity? Those sorts of questions. So we really kind of build up, there’s multiple levels of complexity there. And I try to get my students at least beginning to climb that ladder to some of those higher levels, even if they’re just doing it in a really rudimentary way. But then I hope they can really see the value of history that goes beyond just kind of rote recollection of something that happened in the past.

John: Do students sometimes resist moving into this more appropriate view of history? They’ve learned through elementary and secondary schools what history is. How do you break their expectations and get them into this more active role?

David: I’m kind of smiling, or maybe even grimacing right now, because I usually am able to win students over. I mean, there’s so much about the past that’s exciting, that’s interesting, that’s unexpected. When we start to dig deeply into some of these topics, they’re interested and they’re curious and sometimes they’re also pretty frustrated or even angry because they feel like they’ve been misled in the past. In other words, that maybe the basic information they had from high school—of course the quality of that instruction varies—but I do have a lot of students who come in who are upset. They feel like they’ve been misled. And I want to rush to add that I have many students who have had excellent teachers in the past and are already really interested in history in part for that reason. So I think there’s a lot for students to be curious about. Occasionally I do run into a student who is very resistant. Most students find that, “Hey, turns out this is a lot more interesting and engaging,” than the history they had been exposed to in the past, or it’s deepening and complicating the pretty good understanding that they already had.

Rebecca: Do you find that they’re surprised by the first assessment or exam because they have these prior expectations and then your methodology in your class is different from their past experience?

David: Well that’s something that I anticipated and so we actually have a practice exam, so to speak, very early in the semester. In fact, that example I gave a moment ago of Christopher Columbus discovered a new world and 1492 is a sort of maybe second week of class practice exam prompt. And so I have them do that outside of class and then bring it in and we essentially go over that together. They share their thoughts with their classmates, we talk it over, I give them a number of examples. So for a student who’s coming to class and whose keeping up, there aren’t really going to be any big surprises and I give them some samples of what the exams are going to look like from other classes. I used to teach a lot more modern American history and so I can easily just share some sample exams that students told me it was fine if I shared and I think that’s really helpful, especially if there’s a big change in expectations, then I think to be fair to students, you should really be clear with them about how the expectations might be different than what they’ve encountered in the past. And a lot of students actually like the assessments I’m using. I think they find them more interesting. They’re perhaps a little harder to study for in the sense that it’s a little less clear cut but they have a pretty good idea of what the major topics are and I give them some suggestions for what they should be reviewing in terms of sources. So they really come in thinking, “Well, I’m going to need to refer to least a couple of these primary documents and the perspective or argument of this historian.” We practice all that in class because again, that’s what I want them to be able to do. So I make sure that we spend a lot of class time essentially practicing the very thing that they’re going to be assessed on.

John: If someone were to come into your class, what would it typically look like? Are the students broken up into really small groups? Bigger groups? How do you have them engage with the material that way?

David: Well I typically have about 65 students in my introductory US history class. I have another class that I teach in a similar fashion that’s actually a writing e mphasis class though it’s a little smaller at 45. So an intro level class and environmental history. But in both of those classes, we will typically start out with the whole group and I make some introductory comments. Sometimes I give a brief lecture on background context. I mean, historians really do value context. This is part of how you make sense of the primary documents and so forth. So I do all of that and you would expect to see that in a history classroom, but then before too long, I’m giving the students some kind of a series of questions and they’re working with primary documents or they’re working with some short essays by historians and talking those through in their small groups and then we have various ways of reporting out. Like I said, sometimes I will send them to the board so they’re kind of recording their analysis on the whiteboard. Other times it’s just more like I’ll choose particular groups to respond and sometimes I just open it up. So, I think that’s all pretty typical except that I’m constantly going back and forth from maybe a brief lecture to small group discussions to a large group discussion. I suspect there’s more back and forth than you would typically see in a more traditional coverage oriented history classroom. Again, many instructors do use primary documents and they would have students talking about them. I think maybe where I’m going further is just that they’re going to have to write about those sources on an exam.

Rebecca: I think sometimes demonstrating that you really value a particular practice by spending so much class time on it really helps students understand how that’s a part of the discipline or a way a discipline works.

David: Yeah I think so, and many students really seem to appreciate having quite a bit of discussion time. I mean they find it productive, they have a clear task, they have something they’re working with. So at the end of the semester I do have a lot of students who say, “Wow that was so valuable to hear different perspectives.” And I think especially for first-year students, many of their classes are very large, many of their classes are lecture based, and so they don’t actually have very many opportunities to talk with other students in a meaningful way about the course material. And I realized that maybe at a small liberal arts college it would be a whole different story, but I’m at a regional comprehensive public university and I know that a lot of my students are in that kind of situation.

Rebecca: I think sometimes there’s a mismatch between the assessments that we assign and what our real objectives are…because it’s easier to assess other things than what we really want to know, and that tends to lead faculty to be resistant because of workload concerns. Can you talk a little bit about the grading and how you might manage that?

David: I have to agree with everything you’re saying there. That’s a real conundrum. The most meaningful assessments are also usually the most labor intensive and so I’ve had to be very deliberate in my case about what compromises I’m going to make. And one thing that I’ve had to do with the for and against essays, students write maybe six of these essays during the semester and they’re fairly brief. I really emphasize with the students that there’s not any room for BS in these essays. So there aren’t a lot of preliminaries and so forth, they just dive right in and make their arguments for and against. So that’s one thing that I’m doing. It’s not a formal essay because I don’t have time to go over all that extra content when I’m grading. I think the other thing that I do is most of the feedback I give on those essays I do with the whole class. So I give a kind of collective feedback where I show some examples. I actually fabricate a kind of weak response [LAUGHTER] and put that up on the screen and ask the students to assess it and then I show them a stronger response and then we talk about that. So they’re getting some examples of work that could use improvement and that’s generally a composite. I mean, I don’t ever show bad student work. But then often the strong examples…I’ve asked the students, “Hey can I share this with the class?” and we’ll take a look at that as well. So what I’m getting at here is one way I save time is a lot of the feedback I’m giving is collective feedback. If I were writing extensive comments on every one of these essays for 65 students, it just wouldn’t be feasible. I would have to do something else. So I’ve decided that’s a compromise that’s worth it.

John: When they’re looking at someone else’s work, the work that you fabricated, it’s probably a lot easier for them to recognize problems than when they try to diagnose it in their own, because they’ve taken ownership of theirs and they become committed. But when you prompt them by giving them this type of thing, it’s a whole lot easier for them to see mistakes and recognize them and perhaps avoid them in their own work.

David: Yeah, I think so, or at least that’s my hope. And, you know, they talk with their classmates about it. So they’re getting multiple perspectives on the shortcomings of a weak historical interpretation.

Rebecca: Have you or your colleagues seen a difference in the upper-level classes that build upon this like introductory class where the move has gone from coverage to uncoverage? Have they been more successful at the upper level or do you have any evidence related to that?

David: That’s a really good question and I certainly don’t have systematically collected evidence to respond to that. Now I do see a lot of these students again in upper-level classes and the students I’ve worked with before do seem well prepared to jump into deeper conversations, whether that’s using more primary documents or more extensive secondary sources. So I find that they’re well prepared for upper-level classes as far as what I’m able to observe among my own students. Now, we do still have courses that are closer to the coverage model on my campus, for sure. I mean, different colleagues of mine, they all have their own approaches. And just to be clear, I would never say my approach is the one right way to do anything. It’s something that works for me and that I think really serves the learning outcomes that are important in my discipline and in my department, but also to me. But I think there’s a lot of different ways to get to that. I guess I would say I don’t have any regrets when it comes to upper level classes. I don’t feel like, “Well, these students aren’t well prepared because we didn’t cover everything at the intro level.” I am always reminding myself, “Well, just because I’ve covered something doesn’t mean that anyone has learned it.”

Rebecca: We’ve been talking about this a lot. We have these first-year signature classes on our campus that are for first-year students that may or may not be continuing in a particular discipline but are really meant to help students integrate into our college and acting like a college student, et cetera. So they don’t have as many coverage concerns in terms of topics that we’re historically thinking about. So we’ve been talking a lot about how that frees you up a lot to really focus on some of these ways of thinking. So we’ve been having this conversation a lot, we’ve had a couple of recent podcast episodes related to that but we’ve also had these conversations on campus that have really made me, and I think some of my colleagues, think about, what is it that we really want students to learn even at the upper levels? We think about, “Oh we got to make sure we get through the whole textbook,” or whatever but really at the end of the day, I think sometimes we place value on things that we don’t actually value.

David: Well what you said really resonates for me. I’ve been teaching a first-year seminar for I think five years now and that’s actually in the Environmental Humanities where there is no canon. There is no textbook. And I’ve certainly had to build on scholarship that’s out there. I mean, I’m not just creating this out of thin air, but I didn’t have to worry about the pressure to cover some specific body of material. Nobody was counting on me to be sure to cover these 10 things. And I learned a lot from teaching in that first-year seminar context where the pressure for coverage is largely eliminated and I think it’s made me a better teacher in all of my classes. I really do believe that, because I’ve taken some of that spirit into some more traditional history courses.

John: I think this is an issue that comes up in many disciplines. I know in economics there have been discussions for years. The economics textbooks—just like history textbooks—keep getting bigger, there’s new chapters added. Since I was in the history class, there’s been a whole lot of time included, [LAUGHTER] you know, in terms of the chronological study. But more generally, in economics, there’s a lot of discussion about how difficult it is to cover all of this and how little students seem to come out learning. But there’s a trade-off when sometimes an introductory economics class is the only thing required for perhaps all upper-level courses, so there is some pressure to make sure you cover it. It’s a very active dialogue right now and there’s many attempts to reach a consensus, but I think this is happening in many disciplines, isn’t it?

David: I think so. Several years ago, I was reading some workshops on this very topic for the UW system and when I was doing that, I interviewed some colleagues in other disciplines. I talked with Angela Bauer, who’s now at High Point University in biology and I talked with my colleague Brian Martin in psychology here at UW Green Bay. And what I found was that they were really trying to go down this road in disciplines where the knowledge is more structured. As you go through you really are building on stuff that you learned earlier, maybe in more linear fashion. And in the history major, we’re looking at the history of so many different times and places that no one person can master all of that. So you don’t have quite the same expectation. But in any case, what they were doing was starting out with some big questions and this is something that Wiggins and McTighe recommend in Understanding by Design as well. So you can structure a course around big questions. And those questions should be interesting to most students who are engaging in the discipline particularly if they want to go on. And I find that really helpful. So I do that in my history courses but I think that you can do that in any discipline. Like what are the big issues here? What are the big questions in the discipline? And to answer those questions, you’re going to be using specific disciplinary ways of thinking and disciplinary tools, so you can really start to align things there. Here’s kind of the raw material you’re working with, here are the skills and other kinds of thinking tools you’re going to be using, and here’s what you need to be able to do in the end. If you can really line those things up, I think students walk away with so much more than if they are simply approaching it as, “Okay, we’re covering this, I’m going to take a test on this, and then probably I can forget most of it.”

John: Ken Bain has written about this extensively in terms of picking those big goals and developing it in What the Best College Teachers Do.

David: Yeah.

John: And we just recently had a podcast with Christine Harrington, where she talked about that in terms of building a syllabus, starting with those big questions and building your course around that. It’s a really important topic to help build student motivation and interests and tie everything together.

David: I really admire the work of both Ken Bain and Christine Harrington. Back when I got started with all of this, in addition to Understanding by Design and history, Lendol Calder was doing a lot of work in this area—and he still is today—and I think he pulled this word out of Understanding by Design, but “uncoverage.” So what can we uncover about discipline and disciplinary thinking? Because if we’re using the coverage model, yeah, we’re covering lot of things, but we’re also covering up many of the fundamental aspects of the discipline. If what you have in front of you is a history textbook, it’s written in a single authoritative voice. This is what happened, this is how it was, this is how it is. There’s something really misleading about that, because that’s not how the discipline of history really works. Where did all that knowledge come from? What are the debates? And so forth. So I really liked that idea of uncoverage and I wanted to acknowledge both Understanding by Design and Lendol Calder for sharing that concept with me. I think it’s a really helpful way of thinking about what we should be doing even in our introductory courses, or maybe especially in our introductory courses. What can we do to uncover the key concepts and ways of thinking in our disciplines?

Rebecca: When you decided to shift from a coverage model to more of an uncoverage model, did you decide to just jump in with two feet or did you take more of an iterative approach? I’m sure other faculty who are interested in this idea would like some guidance on that.

David: Well, yeah, so do as I say, not as I did. I really jumped in with two feet and maybe even got in over my head for a while. And you know, that’s something you can do from time to time. I don’t regret doing that. At the time I was a participant in the Wisconsin Teaching Fellows and Scholars Program—which I’ve now been co-director of for six years. It’s actually my final year—but back then I was just getting my feet wet in the scholarship of teaching and learning. And I believe I had a one-course reassignment that year and so I had just a little bit more time to try to implement a pretty radical makeover for my intro history class and there have been many iterations since then, including some pretty significant changes because, of course, you always learn a lot when you make big changes in a classroom. There are always things you would do differently. That said, I think you really can just choose some part of the class and really start working on getting to some of that deeper learning and assessing in new ways. You can try out new ways to assess with new ways to get students working more directly toward your learning outcomes.

Rebecca: You just mentioned the Teaching Fellows and Scholars Program at your institution. Can you tell us a little bit about what that is?

David: This is actually a UW System program so it’s statewide and it’s been around for probably 30 years, although it has evolved quite a bit ―I think by the late 90s, it was taking a turn toward the scholarship of teaching and learning—so it’s a year-long professional development opportunity for instructors in the UW system and we have many campuses here across the state and each of those campuses sends a couple of representatives to participate in this program. We have something we call Faculty College that meets every year and that’s several days, usually in late May, so kind of right after the academic year has ended. You have these really dedicated teachers going right back at it, like, “Hey, we’re going to spend almost a week working on improving teaching and learning.” So that’s actually a broader program that the Wisconsin Teaching Fellows and Scholars participate in that and that’s sort of where we launch the cohort each year. Then a little later in the summer we have a week-long summer institute and the fellows and scholars are actually designing scholarship of teaching and learning projects at that point. So they’re figuring out what kind of research questions would they like to look at in their classes, what kind of evidence of student learning are they going to gather, and so forth. Then we come back together again in January and do some troubleshooting on the project which they’ve been working on through the fall semester and they typically are gathering more data in the spring semester or more evidence to analyze. And then they actually do poster presentations in April at a statewide teaching and learning conference. So it’s a pretty substantive program. A lot of folks do end up continuing to work on their project and ultimately publishing something. It’s not necessarily a requirement, but many of the participants do go on and remain active in the scholarship of teaching and learning. For me, I think it’s really important because it provides a community in the UW system around teaching and learning, and so while you’re actually working on your projects for that year, you have some support. You have a support network to help you figure out how to handle your data analysis or how to get through the IRB process or whatever it might be. But then that community continues on afterwards and so we’ve had quite a few…thinking of three or four different books…published on the scholarship of teaching and learning where many of the participants that were in the book project came out of the Wisconsin Teaching Fellows and Scholars program so there’s the signature pedagogies book, it’s called Exploring Signature Pedagogies and then Exploring More Signature Pedagogies, two wonderful books and Regan Gurung and I recently published collection of essays on Big Picture Pedagogy, and that’s taking a kind of interdisciplinary approach to the scholarship of teaching and learning. And most of the folks in there came out of the UW system, and many of them were former teaching fellows and scholars. So it’s a really interesting program that I think really enriches what we’re doing here in Wisconsin.

John: That sounds like a wonderful program. I wish we had more of that here.

Rebecca: Yeah. Do you have a little bit of advice for people who are interested in starting in the scholarship of teaching and learning? That’s one of the things that we’ve talked about quite a bit on this podcast and anytime we can get someone to provide a little insight into getting that started…

John: In fact, Regan was on a few months back.

David: It can be intimidating to get started in the scholarship of teaching and learning, especially if you’re coming from a discipline that doesn’t normally do research on or with living people. Like, you know, for me as a historian, my research subjects are typically dead and gone. [LAUGHTER] So it’s quite different to think about doing research on my student learning or with my students and their learning. So I think it’s helpful to start small and just to try to think of the question about student learning that you can answer using methodologies that you’re already pretty familiar with. So you don’t want to tackle a project that’s going to require you to learn how to do advanced statistical analysis if you don’t have experience with that already. So I think it’s important to realize there are a lot of different ways to collect and analyze evidence of student learning and if you continue to develop as a SOTL scholar you will explore more and more of those ways of gathering and analyzing evidence. But it’s completely fine to start out with something that you’re already comfortable with and familiar with and you’re really asking a question—or you can start with a simple question about—what are my students learning in this particular area? I know there’s a struggle here, I know there’s something difficult here, and I’m going to pay some extra attention. I’m going to look at this in a more systematic way than I normally would just in the regular course of grading some papers or exams.

John: In your paper on the rise and fall of the coverage model, I think you noted that there’s a growing emphasis on the scholarship of teaching and learning within history and that seems to be resulting in a growing network of scholars who are working in this. Could you tell us just a little bit about that?

David: Sure. Let’s see, back in 2006 a group came together called the International Society for the Scholarship of Teaching and Learning in History and I don’t know exactly what the membership or the participation in that group is but it’s a substantial group of history scholars who are really interested in promoting the scholarship of teaching and learning in history. Along with that, the American Historical Association now maybe for a decade or so has been working on something called the Tuning Project and that’s focusing on learning outcomes for history majors and that project has involved many, many people across the country in all different subfields of history thinking about what is it that we really expect and want history majors to learn and that transcends any particular content area. In other words, if you’re asking that for about history, and we’re thinking about historians coming in who are, not only United States historians, but looking at many different periods of time and in places around the world, you can’t just come up with, “Well here’s a list of twenty thousand facts [LAUGHTER] that we expect all history majors to know.” It was apparent that wasn’t going to work. And so what the Tuning Project has done, it’s allowed our discipline here in the United States to really think carefully about historical thinking and the basic skills and practices of being a historian and that at this point, it’s all very well spelled out. And history departments can then use that to improve the learning outcomes at their program level and you can take that down to the level of individual courses as well. So I think between the scholarship of teaching and learning and this assessment oriented Tuning Project, that we’ve had a real increase in interest and commitment here when it comes to thinking about what does it mean to learn history and to learn how to do history?

John: The last question we always ask is, what are you doing next?

David: I’m happy to say I have a sabbatical a year from now, just a one-semester sabbatical in the spring of 2020, and I’ve really become interested in environmental history and more broadly environmental humanities (I mentioned that earlier in terms of the first-year seminar I’ve been teaching). And my project is going to be to try to articulate a pedagogy for environmental history and environmental humanities and what I’m thinking of there is that there’s more to this field of study than the usual kind of content and skill mastery. And I think there are a lot of areas where this is true, but there seems to me to be a kind of affective component here, an emotional component, as we’re facing up to the environmental degradation and numerous environmental crises in our time. I say facing up to—sometimes we’re not facing up to those problems—and it can just be overwhelming from the standpoint of a learner—especially a young person—to really come to terms with the content that we would have to look at to the areas of environmental studies more generally. So I’m looking at contemplative pedagogies to think about one way to help students come to terms with and to deeply process what it is we’re studying when we look at problems with sustainability. For example, climate change. I’m really excited to dig into that next year.

John: Those sound like fascinating topics, and they should be fun to look at. And it’ll be an interesting challenge working with those issues, especially now.

Rebecca: Yeah it sounds really intense, but really needed.

David: Yeah, I think it is going to be necessary to grapple with those issues going forward.

John: Thank you for joining us. This has been really interesting.

Rebecca: Yeah, thank you so much.

David: Oh, thank you. I really enjoyed the conversation and my tea cup is empty, I don’t know about yours but I think I’m going to have to go get another cup.

John: Yeah, mine’s empty too.

Rebecca: Yeah mine’s getting really close.

John: I don’t know if you can see it, but we have a whole table covered with tea back there in the back of the room.

David: Oh I see that now.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

68. Mobile Music instruction

There are apps for just about everything but choosing when to embrace them for instruction needs to be a careful decision. In this episode, Trevor Jorgensen joins us to discuss how the decision to use mobile apps in music instruction is affected by where students are developmentally, convenience, cost, and other factors. Trevor is an Assistant Professor of Music and the Student Learning Outcomes Assessment Coordinator at SUNY Oswego.

Show Notes

Additional Resources

Transcript

John: There are apps for just about everything but choosing when to embrace them for instruction needs to be a careful decision. In this episode we consider where students are developmentally in a discipline, convenience, cost, and other factors on the choice of supporting music instruction using mobile applications.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Trevor Jorgensen, an Assistant Professor of Music and Student Learning Outcomes Assessment Coordinator at SUNY Oswego. Welcome, Trevor.

John: Welcome.

Trevor: Hey.

John: Our teas today are:

Trevor: Pure Leaf Peach Flavor tea.

John: Iced tea.

Trevor: Iced tea. Sorry. Thank you.

John: …and Rebecca.

Rebecca: I have Chai today…

John:…and I have black raspberry green tea. We’ve invited you here to talk about how you’ve been using mobile technology in applied music instruction, specifically related to performance. Could you tell us a bit about that?

Trevor: Yeah, I think one of the most important things is to decide when to use technology where it benefits as opposed to be either a substitute for something you should learn in a different way, or whether you should implement it because it’s just convenient. Sometimes when it’s convenient, though, it takes away some of the learning behind it, and I’ll get into some of that later. So I think that’s the first thing. I have to make that decision on every app that I use, or every device that I employ. So, one of the main things that I do is decide what’s best for the student and what’s best for me. And a lot of it does substitute for other things, but in the end, it can’t. For instance, if I do a collaborative piano thing with my colleague, there’s a lot of tools that we’ll talk about today that will substitute for that collaboration. Except for, I cannot phrase with the app in the the same way that I can. We can’t make decisions on a moment’s notice, like I can with another musician. We can’t have talks… at least I shouldn’t be talking to my apps… about what the best way to do something is and then make that interpretation or change colors of stuff. But what it allows me to do is, instead of having a more rehearsals with somebody that’s really busy, it allows me to rehearse by myself. And then therefore, save some time when we’re actually in the rehearsal to talk about those musical decisions as opposed to just to try to learn notes or trying to learn one anothers parts.

John: What are some of the apps that you use with instruction or performance?

Trevor: One of my favorite ones is Amazing Slow Downer, and I use this for both jazz and classical, being a saxophonist, and also a multi-reed player…I play clarinet, oboe, and bassoon. I like to use it in every circumstance I can. And it’s really a great app because even though there’s things like YouTube where you can now click over the three little dots in the corner, and you can slow down to 25%, 50% and 75% and speed it up if you so desire. This tool is just something that I always have with me and I can always use. And I’m also more familiar with it, it allows me to do multiple things. So let me give you example if I can play musical selection here. The first two are not going to be on the music slower downer, it’s going to be on Amazon Music. But I want you to listen. These are Brahms’ Clarinet Trio. I performed this with a colleague and somebody from Symphoria recently. And one of the things you’re doing as a musician is you have to interpret the history of music, and that includes listening. So if I listened to multiple recordings of something, you know, we always ask “Why are there a thousand recordings—might be overstated—but of Beethoven’s Ninth Symphony?” That’s because the composer had a certain thing in mind. And then the conductor who prepares the music, and the individuals in that group have different ways of interpreting that music and every one is different. But for a lot of younger students, and sometimes myself when it’s really advanced players, I’m not exactly sure what they’re doing, either articulation wise or what they’re doing phrasing wise. And the slower downer allows me to think about what they’re doing and compare all these great artists together. So I’ll do multiple recordings on something like this Brahms’ clarinet trio in A minor and have my students listen to it. I’m going to play one of them for you, and then I’ll play another one, and then I’ll play the third one. And then the third one I’ll slow down to show you how we can listen for different articulations. You may or may not tell the difference between the three, some of them are obvious, like there’s a tempo difference between them. But as far as the phrasing and the articulation used by the cellist, and by that clarinetist, and by the pianist, is something to think about. What do I want to sound like? It’s great to imitate people, but it’s also great to make decisions between all these different things you’re, in a way, sourcing and decide what you want to do and why you’re doing those musical decisions.

Rebecca: Before we jump into listening, for our audience who maybe is not as familiar with the music terms that you’re using? Can you clarify the difference between phrasing and articulation?

Trevor: Sure. Great. So articulation is the front of the note, and thank you for your much for that. So as a saxophonist, it’s how I tongue the note, if I put an emphasis through air or through my tongue. Another word for that is “attack,” how it attacks into it. For a string player, it might be the direction—up bow or down bow—how fast they’re moving the bow. For the pianist, how they voice the chords and are there some fingers stronger than the others. A third comes out of the chord and the fifth comes out of the chord. For a chord you have multiple, you know 1-3-5s, or the seventh of the chord. And each one of those, as a pianist, when they’re putting down multiple fingers, they can decide which ones are the most important. Whether it be the melody, whether it be the bass part, whether it be the inner harmony. So, from a pianist’s point-of-view, they can listen for that, or how the touches on the piano, how much they pressurize it, or if they use pedal in this area that’s not dictated by the score exactly. So phrasing, the best way to describe it is like minute dynamics. Or just using what we normally do. We, as human beings phrase, if you like somebody you change your voice, or if you get excited like, “I have go to the bathroom,” you don’t think, “Hey, I should elevate the dynamic of my voice. I should make it sound strained, so that I show the excitement of me having to go to the bathroom.” But in music, we have to think about that. We have to think about where, based on the harmony that’s below the music, how much am I going to emphasize that note or more importantly, maybe even growth of that note, or that musical phrase. So phrasing’s how people interpret the music beyond playing the right notes in the right place, and doing the pianoforte dynamics; it’s how they’re going to go to a note and resolve a note. And not only that, they think about overall structure of a piece for phrasing and how everything fits together, if they’re amazing musicians. When we listen to amazing musicians—like the ones who found in these recordings, then what we’re going to do is interpret what they did. And then maybe in our own minds figure out why you think they did that, if that’s a choice that you like to do based on the harmony or just a different tone color they do. And that’s obviously easier to figure out when you slow down to 70% than it is when you’re listening to it at 100%. So before I play the examples, the actual artists will be in the show notes below in the description. First one is, I’m just going to name the clarinetist, Karl Leister, and this is them playing the Brahms’ Trio. And I’ll do the first 30 seconds or so, so you can hear how it sounds.

[KARL LEISTER PLAYING BRAHMS’ TRIO]

Trevor: The next example will be by the clarinetist David Shifrin, that trio that he’s in. I’ll play about the same amount of time for it.

[DAVID SHIFRIN PLAYING BRAHMS’ TRIO]

Trevor: The last one I’ll do on Amazing Slower Downer, the app. And we’ll actually play it in tempo first, since its Martin Frost as the clarinetist, playing the same piece.

[MARTIN FROST PLAYING BRAHMS’ TRIO]

Trevor: So what we’ll do now is we’ll actually slow it down to like 70% so we can actually hear a little bit better the articulations, the dynamics—as the artist gets louder, you’ll be able to hear it more individual. And we can even slow it down after that into something ridiculous like 25%, where you can actually hear the front of each note. It’s almost painstaking how slow it is, but interpreting different things, it’s interesting to see if they use a breath attack or a tongue attack, and some of them are so subtle, I’d have to listen to it 25 times without slowing it down where I can actually hear it at 25% right away. So let’s go back and do it about 70%.

[BRAHMS’ TRIO PLAYED AT 70% SPEED]

Trevor: Here’s 25%.

[BRAHMS’ TRIO PLAYED AT 25% SPEED]

Trevor: I’m just going to jump ahead a little bit so we get into the clarinetist and stuff. I’m just going to scroll ahead.

[BRAHMS’ TRIO PLAYED AT 25% SPEED]

Trevor: First of all it’s very unfair because I would never want to be heard at 25%—it’s just impossible to keep a phrase sometimes or keep a note perfectly that way—but what it does is for us is we can actually hear them the front of the note… we can hear the shape of that note even if it’s a quarter note if he—in this case it’s a he—de-crescendos or crescendos into it and we can look at the overall structure of the dynamics too. You can see if he’s actually growing on that note, if you’re taping the note as I said before… the overall picture or phrase too. I think 25% might be a little excessive, you have to find the happy medium based on the tempo of the piece. I think initially 25% for this one might have been a little painstaking at the beginning.

John: But for students, perhaps having the ability to slow it down to that level would be much more helpful than for someone with more experience?

Trevor: Right, there’s definitely that and then we’re looking into transcription later on too. There’s this great book called School for Cool: The Academic Jazz Program and the Paradox of Institutionalized Creativity by Eitan Wilf and I hope I’m pronouncing his name right. He wrote it in 2014 and he basically went to Berklee School of Music and then The New School in Manhattan. And he noticed that students were using this application even back then—and when I say back then is only five years ago, or four years ago, or three, because in the time of technology, that’s way back then—and they were actually interpreting, transcribing not only John Coltrane and getting his notes like I would have done when I was in grad school or undergrad, but also then listening to his articulation at the front of the note. So trying to make their tongue sound exactly like that slow, and then they’d be speeding it up. Also, in a real fast passage we use a fake fingering. What fake fingering is, is there’s a normal way of fingering D on saxophone which is like 123-123 fingers down and then there’s a way of playing the high D but not using the octave key so it comes out in kind of a muffled way, I could probably demonstrate it for you if you’d like.

[TREVOR PLAYS D NOTE 3 DIFFERENT WAYS]

Trevor: So you use three different fingerings for the same note, which I’d probably not do in classical, but for jazz, it adds a different color. A lot of people know certain people will play that way so that you can actually determine those notes when it’s really fast it’s hard to tell. Just that nuanced playing that you can actually interpret when you’re listening to John Coltrane, that I didn’t when I was that age of the students that he was, you know, surveying. But, slowing it down, it’s not something new. There’s another great book too, Thinking in Jazz: The Infinite Art of Improvisation by Paul Berliner who was published in 1994. He talks about how the old school players would get a vinyl record and they’d put their thumb on it and slow it down to like a third of the speed and then obviously transpose. Now they would be in the wrong key and way too low, and then they just transpose it up. So this technology, or using this technology, has been available as far as transcription. But if you did put your thumb on a record, it would probably distort the sound and the way that you can figure out articulation and you couldn’t figure out phrasing.

John: And the advantage of this is that students can preserve the pitch because all these apps maintain the pitch and you can hear the attack… just more slowly.

Trevor: This one is specifically and I’m sure a lot of things you can probably do this on YouTube. Also, a lot of my students have to in order to avoid buying a $15 app like this one is—at least when I bought it it was—they’ll find other sources, which I think is great. It doesn’t matter to me what they use as long as it does the same thing. I think on YouTube, you can do the same thing where you can loop it, and there is these three seconds or 10 seconds or whatever, and just loop it and here you can loop it. You can also save the file, save the section of the piece, call it what you want, and the next time you open up the app, it’s there. And you can also share your ideas with people. You can download stuff from things you buy on iTunes or you can use Spotify and there are other ways of importing into this app that’s really beneficial too. The nice part about this app is it can be used for classical and jazz. Most people use it mainly for jazz and transcription but I use it for everything and additionally you can actually slow your own self down which is always painful and hear that you weren’t phrasing that well or that your end of notes aren’t as tapered as they could be. And once you listen to amazing players you realize that they have control over everything. I’ve slowed myself down. I always know the differences between me and like the best person in the world, although I knew already but but it’s so evident when you slow it down to 50% and you hear the attack of your note is not as clear and not as shaped as the people that are just geniuses.

John: And you can also use that with students, though, to show them what they’re doing, right?

Trevor: Exactly. I mean, it’s a very revealing tool. So as far as students are concerned, I think it’s great tool for them but also can be somewhat depressing to hear yourself. It has to be a good balance and the student has to be at the point where we’re finding things not initially at the beginning, or maybe they’re even in their freshman or sophomore year, but at the end when they’re preparing for grad school or something like that, and it depends on the student as it does with every situation… what you can push them on and what you can’t. But, it’s just a great tool for that. So I also use the Amazing Slower Downer when I have a piece that I don’t have a—we’ll talk about Smart Music later where it is an intuitive accompaniment tool that takes away the saxophone or takes away or whatever it is you’re working on—but when it’s in pieces not in there or I don’t know the piece, I’ll find a recording of it and then I’ll play along with it. And this I can adjust a little bit of pitch if I need to and more importantly speed if I needed to get slower or I decided to take the piece at the end faster. Maybe now’s the time to talk about what the disadvantage of some technology is. I’ve got colleagues that don’t like using slower downers, not just colleagues but I’ve read articles and everything else, because you don’t have to go through the work and the persistence of doing so, and you should be able to hear it in the moment in the spot. And I think there’s something to be said for that. But I look at myself as one of the things where I get discouraged. I always think about one of my favorite reads, which is a psychologist by the name of Csikszentmihalyi and he’s got the flow theory, which is a book he published in 1990, which a lot of people are aware of. And number three of his flow theory is that there has to be a balance between the challenge and skills. For me, there wasn’t, my skills were low and my patience were also maybe lower when I was younger. [LAUGHTER]

John: And I think that’s true for most starting students. And a lot of people get discouraged when they don’t see that immediate progress.

Trevor: Yeah, that’s exactly it. So for me, I was able to use the tools that helped me a lot more. I painstakingly did it when I was my students’ age that was also because of the slower downers were $400 and I didn’t have the money. Now most students have phones but they use them for social media or actually using them as phones, or texting more likely. But now you can just download an app instead of buying a $400 machine for about $15, or you can go to YouTube and use that function or other tools that come with any kind of PC nowadays. Microsoft has their recording software and you can slow it down too. Not in such a certain way, but any of those tools are great too. Anyways I’ll slow my own recordings down of me playing along with other things. But the problem is trying to figure out that balance where I should be taking it to the tempo there I just took it or slowing it down. You just have to progress and move it forward so that you take it faster, faster each time. The thing about anything is once we do it more often, it becomes easier.

John: And the more they practice with this at a slow pace, the better they’ll be able to do it in real time as well.

Trevor: Right, and their ears develop immediately, faster and faster, and then therefore they can get to that point. I commend my colleagues and all the articles that I’ve read where you know, it’s better to do it that way. But for me, it was a wall, as it is for some of my students.

Rebecca: Sometimes it’s good to recognize when you are a beginner and that sometimes you need those training wheels just to feel like you got a little success.

Trevor: Right. That success as we all know it from pedagogy is that wields more success and it gets better and better every time. Let’s go on to a different app. Some of the obvious apps are metronome apps, every student can get a free metronome. I was excited in the mid 2000s when they dropped down to $20 as opposed to like $200, or the tuner is the same way, and then balancing that out. For me it’s I could buy a $20 tuner and put one on every one of my instruments but I always have my cell phone on me that’s what I love about it. I always have a tuner, I always have a metronome and I have no excuse, nor do my students. You know it’s a free app; you already own the phone. But there are some tuners and some metronomes that are better than others and I am not an authority, I don’t know every app that’s out there because there’s thousands. The ones that I know are the ones that I like… people have introduced to me and I’ll switch apps all the time if I find one that’s better. As far as the tuner or metronome it doesn’t really matter what my student brings in… whether she brings in the one that I prefer, like Cleartune for everyday apps, or if she brings in like TonalEnergy Tuner, but I’m going to talk about that one because it’s one that I found really interesting and it has a lot of functions on it. And it’s not only a tuner but a metronome. I use Tempo app for metronome primarily when I’m practicing but I’m getting into the TonalEnergy app for inside of it has also a metronome that I’m trying to investigate. So the app has so many different features to it, and I’ll show you and then maybe both of you can also chime in on what you think they look like, but…

John: I use the Cleartune app myself.

Trevor: Yeah, I loved that app until I saw this app. To give you a total side story, this one does analysis by waveform as it does it by another analysis and note names, too. This weekend, I judged a competition for high school students to get into all-county bands. And one of the other judges in the brass room said one of the students—and I don’t know the name of the student nor the judge—but the performance was really not great at all, where he actually pulled out his TonalEnergy Tuner to figure out what note the student was on to see if he could find out where he or she was was in the piece. That being the extreme of what the app is used for… nor do you want to use it for transcription because there are transcription apps where you plug it in and it’ll transcribe it for you. For playing our transcriptions—there’s worth to all those things—as far as actual transcriptions… a tool that you should have so that it ingrains in your memory and you have those phrases for jazz specifically. So this one here you can choose from waveform, you can do spectral harmonic analysis and note analysis. As you can see right now as we’re talking, sometimes I’m talking in tune and he gives me a happy face—it could be a she—gives me a smiley face, or if I’m really badly out of tune, there’s this little question mark with the thumb and finger or very sadness as you get way out of tune. But more importantly, as I do the waveform thing, it will record what I do.

Rebecca: I was also noticing that the color was changing as well. So, there was a lot of different identifiers to the user so if you had a certain kind of disability it’s actually providing that information in multiple ways.

Trevor: Yeah, and that’s great. And then over on the side it tells you pluses and minuses, I’m 13 cents sharp or whatever, 13.6 or flat. So, I can show the pitch, I can show the wave, and I can record it. Let’s go ahead and record it. You’ll see as we’re talking the spectrum, and then if I do with the notes, it’ll show me the notes, which ones they are, and how badly out of tune they are and we’re just talking. I could play my saxophone. But it would give you the same thing.

Rebecca: Right.

Trevor: So as far as I’m concerned, if I turn this away, and I can record it, and then go back and look at it, I’ll know that if I do nothing, or how much I need to do on the D. The D that I played for you before, nd gave you three different examples is the worse out-of-tune note on saxophone. So, I probably was sharp but this will tell me every note that I need to. You can even slow it down and stuff. You can also record your student visually and with audio. The visual thing is interesting because a lot of movement in the face will affect tuning. When I came to college, I chewed a lot, I had movements in my throat and my face and my teacher would just tell me what they were. But now I can like hold the video against them, show what their movements are, and how it’s affecting tuning and it’s, you know, just another another tool to get you there beyond just recording their facial features and hearing it.

Rebecca: It’s an interesting way to note take because it’s a visual note-taking method that you’re not just having to remember the conversation that you had with your faculty member or mentor, but that you can go back to and see again, if you’re able to record it and save it for later.

Trevor: Right. Yeah, I know, it’s great. It’s a neat tool. So, I’m starting to love this tuner and this metronome as much as the others. It does a lot of specific things where if I’m just pulling out a tuner like Cleartune, as we talked about, or Tempo, that’s more my functional as opposed to how badly I’m out of tune when I’m playing this passage. And the same thing I notice whether I’m using this app, TonalEnergy Tuner, or I’m using Amazing Slower Downer, or I’m using what we’ll talk about later, Smart Music, is that when I’m playing along with the pitch, in the case of Amazing Slower Downer, I know I can hear I’m out of tune, and then I can check it against this or if I’m playing along with Smart Music and you check it against here and see where I’m out. And I play multiple instruments so I’d like to tell you I’m perfectly in tune on every instrument but sometimes I’ll be playing A clarinet and I haven’t played that for a while where I normally do B-flat clarinet and I’m like, “Oh, that’s right. This B-flat is flat on the A but sharp on the…” you know, you have to adjust and unlearn and it allows me to do that prior to getting in there with my collaborative pianist and realizing that I’m out of tune. It’s a great tool. It’s got so many more features that I’m going to start looking into but there are funny features like… not really funny… but the band director or the the judge using it to try to figure out where they were, which hopefully we won’t have to use it very often for.

Rebecca: I wonder for a beginning student who has very little experience… maybe someone who’s not in the music program, but is learning an instrument. Have you worked with students using these kinds of apps with that population of students… like total beginners?

Trevor: Prior to teaching here at SUNY Oswego, I was teaching beginning students… so, seven to eighth in junior high program and then had fifth and sixth grade students. I haven’t used it and mainly because of physicality. Until you develop your armature, until you learn how to use the air, it’s almost impossible to use this effectively because if you don’t have the proper air you can’t make those minute adjustments. That being said, to get the initial tuning for every instrument, whether you’re in fifth grade or fourth grade, you do want to make sure the violin is in tune so students will start using that, you do want to make sure that your concert B-flat’s in tune, which only means after that all the other 11 notes are out of tune, but it’s a good reference point. And then students start to use tuners that way, and amazingly some of them like apps and stuff, and they get into and they show me different things they’re using for that I never thought. But initially, I try not… until those physical things are employed properly. It’s almost futile to actually do it all the time.

John: They need the fundamentals before you can work on tweaking.

Trevor: Right, yeah, especially for you know, with piano you don’t have to worry about tuning. Guitar, that’s got frets so it’s good to know how to tune it. You know that more…

John: You’ve never seen my piano, but… [LAUGHTER]

Trevor: Yeah, but yeah, no… We’ll talk about the next thing, Smart Music, and how it’s used more, or at least, I think more band directors in the public schools use it at elementary all the way through high school, then maybe as far as band music and material is than college professors. I use it for the solo literature, like clarinet concertos and sonatas ….but in the public schools they’re using this technology we’ll get into to not only evaluate their students—how they’re doing, to keep track of their practicing—but as a preparation tool, it’s great. And it can also be not as great for certain specific things and we’ll talk about that when we get there. The next tool—and we’re kind of focusing on my phone right now—the only app that is not available on a phone, a mobile platform that way, but is only on an iPad—is the Smart Music. As I said, we’ll get into that later. The next app I like a lot and I just got an email right before this, there’s a physics professor on campus who’s named Shashi and he’s a saxophone player. And he does exceptionally well, he’s way better of a saxophone player than I am in physics by any means. [LAUGHTER] He loves it. But he gave me an update “I used iReal Pro yesterday at an open jam session. I played these tunes” and went “Great, I’m excited.” It’s great to see that it can be used by by anyone. And as I said, he’s come miles and he loves this because for a physics professor he probably jams with people and he’s been doing a lot more of that. But initially, he’s like, “Who do I get to play with?” and if you don’t play music with somebody—it’s a social thing too… or people hear it—then you kind of like “Why am I doing this?” Even myself as a professional musician, my colleagues and I will prepare harder and more for either recordings or performances coming up then we will during the summer when there’s nothing to do. That being said, we’re preparing always for the next semester, but it’s always great to have that motivation. If I never played with anybody I probably wouldn’t continue playing, especially being a saxophone. A pianist is different, a guitarist is different, perhaps. Drummer is probably different too. I can’t imagine one person only being a drummer and doing that for the rest of life without collaborating with someone.

John: Did you tell us what iReal Pro is?

Trevor: In jazz, there’s these things called real books or fake books. Now, they’re legal. Back about 20, 30 years ago you can only buy them through illegal sources, and they were the melodies of famous popular music standards and jazz compositions, with chord changes behind them. And then you would get together with musicians and you’d play. Now back in the 70s, a guy by the name of Jamey Aebersold used professional musicians who recorded them on CDs—or back then I guess vinyl—and then gave a play along book with it. So he’s got the Jamey Aebersold play along. And what this is done is taking non-live musicians and put it into an app where you can play along with. You can see the chord changes. There are no melodies because you’d have to pay money and royalties for the melodies. But there is no royalties needed for chord changes, and so of the title of the tune and then the chord changes. It comes with no actual pieces on it until you go to the forum and then you can download thousands of pieces that are already programmed in that you can play along with. So I’m just going to pick up right now blues “Straight, No Chaser” by Thelonious Monk. Although it could be any blues until you hear the head and I’m just going to play you a little bit of how it sounds and then I’ll give you a visual description later about it.

[“Straight, No Chaser” PLAYS]

As you notice, when the music goes by, or the chord changes goes by, there’s a highlighted measure so that you know where you are, which is handy for students that are learning to improvise. Later on it’s not necessary, you could just look off a real book lead sheet and play along with it. There’s many features, one I can make the transposing instrument like I’m playing saxophone in the key of B-flat instead of C. I can see the chords transposed in front of me even though it’s playing the key of C. I can do that for any instrument. There’s like a balancing… if I’m a pianist, I can take the piano out or if I’m a drummer, I can take the drummer out. I’m not sure why a drummer would want to do that. And I can control the reverb… I can change the tempo… I can change the style—this is a jazz medium up swing—I can change it to almost anything—Latin, Gypsy jazz, trio guitar, slow swing, traditional jazz—and there’s pop and Latin stuff so I can change the beat behind. I don’t do that as much unless you’re working in the Latin situation but it’s nice to have and as I said, you can also program pieces in. So we just did a concert with this great alto saxophone player by the name of Dick Oatts down in the Village Vanguard Band—that’s where he is primarily and teaches at Temple University—a world renowned musician. And he sent us his charts which are really hard and don’t follow normal progressions. They’re really kind of tricky so I actually programmed them, or one of them, into here just so I could play along with it a little bit… hear the changes a little bit better. I mean I’ll voice them out on my saxophone like normal but it’s a really handy tool to have. And speed it up too like I can’t take it to the tempo he wants it so I’m gonna slow it down and feel good about myself and they’re not really that odd but they’re more unusual chord changes. They made sense but just needed more time to practice and I just programmed in it by…you kind of scroll up and type in F major and put a seven and you can do all the different things. It’s really interesting and you can save them and if you want and you can share them on the forum so other people can look at tunes. So this is an amazing tool. I love it for that because I can speed it up and slow it down, but there are other options. I think Aebersold things are still available online unfortunately for Aebersold ‘cause you know, he’s losing royalties and stuff on that. But then on YouTube, there’s “Learn to Play Jazz” I think if you type that in and whatever tune you want to do there’s that for the people that actually have iReal Pro copies on YouTube. You can’t slow it down, you can’t transpose it. You can adjust the balance and blend and stuff but it’s there. So my students will find those things too and use them. And I’m totally fine with that. You know, financially it makes sense. One of my favorite features is…and John, you have this app you said?

John: Yeah, I do.

Trevor: So you use this app, and have you have you looked at the tablature part of the app?

John: I haven’t been using it recently. Since we started the podcast I haven’t been playing so much music.

[LAUGHTER]

Trevor: Got ya. Give me a second here. So I’m going to show you…and do you play an instrument at all?

Rebecca: No. That’s why I was asking all those music questions.

Trevor: And please continue to do so. [LAUGHTER] In the corner of the app there’s this little thing that looks like a tablature and you can choose from multiple things. One is for guitar so you can select guitar chords. You can do piano, and the piano you can do one hand or two hands. You can do ukulele, which is a new feature and more importantly for me is the chord scale library which I use all the time. So let’s go back and add the chord scale library. So if I press play and then I pause it right away it’ll give me chords down at the bottom. So for that C7 chord I can do either dominant mixolydian chord if I scroll over. Actually for this one, there’s only one, but for a lot of times when it’s a different chord like flat 5 or something like that it’ll give you multiple options of what scaless you can use over top of it. Once you talk to a pianist—they’re not really fond of this app, because it’s not exactly like you would teach jazz piano—but if you have no skills, and you want to learn from an app, it can give you some things. It’s a really nice tool for myself and all of my students too. And as I said, our physics professor on campus uses it a lot and as do you prior to doing the podcast.

John: I used to use it more before we started this podcast.

[LAUGHTER]

Rebecca: Trevor, we found that students were using these tools—or when you’re using these as part of your teaching—that the students abilities have improved or that they’re playing has improved. Have you noticed a difference in the integration or?

Trevor: In some regards more about how they practice and how long they practice. But yeah, I’ve noticed an improvement. By having no one to play along with it’s difficult to improvise—you improvise in different ways—it’s a very valuable skill to be able to cut the changes… and me as a saxophone player, if I play and my colleague walks in the room they say, “Oh, you’re playing this tune” based on the solo I’m doing then I know I’m really cutting the changes. But when I don’t know a tune as well, it’s nice to have the harmonic progression so I can use my ear a little bit more than I can my mind and eventually, it’s a combination of both, I’d hope. But my students will then play along with the blues a lot more than they would if they were just doing it by themselves.

John: And much of the benefit is students can work and get up to a point where they’re ready to play with other students so that everyone can use their time more effectively when they actually do collaborate and meet at once. Rather than each person struggling just to do their own parts, they get to work out playing it together.

Trevor: Exactly, it saves so much time. In addition to that, when we think about rural areas. A lot of times you’ll be able to find a great collaborative pianist for doing your solo so it might be okay to use Smart Music—I hate to say that—or you might not have a chance to prepare so you play along with records—or you play along with, in the old days—and now you can play along with this. You know, even getting out and playing along with it at open mic night it’s just totally acceptable because it’s expensive to find musicians and especially in different places. Where in New York City I think would never see anybody using iReal Pro to play along with at a club, you know, just because…

John: …There’s a few musicians there, yeah.

Trevor: But I’ve heard of people pulling it out and it does irritate people. There is the idea that once you graduate from a jazz program—specifically, if you focus on jazz. I myself, did music education. I play jazz a lot and I also did classical clarinet, classical saxophone—but if you’re only a jazz major, or if you’re good jazz player, you’ll have like 150 to 200 tunes memorized. So you’ll call a tune, you’ll have the head, and you’ll know the chord progressions. But nowadays, a lot of the younger players are… from what I hear and what I hear from complaints from older generation people is that they’ll pull out the iReal Pro if they don’t know a tune and play along with it. And there’s pluses and minuses. Now you can actually play that tune. The minus is it’s not as ingrained as if you had to learn it and memorize it and ingrain in yourself, and that you won’t be able to interact with the high level players you’re playing with in the same way as if you did that work. So there’s, you know, again, pluses and minuses. It never should be a substitute for the ideal, but it should be a tool like you just said to get you there. Or in the cases of where there’s not a bass player to play with, at least it gives you a way to proceed until you can find one.

Rebecca: How do students respond to using these tools?

Trevor: They like them. They enjoy them. They think they’re really neat. This freshman student that came in—he was using YouTube, one of the play alongs he warmed up to it—but yet again, the disadvantage of this is all electronic and they sound electronic. So at least some of the “Learn to Play Jazz” are actually real jazz musicians. But you can alter the tempo—or you can’t—I guess you can with the 75%, 50%. I’ve never tried it, now that I think of it, on YouTube…but you know, it just it takes adjustment. You know, there’s nothing to substitute live musicians. The drummer can’t react to what phrase you just played. Nor can the keyboard voice omething different if you’re going there harmonically. But it is a tool. And yet again, a tool that’s not a substitute, but a tool that’s effective in preparing. The next tool we’ll talk about is actually on my iPad, and it’s Smart Music. Smart Music is an app which originally was on a computer and way back in the 90s when it came out it looked like the old Atari cartridges you’d buy with the machine for $2,000 and each cartridge would be like $80 or something and have one piece—Mozart’s Clarinet Concerto. But nowadays it comes on an iPad and you can also get it on your computer. The computer has more features on it but obviously is not as portable as the iPad. And what it does and what it’s being used for is amazing, but also can be negative and we’ll talk about that after we talk about the positives. It has all the band music that you’ll want to play in public school and what the band director can do—including myself as a band director—is that I can say, “I want you to play Holst’s ‘First Suite,’” is which we picked here, and gives you the saxophone part but I can also push the button as you can see, and I can pick whatever instrument I’m playing. So I’m playing bassoon, it gives you the bassoon part. It has the music in front of you and then once I play along with the rest of the band without that bassoon part or that saxophone part, it has little X’s and O’s and tells me what percentage of the music I got right rhythmically and everything else. So band directors will use this to say, “I want you to record this and get it to me and then that’s your percentage on it.” Now it does have some flaws. I’ve listened to students play and myself play and it’s no way that I’m getting 60% I’m getting closer to 85 or something like that. So some advantages and it’s gotten better over the years.

Rebecca: So just to clarify, it’s like Guitar Hero video game?

Trevor: To clarify, Guitar Hero has nothing to do with guitar.

Rebecca: Right. Yeah.

Trevor: It’s more of a rhythmic pushing buttons…

John: Pattern matching, yeah.

Trevor: Pattern matching and it has something to do with the music because it’s actually, you’re hitting it rhythmically along with the song, that’s about it. But you’re right, it’s like a play along, it’s like karaoke without the part you’re on. But this karaoke, in this case, records where your notes are wrong with red and green, where you’re behind, and it tells you where you’ve missed things, and then that goes back to the band director and they can grade you accordingly. So it saves them time.

Rebecca: The visual cues are similar is really what I’m…

Trevor: Oh I see what you’re saying. Yeah, I haven’t I haven’t played a lot of Guitar Hero and I apologize. Most of my knowledge of Guitar Hero’s from South Park and we won’t talk about that. So it allows you to do all those different things and then record them. As far as soloists are concerned, that’s where I use it more. I use it more to play along with a collaborative pianist that’s not there. And it follows you. So for instance, I’m going to pick the clarinet Copland concerto that I’ve prepared for, and it’s got a piano reduction to it. It’s a very difficult piece to put together when you’re a clarinetists like me, who’s a saxophone player that also plays clarinet well. So I use it a lot of time to prepare before I meet with my collaborative pianist. And I’m going to pick an arbitrary section in the middle so you can hear the piano part. And the piano part will play and then once I play along with it—and I’m not actually going to play—it will record it, and I can play it back so I hear all my mistakes which is really nice, and also my pitch errors and stuff. And then if I am unsure of myself, I can actually set it so that it will play with the clarinet part along with me so that I can hear that part to make sure I’m rhythmically accurate with it. That comes to one of the problems with being able to play along with something that has 100% rhythmic accuracy, but you don’t. You know, our ear can pick up on things so quickly that… let me give you the example. When I was in high school, there’s this lady named Amanda and she was the first-chair alto saxophone. And Amanda would have the rhythm perfectly the first time and then me, who would after listening to her having it perfect the fifth or sixth time, so I didn’t really learn how to count properly. And by playing along with the Smart Music, where the piano is actually playing your saxophone part or clarinet part, and you’ve been listening to that. Then when it comes to actual real performances, if you play along with it all the time, then you’re relying on it so much and your rhythm goes out the window. So that’s where some technology can get in your way, specifically with rhythmic attributes, but also with other things too. But yet again, it’s a tool and if you recognize that, you know, I prepare it rhythmically with the students first, I don’t let them use Smart Music until they’re at almost the performance level. So we’re going to play a little bit of the Copland Clarinet Concerto somewhere, but 140 measures into it. I’m going to play it initially with the accompaniment but with nobody playing, and then I’m going to set it and you can hear the difference now with the clarinet part being played by a different sounding piano.

[COPLAND’S CLARINET CONCERTO PLAYS ON PIANO]

So that was just the piano alone. Now I will do it with the clarinet part being played by a secondary piano, and you’ll hear how it fits in and that’d be what I’d be playing along with if I needed the rhythmic help or I wanted to check my pitch.

[COPLAND’S CLARINET CONCERTO PLAYS ON PIANO WITH SECONDARY PIANO]

That wasn’t as obvious an example but if you knew the part you’d be able to hear there’s extra additional things added to that which should be the clarinetist. So this tool is just remarkable for that and as I said, for band directors in public school and at the university level too, it seems like an appealing thing. I just haven’t figured out a way to automate it or include it in my, you know, as part of the practicing routine for my students. And if they don’t come from it from the schools, it’s a little bit more difficult to implement. I’m not sure at the university level it’s ideal for me to implement. I’m still wrestling with that as I am with implementing any technology.

John: So how have your colleagues responded?

Trevor: One of my colleagues, Rob Auler, he uses it as much or more than I do, we always check out, “Hey man, check this app out. Do you think it works really well?” So we check out different apps and sometimes those apps apply—or in the case of Rob, a tuning app wouldn’t apply for him because, you know, somebody else tunes his piano for him—but he really loves Amazing Slower Downer. He uses the iReal Pro probably less than I do, because he could collaborate with himself in his left hand while he’s playing solo lines in his right hand. And he doesn’t use it as a tool like Smart Music because, you know, pianists are not usually playing along with other pianists, they’re the collaborators. And we’ve talked about that, too. He knows it’s not a tool that can replace a pianist. But when we work together, Rob is a consummate musician and so he learns things way faster than I do. His mother would feel the TV to see if he was practicing five hours a day when he was younger, where I was probably you know, playing out in the backyard with dirt or something. He learns things so much faster and he doesn’t want to spend time rehearsing. So for me I’ll prepare on Smart Music and I’ll prepare on these other apps and get to that point where now I don’t have to spend so much time rehearsing with him. We’ll just do it once or twice like professional musicians do and he’s always been that way. For me, it’s upped my ante and I’ve really enjoyed that. So we’ll talk about all kinds of technologies, he’s huge into it. Other colleagues, for vocal people, they prefer having a pianist collaborate with them—which makes total sense—so it’s not as prevalent and I understand that. But most of them are not against, they just don’t understand how to tie things into doing it because what they’re doing already is fine. And that’s one of my philosophies too: If something is working, and it’s not going to be any better by adding the technology, there’s no need to add it. But if it can be an efficiency, the fact that usually in history—and as I got older, I realized this—that convenience wins out. And if I look at audio recording, you know, tapes. Small tapes were horrible audio quality compared to vinyl and CD had finally came up to a very good audio quality, probably not as good as vinyl, but most people can’t hear that spectrum anyways for those people that believe it.

John: And usually it’s better the first couple times you play the vinyl. After, the needles have worn down the grooves, unless you’re reading it with a laser, the quality of vinyl degrades really quickly.

Trevor: It does. And then MP3’s everybody thought, “Oh that won’t stick,” you know, everybody kept to analog, and really, the quality doesn’t sound nearly as good. But most people are listening through cheap headsets and on phones and using it for different background thing. Most people are not audiophiles and most musicians are and they think that technology is not going to last. So unless there’s a technology that other than convenience as a musician that I think benefits my students, I don’t really buy into it too much. A lot of people have embraced it and for different reasons. I think everybody owns a tuner on their phone. Everybody has a metronome on their phone. My colleague Juan La Manna uses the digital recordings and visual recordings of Zooms and stuff like that for his conducting students to record them and give them feedback on it and he uses it to record himself on concerts and a lot of technology being used, but not the Smart Music so much outside of wind players, I think. And it’s also ingrained in the band world. Jazz musicians have always used technology as far as like play along recordings with Jamey Aebersold and later on so it’s only a natural progression to use something like iReal Pro, I think. But you know, most of my faculty enjoy technology when it’s beneficial if it doesn’t help them out, there’s no need to learn it.

Rebecca: So I always wrap up our podcast by asking, what’s next?

Trevor: One of the things I’ve held off on and my colleague Rob Auler has not is reading off of tablets for music instead of bringing paper along with them. So for him it made a lot more sense, because he can use the AirTurn Duo Bluetooth pedal, and he doesn’t have to rely on anyone to turn his page. It’ll take them a little time to practice it. You can actually write and notate things in there if you’re in the middle of rehearsal, you know, we want more piano here, or we listen to the cello or whatever. But for me as a single line instrument, it didn’t make that much more sense other than the convenience of it, which we’ve talked about, which is to not have a lot of music in your backpack. But then a few things have changed over the course of the last few weeks that made me think I need to do it for my students and myself. They should know this technology. So I went to a rock gig—I hadn’t played a rock gig for 20 years at a wedding one of my friends asked me to do—The bread, the money was pretty good so I decided to do it for the first time in a while. And most of the people were playing on that, I had this like four inch binder that the guy gave me. He wouldn’t give me his iPad, and I played there and then I did a soloist with the Syracuse Symphoria and I was there just watching the orchestra play in one of the pieces and the harpist… and harpists are one of the people that are not known for…you know, they play an antiquated instrument…. It’s a gorgeous instrument… But she was using an iPad. Ironically she was reading a hard copy book while she was waiting for her turn to play in different pieces [LAUGHTER] so I thought that was ironic but she was using an iPad and I thought, “Oh, maybe I should do this.” And also the advent of the iPad Pro, where it’s large enough. You know I tried it when Vornhagen came here when I was doing some pieces, and I wrote it and sketched it and put it on the iPad, it just wasn’t conducive to reading with low light. And then Rob mentioned something to me that I thought would be the ultimate reason why to get one. He goes, “Wouldn’t it be nice if everybody played off the score?” By that I mean piano score that has a cello and the clarinet part there and we could all take a look to see vertically what’s going on as opposed to what we normally play off, the cellist has a single line as do I. Obviously the problem behind that would…we have to like learn how to read music differently. Instead of going from one line to the other, I’d have to skip them things, but I’m a conductor so that’s not a problem. But it would actually save a lot of time in rehearsal if we got off, or more importantly, if we want to phrase something together or we’re talking about something like, “What happens there?” I’d have to go over and look over Rob’s shoulder. So that to me, outside of the convenience of it, and that I probably should get back into the 21st century and actually get playing off them so our students know how to do it. I think it’s something that my next step is going to be starting to do that. And yet again, you can edit it. You can draw on it. You can put in repeats. It just seems so much more logical to do. I put in for grant for this and at the end, they’re evaluating it and they asked me specifically, “What would you do if you didn’t get this?” and I said, “Probably use paper” other than than what Rob just mentioned to me right after that session where it’d be great to read off the score ourselves. The advantage of it is more mobility and portability. These fake books are two and a half to three inches thick and there’s three or four of them and he has them all on his iPad and I have to slug them around with me, so I have nothing against paper.

John: But it’s a whole lot easier carrying around an iPad where you can have tens of thousands of scores with you.

Trevor: It is. And a battery pack too. That’s the other thing, if you don’t have a way to charge it and stuff like that, but that nowadays that doesn’t seem like it’s a concern at all.

John: And iPads generally got 10 or 12 hours at least of battery life.

Trevor: Right.

Rebecca: Well, thank you so much for joining us, Trevor. It’s been really interesting to hear you talk about your practice with mobile technology. And I know for me as a visual artist, it was making me think about the kinds of tools that we use as a visual artist and kind of translating the kinds of activities that you’re thinking about with students to different kinds of activities that I could do with my students and also thinking about how to use those to kind of push students just a little bit further.

Trevor: Nice. That’s great.

John: Thank you.

Trevor: Thank you, appreciate it.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandla, and Jacob Alverson.

37. Evidence is Trending

Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode, Michelle Miller joins us to discuss recent research that will likely shape the future of higher education.

Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.

Show Notes

Rebecca: Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode we talk to a cognitive psychologist about recent research that will likely shape the future of higher education.
[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Michelle Miller. Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.
Welcome, Michelle!

Michelle: Hi, I’m so glad to be here.

Rebecca: Thanks for joining us.
Today’s teas are:

Michelle: I’m drinking a fresh peppermint infused tea, and it’s my favorite afternoon pick-me-up.

Rebecca: …and it looks like it’s in a really wonderfully designed teapot.

Michelle: Well, thank you… and this is a thrift store find… one of my favorite things to do. Yeah, so I’m enjoying it.

John: I have Twinings Blackcurrant Breeze.

Rebecca: …and I’m drinking chai today.

Michelle: Pretty rough.

John: We invited you here to talk a little bit about things that you’ve been observing in terms of what’s catching on in higher education in terms of new and interesting innovations in teaching.

Michelle: Right, that’s one of things that I really had the luxury of being able to step back and look at over this last semester and over this last spring when I was on sabbatical… One of the really neat things about my book Minds Online, especially now that it’s been out for a few years, is that it does open up all these opportunities to speak with really engaged faculty and others, such as: instructional designers, librarians, academic leadership, educational technology coordinators… all these individuals around the country who are really, really involved in these issues. It’s a great opportunity to see how these trends, how these ideas, how these innovations are rolling out, and these can be some things that have been around for quite some time and just continue to rock along and even pickup steam, and some newer things that are on the horizon.

John: You’ve been doing quite a bit of traveling. You just got back from China recently, I believe.

Michelle: I sure did. It was a short visit and I do hope to go back, both to keep getting involved in educational innovations there and, hopefully, as a tourist as well. So, I was not there for very long but I had the opportunity to speak at Tsinghua University in Beijing, which is a really dynamic institution that’s been around for about a hundred years. For a while in its history it specialized in things like engineering education polytechnic, but now it’s really a selective comprehensive university with very vibrant graduate and undergraduate programs that are really very relatable for those of us in the United States working in similar contexts. My invitation was to be one of the featured speakers at the Future Education, Future Learning Conference, which was a very interdisciplinary gathering of doctoral students, faculty, even others from the community, who were all interested in the intersection of things like technology, online learning, MOOCs even, and educational research (including research into the brain and cognitive psychology), and bringing all of those together… and it was a multilingual conference. I do not speak Chinese but much of the conference was in both English and Chinese and so I was also able to really absorb a lot of these new ideas. So yes, that was a real highlight of my sabbatical semester and one that I’m going to be thinking about for quite some time.

I should say that part of what tied in there as well is that Minds Online, I’ve just learned, is going to be translated into Chinese and that’s going to come out in May 2019. So, I also got to meet with some of the people who were involved in the translation… start to put together some promotional materials such as videos and things like that.

Rebecca: Cool.

John: Excellent.

Rebecca: So, you’ve had a good opportunity, as you’ve been traveling, to almost do a scavenger hunt of what faculty are doing with evidence-based practices related to your book. Can you share some of what you’ve found or heard?

Michelle: This theme of evidence-based practice, and really tying into the findings that have been coming out of cognitive psychology for quite some time, that really is one of the exciting trends and things that I was really excited to see and hear for so many different quarters I visited in different institutions… and so I would say definitely, this is a trend that is continuing and is increasing. There really does continue to be a lot of wonderful interest and wonderful activity around these real cognitively informed approaches to teaching, and what I think we could call scientifically based and evidence-based strategies. One form this has taken is Josh Eyler’s new book, called How Humans Learn: The Science and Stories behind Effective College Teaching. This is a brand new book by a faculty development professional, and a person coming out of the humanities, actually, who’s weaving together even from his humanities background everything from evolutionary biology to classical research in early childhood education to the latest brain-based research. He’s weaving this together into this new book for faculty. So, that’s one of the things that I’ve noticed and then there’s the issue which i think is another great illustration of best-known practice which is the testing effect and retrieval practice.

John: One of the nice things is how so many branches of research are converging… testing in the classroom, brain-based research, and so forth, are all finding those same basic effects. It’s nice to see such robust results, which we don’t always see in all research in all disciplines.

Rebecca: …and just breaking down the silos in general. The things are all related and finding out what those relationships are… exploring those relationships… is really important and it’s nice to see that it’s starting to open up.

John: We should also note that when you visited here, we had a reading group and we had faculty working on trying to apply some of these concepts, and they’re still doing that… and they still keep making references back to your visit. So, it’s had quite a big impact on our campus.

Michelle: This wasn’t true, I don’t think, when I first entered the teaching profession… and even to the extent when I first started getting interested in applied work in course redesign and in faculty professional development. you would get kind of this pushback or just strange looks when you said “Oh, how about we bring in something from cognitive psychology” and now that is just highly normalized and something that people are really speaking across the curriculum… and taking it and running with it in a lasting ongoing way, not just as a “Oh, well that was an interesting idea. I’m going to keep doing what I’m doing” but really people making some deep changes as you mentioned. This theme of breaking down silos… I mean I think if there’s kind of one umbrella trend that all of these things fits under it’s that breakdown of boundaries. So, that’s one that I keep coming back to, I know, in my work.

So, the idea of retrieval practice, drilling down on that one key finding which goes back a very long ways in cognitive psychology. I think of that as such a good example of what we’re talking about here… about how this very detailed effect in cognition and yet it does have these applications across disciplinary silos. Now when I go to conferences and I say “Okay, raise your hand. How many people have ever heard of retrieval practice? How many people have ever heard of the testing effect? How many people have heard of the book Make it Stick (which really places this phenomena at its center)?” and I’m seeing more hands raising.

With retrieval practice, by the way, we’re talking about that principle that taking a test on something, that retrieving something from memory actively, has this huge impact on future memorability of that information. As its proponents like to say, tests are not neutral from a memory or from a learning standpoint… and while some of the research has focused on very kind of stripped-down laboratory style tasks like memorizing words pairs, there are also some other research projects showing that it does flow out to more realistic learning situations.

So, more people simply know about this, and that’s really the first hurdle, oftentimes, with getting this involved disciplinary sometimes jargon riddled research out there to practitioners and getting it into their hands. So, people heard of it and they’re starting to build this into their teaching. As I’ve traveled around I love to hear some of the specific examples and to see it as well crop up in scholarship of teaching and learning.

Just recently, for example, I ran across and really got into the work of Bruce Kirchhoff who is at University of North Carolina – Greensboro and his area is botany and plant identification. He has actually put together some different really technology-based apps and tools that students and teachers can use in something like a botany course to rehearse and review plant identification. He says in one of his articles, for example, that there just isn’t time in class to really adequately master plant identification. It’s just too complex of a perceptual and cognitive and memory test to do that. So, he really built in from the get-go very specific principles drawn from cognitive psychology… so, the testing effect is in there… there’s different varieties of quizzing and it all is about just getting students to retrieve and identify example after example. It brings in also principles such as interleaving, which we could return to in a little bit, but has to do with the sequencing of different examples… their spacing… So, that’s even planned out exactly how and when students encounter different things that they’re studying. It’s really wonderful. So, for example he and his colleagues put out a scholarship of teaching and learning article talking about how this approach was used effectively in veterinary medicine students who have to learn to identify poisonous plants that they’ll see around their practice. This is something that can be time-consuming and very tough, but they have some good data showing that this technology enhanced cognitively based approach really does work. That’s one example. Coincidentally, I’ve seen some other work in the literature, also on plant identification, where the instructors tagged plants in an arboretum… they went around and tagged them with QR codes… that students can walk up to a plant in the real environment with an iPad… hold the iPad over it… and it would immediately start producing quiz questions that were are specific to exactly the plants they were looking at.
So, those are some of the exciting things that people are taking and running with now that this principle is out there.

Rebecca: What I really love about the two stories that you just shared was the faculty are really designing their curriculum and designing the learning experiences with the students in mind… and what students need and when they need it. So, not only is it employing these cognitive science principles, but it’s actually applying design principles as well. It’s really designing for a user experience and thinking about the idea that if I need to identify a plant, being able to identify it in this situation in which I would need to identify it in makes it much more dynamic I think for a student… but also really meets them where they’re at and where they need it.

John: …and there’s so many apps out there now that will do the plant identification just from imagery without the QR code, that I can see it taking it one step further where they can do it in the wild without having that… so they can build it in for plants that are in the region without needing to encode that specifically for the application.

Michelle: I think you’re absolutely right once we put the technology in the hands of faculties who, as I said, they’re the one to know: “Where are my students at? Where are the weak points? Where are the gaps that they really need to bridge?” and that’s where their creativity is giving rise to all these new applications… and sometimes these can be low-tech as well… or also things that we can put in a face-to-face environment… and I’d like to to share just some experiences that I’ve had with this over the last few semesters.

In addition to trying to teach online with a lot of technology, I also have in my teaching rotation a small required course in research methods in psychology which can be a real stumbling block… the big challenge course… it’s kind of a gateway course to continued progress in our major. So, in this research methods course, some of the things that I’ve done around assessment and testing to really try again to stretch that retrieval practice idea… to make assessments really a more dynamic part of the course and more central part of the course… to move away from that idea that tests are just this kind of every now and again this panic mode opportunity for me to kind of measure in sorts of students and judge them… to make good on that idea that tests are part of learning. So, here’s some of the things that I try to do. For one thing, I took time out of the class almost every single class meeting as part of the routine to have students first of all generate quiz questions out of their textbook. So, we do have a certain amount of foundational material in that course as well as a project and a whole lot of other stuff is going on. So they need to get that foundational stuff.

Every Tuesday they would come in and they knew their routine: you get index cards and you crack your textbook and you generate for me three quiz questions. Everybody does it. I’m not policing whether you read the chapter or not. It’s active… they’re generating it… and also that makes it something like frequent quizzing. That’s a great practical advantage for me since I’m not writing everything. They would turn those in and I would select some of my favorites I would turn those into a traditional looking paper quiz and hand that out on Thursday. I said “Hey, take this like a realistic quiz.” I had explained to them that quizzes can really boost their learning, so that was the justification for spending time on it and then I said: “You know what? I’m not going to grade it either. You take it home because this is a learning experience for you. It’s a learning activity.” so we did that every single week as those students got into that routine.

The second thing that I did to really re-envision how assessment testing and quizzing worked in this particular course, was something inspired by different kinds of group testing and exam wrapper activities I’ve seen, particularly coming out of the STEM field, where there’s been a lot of innovation in this area. What I would do is… we had these high stakes exams at a few points during the semester. But, the class day after the exam, we didn’t do the traditional “Let’s go over the exam.” [LAUGHTER] That’s kind of deadly dull, and it just tends to generate a lot of pushback from students… and as we know from the research, simply reviewing… passing your eyes over the information… is not going to do much to advance your learning. So, what I would do is… I would photocopy all those exams, so it has a secure copy. They were not graded. I would not look at this before we did this… and I would pass everybody’s exams back to them along with a blank copy of that same exam. I assigned them to small groups and I said “Okay, here’s your job. Go back over this exam, fill it out as perfectly as you can as a group, and to make it interesting I said I will grade that exam as well, the one you do with your group, and anything you get over 90% gets added to everybody’s grade. This time it was open book, it was open Google, it was everything except you can’t ask me questions. So, you have each other and that’s where these great conversations started to happen. The things that we always want students to say. So, I would eavesdrop and hear students say “Oh, well you know what, I think on this question she was really talking about validity because reliability is this other thing…” and they’d have a deep conversation about it. I’m still kind of going back through the numbers to see what are the impacts of learning? Are there any trends that I can identify? But, I will say this: in the semesters that I did this, I didn’t have a single question ever come back to me along the lines of “Well, this question was unclear. I didn’t understand it. I think I was graded unfairly.” it really did shut all that down and again extended the learning that I feel students got out of that. Now it meant a big sacrifice of class time, but I feel strongly enough about these principles that I’m always going to do this in one form or another anytime I can can in face-to-face classes.

Rebecca: This sounds really familiar, John.

John: I’ve just done the same, or something remarkably similar, this semester, in my econometrics class which is very similar to the psych research methods class. I actually picked it up following a discussion with Doug McKee. He actually was doing it this semester too. He had a podcast episode on it. It sounded so exciting, I did something… a little bit different. I actually graded it but I didn’t give it back to them because I wanted to see what they had the most trouble with, and then I was going to have them only answer the ones in a group that they struggled with… and it turned out that that was pretty much all them anyway. So, it’s very similar to what you did except I gave them a weighted average of their original grade and the group grade and all except one person improved and the one person’s score went down by two points because the group grade was just slightly lower… but he did extremely well and he wasn’t that confident. The benefits to them of that peer explanation and explaining was just tremendous and it was so much more fun for them and for me and, as you said, it just completely wiped out all those things like “Well, that was tricky” because when they hear their peers explaining it to them the students were much more likely to respond by saying “Oh yeah, I remember that now” and it was a wonderful experience and I’m gonna do that everywhere I can.

In fact. I was talking about it with my TA just this morning here at Duke and we’re planning to do something like that in our classes here at TIP this summer, which i think is somewhat familiar to you from earlier in your academic career.

Michelle: That is right we do have this connection. I was among, not the very first year, but I believe the second cohort of Talent Identification Program students who came in, I guess you would call it now, middle school (back then, it was called junior high) and what a life-transforming experience. We’ve had even more opportunities to talk about the development of all these educational ideas through that experience.

John: That two-stage exam is wonderful and it’s so much more positive… because it didn’t really take, in my class, much more time, because I would have spent most of that class period going over the exam and problems they had. But the students who did well would have been bored and not paying much attention to it; the students who did poorly would just be depressed and upset that they did so poorly… and here, they were actively processing the information and it was so positive.

Michelle: That’s a big shift. We really have to step back and acknowledge that, I think. that is a huge shift in how we look at assessment, and how we think about the use of class time… and it’s not just “Oh my gosh, I have to use every minute to put such content in front of the students.” Just the fact that more of us are making that leap, I think, really is evidence this progress is happening… and we see also a lot of raised consciousness around issues such as learning styles. That’s another one that, when I go out and speak to faculty audiences, 10 years ago you would get these shocked looks or even very indignant commentary when you say “Ok, this idea of learning styles, in the sense that say there are visual learners, auditory learners, what I call sensory learning styles (VAK is another name it sometimes goes by). The idea that that just holds no water from a cognitive point of view…” People were not good with that, and now when I mentioned that at a conference, I get the knowing nods and even a few groans… people like “Oh, yeah. we get that. Now, K-12, which I want to acknowledge it’s not my area, but I’m constantly reminded by people across the spectrum that it’s a very different story in K-12. So, setting that aside… but this is what I’m seeing… that faculty are realizing… they’re saying “Oh, this is what the evidence says…” and maybe they even take the time to look at some of the really great thinkers and writers who put together the facts on this. They say “You know what? I’m not going to take my limited time and resources and spend that on this matching to styles when the styles can’t even be accurately diagnosed and are of no use in a learning situation. So, that’s another area of real progress.

Rebecca: What I am hearing is not just progress here in terms of cognitive science, but a real shift towards really thinking about how students learn and designing for that rather than something that would sound more like a penalty for grade like “Oh, did you achieve? Yes or no…” but, rather here’s an opportunity if you didn’t achieve to now actually learn it… and recognize that you haven’t learned it, even though it might seem really familiar.

John: Going back to that point about learning styles. It is spreading in colleges. I wish it was true at all the departments at our institution, but it’s getting there gradually… and whenever people bring it up, we generally remind them that there’s a whole body of research on this and I’ll send them references but what’s really troubling is in my classes the last couple years now, I’ve been using this metacognitive cafe discussion forum to focus on student learning… and one of the week’s discussions is on learning styles and generally about 95 percent of the students who are freshmen or sophomores (typically) come in with a strong belief in learning styles… where they’ve been tested multiple times in elementary or middle school… they’ve been told what their learning styles are… they’ve been told they can only learn that way… It discourages them from trying to learn in other ways and it does a lot of damage… and I hope we eventually reach out further so that it just goes away throughout the educational system.

Rebecca: You’ve worked in your classes, Michelle, haven’t you to help students understand the science of learning and use that to help students understand the methods and things that you’re doing>

Michelle: Yes, I have. I’ve done this in a couple of different ways. Now, partly, I get a little bit of a free pass in some of my teaching because I’m teaching the introduction to psychology or I’m teaching research methods where I just happen to sneak in as the research example will be some work on say attention or distraction or the testing effect. So, I get to do it in those ways covertly. I’ve also had the chance, although it’s not on my current teaching rotation… I’ve had the chance to also take it on as in freestanding courses. As many institutions are doing these days… it’s another trend… and what Northern Arizona University, where I work, has different kinds of freshmen or first-year student offering for courses they can take, not in a specific disciplinary area, but that really crossed some different areas of the student success or even wellbeing. So, I taught a class for awhile called Maximizing Brain Power that was about a lot of these different topics. Not just the kind of very generic study skills tip… “get a good night’s sleep…” that kind of thing… but really some again more evidence-based things that we can tell students and you can really kind of market it… and I think that we do sometimes have to play marketers to say “Hey, I’m going to give you some inside information here. This is sort of gonna be your secret weapon. So, let me tell you what the research has found.”

So, those are some of the things that I share with students… as well as when the right moment arises, say after an exam or before their first round of small stakes assessments, where they’re taking a lot of quizzes… to really explain the difference between this and high stakes or standardized tests they may have taken in the past. So, I do it on a continuing basis. I try to weave it into the disciplinary aspect and I do it in these free-standing ways as well… and I think here’s another area where I’m seeing this take hold in some different places… which is to have these free-standing resources that also just live outside of a traditional class that people can even incorporate into their courses… if say cognitive psychology or learning science isn’t their area… that they can bring in, because faculty really do care about these things. We just don’t always have the means to bring them in in as many ways as we would like.

John: …and your Attention Matters project was an example of that wasn’t it? Could you tell us a little bit about that?

Michelle: Oh, I’d love to… and you know this connects to what it seems to be kind of an evergreen topic in the teaching and learning community these days, which is the role of distracted students… and I know this past year there just have been these one op-ed versus another. There’s been some really good blog posts by some people I really like to follow in the teaching and learning community such as Kevin Gannon talking about “Okay, do you have laptops in the classroom? and what happens when you do?” and so I don’t think that this is just a fad that’s going away. This is something that the people do continue to care about, and this is where the attention matters project comes in.

This was something that we conceptualized and put together a couple years ago at Northern Arizona University with myself, and primarily I collaborated with a wonderful instructional designer who also teaches a great deal… John Doherty. So, how this came about is I was seeing all the information on distraction… I’m really getting into this as a cognitive psychologist and going “Wow, students need to know that if they’re texting five friends and watching a video in their class. It’s not going to happen for them.” I was really concerned about “What can I actually do to change students minds?” So, my way of doing this was to go around giving guests presentations in every classes where people would let me burn an hour of their class time… and not a very scalable model… and John Doherty respectfully sat through one my presentations on this and then he approached me and said “Look, you know, we could make a module and put this online… and it could be an open access within the institution module, so that anybody at my school can just click in and they’re signed up. We could put this together. We could use some really great instructional design principles and we could just see what happens… and I bet more people would take that if it were done in that format. We did this with no resources. We just were passionate about the project and that’s what we did. We had no grant backing or anything. We got behind it. So, what this is is about a one- to two-hour module that, it’s a lot like a MOOC in that it there’s not a whole lot of interaction or feedback, but there are discussion forums and it’s very self-paced in that way… so one- to two-hour mini MOOCs that really puts at the forefront demonstrations and activities… so we don’t try to convince students about problems with distraction and multitasking… we don’t try to address that just by laying a bunch of research articles on them… I think that’s great if this were a psychology course, but it’s not. So, we come at it by linking them out to videos, for example, that we were able to choose, that we feel really demonstrate in some memorable ways what gets by us when we aren’t paying attention… and we also give students some research-based tips on how to set a behavioral plan and stick to it… because just like with so many areas of life, just knowing that something is bad for you is not enough to really change your behavior and get you not to do that thing. so we have students talking about their own plans and what they do when, say, they’re having a boring moment in class, or they’re really really tempted to go online while they’re doing homework at home. What kinds of resolutions can they set or what kind of conditions can make that that will help them accomplish that. Things like the software blockers… you set a timer on your computer and it can lock you out of problematic sites… or we learned about a great app called Pocket Points where you actually earn spendable coupon points for keeping your phone off during certain hours. This is students talking to students about things that really concern them and really concern us all because I think a lot of us struggle with that.

So, we try to do that… and the bigger frame for this as well is this is, I feel, a life skill for the 21st century… thinking about how technology is going to be an asset to you and not detract from what you accomplish in your life. What a great time to be reflecting on that, when you’re in this early college career. so that’s what we try to do with the project…and we’ve had over a thousand students come through. They oftentimes earn extra credit. Our faculty are great about offering small amounts of extra credit for completing this and we’re just starting to roll out some research showing some of the impacts… and showing it in a bigger way just how you can go about setting up something like this.

Rebecca: I like that the focus seems to be on helping students with a life skill rather than using technology is just a blame or an excuse. We’re in control of our own behaviors and taking ownership over our behaviors is important rather than just kind of object blaming.

Michelle: So, looking at future trends, I would like to see more faculty looking at it in the way that you just described, Rebecca, as this is a life skill and it’s something that we collaborate on with our students… not lay down the law… because, after all, students are in online environments where we’re not there policing that and they do need to go out into work environments and further study and things like that. So, that’s what I feel is the best value. For faculty who are looking at this, if they don’t want to do… or don’t have the means to do something really formal like our Attention Matters approach, just thinking about it ahead of time… I think nobody can afford to ignore this issue anymore and whether you go the route of “No tech in my classroom” or “We’re going to use the technology in my classroom“ or something in between… just reading over, in a very mindful way, not just the opinion pieces, but hopefully also a bit of the research, I think, can help faculty as they go in to deal with this… and really to look at it in another way, just to be honest, we also have to consider how much of this is driven by our egos as teachers and how much of it is driven by a real concern for student learning and those student life skills. I think that’s where we can really take this on effectively and make some progress when we are de-emphasizing that ego aspect and making sure that it really is about the students.

John: We should note there’s a really nice chapter in this book called Minds Online: Teaching Effectively with Technology that deals with these types of issues. It was one of the chapters that got our faculty particularly interested in these issues… on to what extent technology should be used in the classroom… and to what extent it serves as a distraction.

Michelle: I think that really speaks to another thing which I think is an enduring trend… which is the emphasis on really supporting the whole student in success and what we’ve come to call academic persistence… kind of a big umbrella term that has to do with, not just succeeding in a given class, but also being retained… coming back after the first year. As many leaders in higher education point out, this is as a financial issue. As someone pointed out, it does cost a lot less to hang on to the students you have instead of recruiting more students to replace ones who are lost. This is, of course, yet another really big shift in mindset of our own, because after all we did used to measure our success by “Hey, I flunked this many students out of this course” or” Look at how many people have to switch into different majors…our major is so challenging…”

So, we really have turned that thinking around and this does include faculty now. I think that we did used to see those silos. We had that very narrow view of “I’m here to convey content. I’m here to be an expert in this discipline, and that’s what I’m gonna do…” and sure, we want to think about things like do students have learning skills? Do they have metacognition? Are they happy and socially connected at the school? Are they likely to be retained so that we can have this robust university environment?

We had people for that, right? It used to be somebody else’s job… student services or upper administration. They were the ones who heard about that and now I think on both sides we really are changing our vision. More and more forward-thinking faculty are saying “You know what? Besides being a disciplinary expert, I want to become at least conversant with learning science. I want to become at least conversant with the science of academic persistence…” There is a robust early literature on this and that’s something that we’ve been working on at NAU over this past year as well… kind of an exciting newer project that I like very much. We’ve started to engage faculty in a new faculty development program called Persistence Scholars and this is there to really speak to people’s academic and evidence-based side, as well as get them to engage in some perspective-taking around things like the challenges that students face and what it is like to be a student at our institution. We do some really selected readings in the area we look at things like mindset… belongingness… these are really hot areas in that science of persistence… in that emerging field. But, we have to look at it in a really integrated way.

It’s easy for people to say just go to a workshop on mindset and that’s a nice concept, but we wanted to think about it in this bigger picture… really know what are some of the strengths of that and why? Where do these concepts come from? What’s the evidence? That’s something that I think is another real trend and I think as well we will see more academic leaders and people in staff and support roles all over universities needing to know more about learning science. There are still some misconceptions that persist, as we’ve talked about. We’re making progress in getting rid of some of these myths around learning, but I will say… I’m not gonna name any names… but, every now and again I will hear from somebody who says “Oh well, we need to match student learning styles” or “Digital natives think differently, don’t you know?” and I have to wonder whether that’s a great thing. I mean, these are oftentimes individuals that have the power to set the agenda for learning all over a campus. Faculty need to be in the retention arena and I think that leaders need to be in the learning science arena. The boundaries is breaking down and it’s about time.

Rebecca: One of the things that I thought was really exciting with the reading groups that we’ve been having on our campus… that we started with your book, but then we’ve read Make it Stick and Small Teaching since… is that a lot of administrators in a lot of different kinds of roles engaged with us in those reading groups, it wasn’t just faculty. There was a mix of faculty, staff, and some administrators, and I think that that was really exciting. For people who don’t have the luxury of being in your persistence scholar program, what would you recommend they read to get started to learn more about the science of persistence?

Michelle: I really, even after working with this for quite some time, I loved the core text that we have in that program, which is Completing College by Vincent Tinto. It’s just got a great combination of passionate and very direct writing style. So, there’s no ambiguity, there’s not a whole lot of “on the one hand this and on the other hand that.” It’s got an absolutely stellar research base, which faculty of course appreciate… and it has a great deal of concrete examples. So, in that book they talk about “okay, what does it mean to give really good support to first semester college students? What does that look like?” and they’ll go out and they’ll cite very specific “Here’s a school and here’s what they’re doing… here’s what their program looks like… here’s another example that looks very different but gets at the same thing.” So, that’s one of the things that really speak to our faculty… that they really appreciated and enjoyed.

I think that as well we tested good feedback about work that’s come out of the David Yeager and his research group on belongingness and lay theories, and lay theories is maybe a counterintuitive term for kind of a body of ideas about what students believe about academic success and why some people are successful and others are not and how those beliefs can be changed sometimes through relatively simple interventions and when it happens we see great effects such as the narrowing of achievement gaps among students who have more privilege or less privileged backgrounds… and that’s something that, philosophically, many faculty really really care about but they’ve never had the chance to really learn “Okay, how can I actually address something like that with what I’m doing in my classroom, and how can I really know that the things that I’m choosing do have that great evidence base…”

John: …and I think that whole issue is more important now and is very much a social justice issue because, with the rate of increase we’ve seen in college cost inflation, people who start college and don’t finish it are saddled with an awfully high burden of debt. The rate of return to a college degree is the highest that we’ve ever seen and college graduates end up not only getting paid a lot more but they end up with more comfortable jobs and so forth… and if we really want to move people out of poverty and try to reduce income inequality, getting more people into higher education and successfully completing higher education is a really important issue. I’m glad to see that your institution is doing this so heavily and I know a lot of SUNY schools have been hiring Student Success specialists. At our institution they’ve been very actively involved in the reading group, so that message is spreading and I think some of them started with your book and then moved to each of the others. So, they are working with students in trying to help the students who are struggling the most with evidence-based practices …and I think that’s becoming more and more common and it’s a wonderful thing.

Rebecca: So, I really liked Michelle that you were talking about faculty getting involved in retention and this idea of helping students develop persistence skills, and also administrators learning more about evidence-based practices. There’s these grassroots movements happening in both of these areas. Can you talk about some of the other grassroots movements that are working toward, or efforts that faculty are making to engage students and capture their attention and their excitement for education?

Michelle: Right, and here I think a neat thing to think about too is just it’s the big ambitious projects… the big textbook replacement projects or the artificial intelligence informed adaptive learning systems… those are the things that get a lot of the press and end up in The Chronicle of Higher Education that we read about… But, outside of that, there is this very vibrant community and grassroots led scene of developing different technologies and approaches. So, it really goes back for a while. I mean, the MERLOT database that I do talk about in Minds Online has been trove for years of well hidden gems that take on one thing in a discipline and come at it from a way that’s not just great from a subject-matter perspective but brings up the new creative approaches. In the MERLOT database, for example, there’s a great tutorial on statistical significance and the interrelationship between statistical significance and issues like simple sizes. You know, that’s a tough one for students, but it has a little animation involving a horse and a rider that really turns it into something that’s very visual… that’s very tangible… and it really actually tying into analogies, which is a well-known cognitive process that can support the advancement of learning something new. There is something on fluid pressures in the body that was treated for nursing students by nurses, and it’s got an analogy of a soaker hose that this is really fun and is actually interactive. So, those are the kinds of things. The PhET project, P-h-E-T which comes out of University of Colorado, that has been around for a while… again, faculty-led and a way to have these very useful interactive simulations for concepts in physics and chemistry. So, that’s one. CogLab, that’s an auxiliary product that I’ve used for some time in like hundred psychology courses that simulates very famous experimental paradigms which are notoriously difficult to describe on stage for cognitive psychology students. That started out many years ago as a project that very much has this flavor of “We have this need in our classroom. We need something interactive. There’s nothing out there. Let’s see what we can build.” It has since then picked up and turned into a commercial product, but that’s the type of thing that I’m seeing out there.

Another thing that you’ll definitely hear about if you’re circulating and hearing about the latest project is virtual reality for education. So, with this it seems like, unlike just a few years ago, almost everywhere you visit you’re going to hear that “Oh, we’ve just set up a facility. We’re trying out some new things.” This is something that I also heard about when I was talking to people when I was over in China. So, this is an international phenomenon. It’s going to pick up steam and definitely go some places.

What also strikes me about that is just how many different projects there are. Just when you’re worried that you’re going to be scooped because somebody else is going to get there first with their virtual reality project you realize you’re doing very very different things. So, I’ve seen, for example, it used in a medical application to increase empathy among medical students… and I took a six or seven minute demonstration that just was really heart-rending, simulating the patient experience with a particular set of sensory disorders… and at Northern Arizona University we have a lab that is just going full-steam in coming up with educational applications such as interactive organic chemistry tutorial that is is just fascinating. We actually completed a pilot project and are planning to gear up a much larger study next semester looking at the impacts of this. So, this is really taking off for sure.

But, I think there are some caveats here. We still really need some basic research on this… not just what should we be setting up and what the impacts are but how does this even work? In particular, what I would like to research in the future, or at least see some research on, is what kinds of students… what sort of student profile… really gets the most out of virtual reality for education. Because amidst all the very breathless press that’s going on about this now and all the excitement, we do have to remember this is a very, very labor intensive type of resource to set up. You’re not just going to go home and throw something together for the next week. It takes a team to build these things and to complete them as well. If you have, say, a 300 student chemistry course (which is not atypical at all… these large courses), you’re not going to just have all of them spend hours and hours and hours doing this even with a fairly large facility. It’s a very hands-on thing to guide them through this process, to provide the tech support, and everything else.

So, I think really knowing how we can best target our efforts in this area, so that we can build the absolute best, with the resources we have, and maybe even target and ask the students who are most likely to benefit. I think those are some of the things that we just need to know about this. So, it’s exciting for somebody like me who’s in the research area. I see this as a wonderful open opportunity… but those are some of the real crossroads we’re at with virtual reality right now.

Rebecca: I can imagine there’s a big weighing that would have to happen in terms of expense and time and resources needed to startup versus what that might be saving in the long run. I can imagine if it’s a safety thing that you want to do a virtual reality experience, like saving people’s lives and making sure that they’re not going to be in danger as they practice particular skills, could be a really good investment in these… spending the resources to make that investment… or if it’s a lot of travel that would just be way too expensive to bring a bunch of students to a particular location… but you could virtually… it seems like it would be worth the start-up costs and those are just two ideas off the top of my head where it would make sense to bend all of that resource and time.

John: …and equipment will get cheaper. Right now, it’s really expensive for computers that have sufficient speed and graphics processing capability and the headsets are expensive, but they will come down in price, but as you said, it’s still one person typically and one device… so it doesn’t scale quite as well as a lot of other tools or at least not at this stage.

Rebecca: From what I remember, Michelle, you wrote a blog post about [a] virtual reality experience that you had. Can you share that experience, and maybe what stuck with you from that experience?

Michelle: Right, so I had the opportunity, just as I was getting to collaborate with our incredible team at the immersive virtual reality lab at NAU… one of the things I was treated to was about an hour and a half in the virtual reality setup that they have to explore some of the things that they had… Giovanni Castillo, by the way, is creative director of the lab and he’s the one who was so patient with me through all this. We tried a couple of different things and of course there’s such a huge variety of different things that you can do.
There’s a few things out there like driving simulators that are kind of educational… they’re kind of an entertainment… but he was just trying to give me, first of all, just a view of those… and I had to reject a few of them… I will say, initially, because I am one of the individuals who tends to be prone to motion sickness. So, that limits what I can personally do in VR and that is yet another thing that we’re gonna have to figure out. At least informally, what we hear is that women in particular tend to experience more of this. So, I needed, first of all, to go to a very low motion VR. I wasn’t gonna be whizzing through these environments. That was not going to happen for me. So, we did something that probably sounds incredibly simplistic, but it just touched me to my core… which is getting to play with Google Earth. You can spin the globe and either just pick a place at random or what Giovanni told me is… “You know, I’ve observed that when people do this, when we have an opportunity to interact with Google Earth, they all either go to where they grew up or they’ll go to someplace that they have visited recently or they plan to visit. So, I went to a place that is very special to me and maybe it doesn’t fit into either one of those categories neatly, but it’s my daughter’s University… her school… and I should say that this is also a different thing for me because my daughter goes to school in Frankfort, Germany… an institute that is connected to a Museum. So, I had only been to part of the physical facility… the museum itself… and it was a long time ago… and part of it was closer to the holiday. So, this is my opportunity to go there and explore what it looks like all over… and so, that was an emotional experience for me. It was a sensory experience… it was a social one… because we were talking the whole time… and he’s asking me questions and what kinds of exhibits do they have here… and what’s this part of it. So, that was wonderful. it really did give me a feel for alright, what is it actually like to be in this sort of environment?

I’m not a gamer. I don’t have that same background that many of our students have. So, it got me up to speed on that… and it did show me how just exploring something that is relatively simple can really acquire a whole new dimension in this kind of immersive environment. Now the postscript that I talked about in that blog post was what happened when I actually visited there earlier in the year. So, I had this very strange experience that human beings have never had before… which is from this… I don’t know whether to call it deja vu or what… of going to the settings and walking around the same environment and seeing the same lighting and all that sort of stuff that was there in that virtual reality environment… but this time, of course, with real human beings in it and the changes… the little subtle changes that take place over time, and so forth.

So, how does it translate into learning? What’s it going to do for our students? I just think that time is going to tell. It won’t take too long, but I think that these are things we need to know. But, sometimes just getting in and being able to explore something like this can really put you back in touch with the things you love about educational technology.

Rebecca: I think one of the things that I’m hearing in your voice is the excitement of experimenting and trying something… and that’s, I think, encouragement for faculty in general… is to just put yourself out there and try something out even if you don’t have something specific in mind with what you might do with it. Experiencing it might give you some insight later on. it might take some time to have an idea of what you might do with it, but having that experience, you understand it better… it could be really useful.

John: …and that’s something that could be experienced on a fairly low budget with just your smartphone and a pair of Google cardboard or something similar. Basically, it’s a seven to twelve dollar addition to your phone and you can have that experience… because there’s a lot of 3D videos and 3D images out there on Google Earth as well as on YouTube. So, you can experience other parts of the world and cultures before visiting… and I could see that being useful in quite a few disciplines.

Rebecca: So, we always wrap up with asking what are you going to do next?

Michelle: I continue to be really excited about getting the word out about cognitive principles and how we can flow those in to teaching face-to-face with technology… everything else in between. So, that’s what I continue to be excited about… leveraging cognitive principles with technology and with just rethinking our teaching techniques. I’m going to be speaking at the Magna Teaching with Technology Conference in October, and so I’m continuing to develop some of these themes… and I’m very excited to be able to do that. I’m right now also… we’re in the early stages of another really exciting project that has to do with what we will call neuromyth… So, that may be a term that you’ve turn across in some of your reading. It’s something that we touched on a few times, I think, in our conversation today… the misconceptions that people have about teaching and learning and how those can potentially impact the choices we make in our teaching. So, I’ve had the opportunity to collaborate with this amazing international group of researchers who’s headed up by Dr. Kristen Betts of Drexel University… and I won’t say too much more about it other than we have a very robust crop of survey responses that have come in from, not just instructors, but also instructional designers and administrators from around the world. So, we’re going to be breaking those survey results down and coming up with some results to roll out probably early in the academic year and we’ll be speaking about that at the Accelerate conference, most likely in November. That’s put out by the Online Learning Consortium. So, we’re right in the midst of that project and it’s going to be so interesting to see what has the progress been? What neuromyths are still out there and how can they be addressed by different professional development experiences. We’re continuing to work on the Persistence Scholars Program on academic persistence. So, we’ll be recruiting another cohort of willing faculty to take that on in the fall at Northern Arizona University. I am going to be continuing to collaborate and really work with and hear from John and his research group with respect to the metacognitive material that they’re flowing into foundational coursework and ways to get students up to speed with a lot of critical metacognitive knowledge. So, we’re going to work on that too… and I like to keep up my blog and work on shall we say longer writing project but we’ll have to stay tuned for that.

Rebecca: Sounds like you need to plan some sleep in there too.

[LAUGHTER]

John: Well, it’s wonderful talking to you, and you’ve given us a lot of great things to reflect on and to share with people.

Rebecca: Yeah. Thank you for being so generous with your time.

John: Thank you.

Michelle: Oh, thank you. Thanks so much. It’s a pleasure, an absolute pleasure. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

36. Peer instruction

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

33. The Marmots of Finance

In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, Alex Butler, a Professor of Finance at Rice University, joins us to discuss how rich imagery can be used to help students make connections and deepen their understanding.

Alex received the Rice University Presidential Mentoring Award and the George R. Brown Award for Superior Teaching in 2018. He is also the recipient of the Jones School’s Award for Scholarship Excellence in 2011 and 2012.

Show Notes

  • Duke Talent Identification Program (TIP)
  • Medina, J. (2011). Brain rules: 12 principles for surviving and thriving at work, home, and school. ReadHowYouWant.com.
  • Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
  • Alex’s web site at Rice

Transcript

Rebecca: In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, we’ll discuss how rich imagery can be used to help students make connections and deepen their understanding.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Alex Butler, a professor of finance at Rice University. Alex received the Jones School’s Award for Scholarship Excellence in 2011 and 2012. Alex teaches financial markets and corporate finance in the undergraduate MBA, MBA, and PhD programs. Sometime in the latter part of the last century, Alex and I spent three summers teaching introductory economics to highly gifted middle school and high school students at the Talent Identification Program at Duke University.
Welcome, Alex.

Alex: Oh, thank you. Thank you for having me, John.

Rebecca: Today’s teas are:

Alex: I am not drinking tea…. just water at this point.

Rebecca: Another one… epidemic.

John: My tea is ginger peach white tea.

Rebecca: I’ve Prince of Wales today.

John: Could you tell us a little bit about the courses that you normally teach?

Alex: Sure. I have, over the years, taught almost every course that we have… either here or at other schools. Right now what I teach (and I’m glad to finally have settled into a group of classes that are my classes year in and year out) and those classes are the undergraduate business finance class and PhD courses – one in corporate finance, and one that’s a topics class on causal inference. I really enjoy teaching the undergrad business finance class in particular because I was an undergraduate student here at Rice many, many years ago, and so it’s fun to be back and be on the other side of the podium teaching the students. One of the things that I really like about teaching the undergraduates here is that they are able to appreciate my dorky sense of humor.

Rebecca: …which is indeed a very important thing.

Alex: Agreed. For years, I tried to suppress my dorky sense of humor in the classroom… ad tried to teach the course sort of straight up and dry, and it made everybody miserable… the students…. me… and so, eventually over time, as I grew more and more confident teaching, I started incorporating more and more jokes here and there… and then I allowed them to become more elaborate, and then before you know it, I’ve developed a full-blown dad sense of humor and full-on dork mode.

John: That’s actually why we invited you here. We read a little bit about that in a teaching award you just received. What do students expect the course to be about when they take an introductory finance course?

Alex: At Rice, the students have a good grapevine of information about what courses are about and so, at this point, the students come in with pretty solid expectations that line up with what the class actually is… and whereas some students are hoping that I’m going to teach them how to become millionaires in the stock market, what the course mostly is about is that’s very hard to become a millionaire investing in the stock market and how to make decisions in a corporate setting that will maximize firm value.

Rebecca: So you mentioned that some students come in with this misperception of becoming a millionaire. What are some of the strategies that you use to dispel that myth?

Alex: I should say this class is one that is a fairly standard course that’s taught in lots of different business schools all over the country, all over the world. One way that I do it differently is in the order of material that I cover… and so the very first week of class, I talk about market efficiency. That’s the notion that it’s very difficult to earn abnormal returns in the stock market. In other words, you can’t beat the stock market very easily, unless it just happens to be by luck… and so I come in the first week talking about the reasons why it’s very difficult to beat the stock market… and the reason why is because there are literally tens of thousands of people who have more money and more resources and who are faster and are doing this as a full-time job who are also trying to find the stocks that are mispriced… and so unless you are the investor who is faster, and smarter, and has more money to throw at the trading strategy that you think of, it’s very very difficult to beat the other 10,000 people… and so I spend the first week of class introducing this concept and then providing copious amounts of evidence… research that highlights how difficult it is for lay people to earn abnormal returns in the stock market. That sets the setting for the rest of the course, which is this notion of how competition affects prices and how that feeds through to other applications and the corporate domain as opposed to the financial markets domain.

Rebecca: What made you switch the order?

Alex: What I realized after teaching this class for a long time is that most of corporate finance, most of business finance, is about discounting cash flows back to the present at some appropriate discount rate, and I found I was having a hard time getting the students to understand the notion of what interest rate, what discount rate, should be used to make these cash flows that are spread through time to get a present value equivalent. I figured if I started with some aspect of that, where that rate comes from, where those prices come from. but that would make the rest of the course easier for them to understand as we go through. So, that’s why. Most people wait until after they’ve introduced things like: “oh, portfolio theory” and “capital asset pricing model” and other asset pricing concepts before they talk about market efficiency… and so I just sort of turned that around backwards… and I open the course with that. So, point number one is pedagogical, and point number two is that the lectures that I do on market efficiency are really fun, and so I really like starting the course off with something that’s really fun. So, we can talk about stories of insider trading, and we can talk about stories of surprise announcements and how that affects stock returns, and we can talk about “oh, so you think you’re going to beat the market, well let me explain to you how hard it is and the reasons why …” also were very fun. So, we can spend the first week talking about fun stuff. It’s a giant bait and switch.

[LAUGHTER]

I lure them in with fun stuff and then beat them to death for the next fourteen weeks with discounted cash flow analysis.

Rebecca: It also sounds like it’s a good way to motivate students. Not only is it fun, but it gets students motivated and interested and they buy into the class, which I wouldn’t discount that. I think that’s an important task.

Alex: Oh, absolutely.

John: …discounting in a different sense, but…

[LAUGHTER]

Rebecca: Yeah, sorry… I’m a designer

John: …but it also starts a class by dispelling that myth that they’re going to learn tools that will allow them to become really wealthy in financial investments and so forth. So, you’re setting it up by getting rid of that myth and they’re ready to start actually learning without having that at the background.

Alex: Correct.

Rebecca: I really like the idea of just meeting students where they’re at. If this is where some students are coming with, and maybe they’re super motivated in some ways but not in others, that you just tackle both of those in one week.

Alex: That’s right.

Rebecca: Cool.
You recently won a 2018 George R. Brown Award for superior teaching, and the Georgia R. Brown Awards are based on an interesting selection process. Can you tell us a little bit about that?

Alex: Yeah, so what the university does is they solicit feedback from recent alumni, people who graduated, I think, it’s two and five years ago. Now, they have graduated and they’re looking back what teachers would they want to see when these awards… and so, it’s really a neat honor, because once the students are two, three, four. five years out, they’re not responding to the short-term incentives that some professors used to gain teaching evaluations and things like that, they’re looking back and they’re actually remembering what the course was, what the professor was like, and whether it was meaningful to them. It’s really been just a phenomenal honor, and one that I honestly thought that I would never win. So, it’s been really special to have gotten that award.

John: It’s a really interesting idea to have an award given in that way, because it focuses on that long-term learning, that you’ve taught them something that’s going to benefit them later rather than, as you said, something that people do short term. I know some faculty, when we’re doing course evaluations, will give out cookies or other things just to boost their short-term course evaluations. But, that’s not going to pay off very much two to five years after graduation.

Alex: That’s correct. I went through the cookies and brownies phase myself years ago and decided that that just wasn’t who I wanted to be, so…. Now, we’re on to just just the pedagogy at this point.

Rebecca: I think there’s a couple of interesting things that I see about those awards, too, and one is that is more meaningful probably to win it because you can feel the impact but two it meshes really well with what we know about evidence-based practices in that students generally don’t like them while they’re happening, but they have longer-term effects… and that students tend to learn that material and transfer that material later on.

Alex: That’s absolutely correct, and I’m a big subscriber to that basic view that students often don’t like things that make them uncomfortable, and learning new and difficult things is uncomfortable. I think one could make students happier in the short run by giving them lots of little assignments where they feel like they’re making progress every day but they’re not actually being challenged, they’re not actually being pushed, and so instead holding them to very high standards for demonstrating their mastery of material while that is very uncomfortable for some of them, it is that that makes them better students and better scholars of that topic.

John: It reminds me of Bjork and Bjork, in their writings on “desirable difficulties.” that the most learning occurs when students are faced with feasible challenges… that if things are too easy, they get bored, and they may be happy with the course if they don’t have to struggle much… but they learn the most when they’re struggling but they see it’s possible.

We’ve heard that this award, though, based on some of the write-ups we’ve seen at your institution, may somehow be tied to marmots, wolves, and The Princess Bride. What do they all have to do with finance?

Alex: Several years ago, I was reading some books I thought would be…. some I thought it’d be helpful for my teaching, some that I just thought would be helpful for me, and one of the books was a book called Brain Rules by John Medina, and it’s basically a book that tries to take cognitive science, brain science, down to a level that lay people can understand and gives several rules of thumb of how the brain works and why the brain works the way it does… and as I’m reading this, reading it mostly for my own consumption so that I can be a better researcher and more thoughtful person, smarter, that sort of thing, I realized “Gosh, a lot of the rules here applied very directly to teaching, at least in the lecture format that I use in most of my courses…” and so one of the things that really stood out to me is how people learn better, remember better, I guess I should say, when they see images images that relate to whatever the topic at hand is. So, text maybe a PowerPoint slide with text, and you remember X percent but if you see an image, you remember much more of that material later on… and so this gave me just a license to, all of a sudden, start having fun on a completely new dimension. Reading this book and sort of embracing the notion that I could maybe help students remember the material better simply by infusing my lecture slides with some relevant images, was just eye opening for me… because now I could take my completely dry, boring slides with words and numbers and equations and now I can have fun with them…. and have this entire new dimension, a degree of freedom, to play around with what the slides are gonna look like and how the students are going to experience them. So, that’s sort of the extensive margin. The first part is: “Hey, I need images…” so the intrinsic margin is “what kind of images do I need?” “what will work best?” and Brain Rules comes to the rescue there again… and it says people respond to images that are faces, that are things that are scary, things that they can eat, things that might want to eat them, and things that they might want to mate with. Well, that last one’s kind of out for most of my lecture slides… [LAUGHTER] I can’t really incorporate that very directly. But it got me thinking “okay, what’s scary?” So, I started looking around for images of things that are scary to people… and I found this great image of this really, just terrifying, snarling wolf. I’m like “Ah, I’m gonna use that to get people’s attention…” But, as I started thinking about it, I wanted to lead into the wolf a little bit… and so the main prey of wolves in North America are marmots… yellow-bellied marmots… and so I found this great image of a little cute yellow-bellied marmot sitting on a rock somewhere in the mountain somewhere… and now when I come in to teach what could be the absolute driest lecture of the entire course which is time value of money and understanding how to discount cash flows (it’s the tool that everything else builds on, so it’s incredibly important to get it right, but it’s also potentially incredibly technical and boring)… and so I start that lecture not with an equation not with numbers but with a giant image of a marmot filling the entire computer’s projection screen…. and I just leave it up there… and the class is all looking at it… and I look back at them… and I pick someone at random. I cold call… and I say: “So, Charlotte, what do you think?”

“What do you mean what do I think?”

“What is it?”

… and we go through a series of guesses, and the guesses range from just ludicrous things: “it’s a gopher.”

“No, no, clearly it’s not a gopher. Gophers are a lowland creature. This is obviously in the mountains.”

“It’s an otter.”

“No, no, no. Otters prefer marine habitats, and this is clearly not there.”

… and go through this for a while until usually somebody recognizes it as a marmot. I say “Yes, very good. Alright..”

…and so then on to the next slide and the next slide is this picture of George Soros with no caption, no explanation, but again filling the entire screen, here’s this giant picture of George Soros… and so I go back to the first person: “Charlotte what do you think about this one? You didn’t get the marmot, how about this one?” …and so invariably somebody will eventually guess it’s a hedge fund manager.

I say: “Yes, very good. That’s George Soros, a famous hedge fund manager.” then the next slide is the wolf, the snarling wolf, and so at this point the captions on the slides read “This is a yellow-bellied marmot,” “This is a hedge fund manager” … and then it’s obvious what the wolf is. People get that right away. So, Charlotte gets to redeem herself at that point.

[LAUGHTER]

“This is a wolf, one of the main predators of the yellow-bellied marmot,” and then the next slide is another picture of a marmot but this one looking somewhat quizzical and the caption here is: “Why do wolves eat marmots, but not hedge fund managers?” …and the answer that I propose is because hedge fund managers understand the time value of money, but marmots do not. Now, that obviously doesn’t actually follow, but the correlation is there… that part is true… and so I just sort of lean into that and the ridiculousness behind that statement and from there on the marmot becomes our time value of money mascot… and so every time throughout the course I introduced a new application of time value of money then I bring in another image of a marmot to sort of tie all that together. iIt’s hard to see that the tie that binds everything together is time value of money and so the marmot, the mascot, is the visual cue, that “oh, this isn’t special, this isn’t different, this is just another application of the same technique we’ve been doing over and over again.

John: That notion of using visual cues goes back to the Greeks who use it to remember long stories before there was much printed word, and one of the arguments is that it’s because visual imagery developed much earlier than the use of language and the things you described in terms of things that might eat you may be tied back to our evolutionary adaptation, and so we’re tying into things that evolve fairly early in the evolution of human beings.

Rebecca: …and it also is just that emotional response.

John: Right, when you trigger an emotional response and fear is a major one…

Rebecca: Yeah. mm-hmm

John: Do the wolf’s chase the Princess Bride? I don’t remember that…

Alex: Well no…. So, the Princess Bride lecture is where I talk about decision rules using discounted cash flow techniques…. and so this sort of standard playbook for any finance professor anywhere as you talk about net present value and internal rate of return and payback period and maybe a few other rules as well, you just sort of go through the… whatever they have in the book in the order in the book, and you talk about each one in turn. But, it’s a lot more fun if you can personify those.. and so that’s what I use the characters from Princess Bride to do… and so the main character, which is net present value, is personified as Wesley / (there’s a spoiler alert here – so you haven’t seen the movie Wesley is the Dread Pirate Roberts). I always show my kids my lecture slides and when I showed them that one my older daughter was mortified that I would ruin the Princess Bride movie for students by revealing that Wesley and the Dread Pirate Roberts are one and the same… and so from there, I personify the different rules based on the different characters in Princess Bride… and so internal rate of return which is sort of flashy and useful but has some flaws as Inigo Montoya; the payback period which is sort of a blunt instrument is Fezzak the Giant, and so forth… modified internal rate of return is the Cliffs of Insanity.

[LAUGHTER]

Rebecca: So, the important thing to ask then is do you have the spoiler alert at the beginning of your lecture.
AEX: I do now.

[LAUGHTER]

John: How old was your daughter at the time when she reacted to that?

Alex: About ten years old.

John: Okay. So, by the time students are in college, they’ve probably either seen it or they may be less likely to, so it probably doesn’t do quite as much damage.

Alex: I hope so, and in fact I even now encourage them to watch the movie before that lecture without really telling them why. So some of them do. Some of them ask around and figure out why, but that makes it more fun.

Rebecca: That was gonna be my follow-up… is that homework?

Alex: It’s the best homework they’ll ever have.

John: So, how have students responded to this? Do they remember this later?

Alex: Yes and no. for the Princess Bride lecture I believe that the students sort of uniformly enjoy that…. but I don’t have a good sense of how much that actually impacts the depth of their learning. For the marmots, it’s a completely mixed bag… some of the students really love it, and I really do lean hard on the fact that this is the mascot… this is the thing that ties it all together… and this is the visual cue so you will now recognize that when we do bond pricing “oh, here’s a marmot, that means it’s just time value of money…” “oh, when we do net present value, here’s a marmot showing up alongside our Princess Bride character it’s just time value of money” and so forth. Many of the students really enjoy that and grab on to it and some students, they’re not having any of it. They want boring… they want dry… and I’m afraid they’ve come to the wrong place. Been there and done that. I don’t do that anymore.

John: But that use of imagery is really common. People who work on developing memory… the memory palace type things where you tie specific concepts and bundles of concepts with chunks of item to key images, helps people remember things long term. I can see how it would be really effective.

Rebecca: I also could see that one of the things that students often struggle with when information is new to them is making those connections. So providing that visual cue like “here’s time to make a connection…” it’s actually really helpful, because those are the kinds of things that might seem really abstract and very separate if you’re not making an explicit. So, I like that you don’t even have to say explicitly like “this is the thing” you have by putting the image up there, and you’re prompting the students to predict what that connection is or challenging them to think of it on their own before you reveal what that connection is.

Alex: That’s right, and so what started off as just a fun way to get attention: “Hey, class is starting let’s all key in… Here’s a scary wolf.” So, now we’re all paying attention to the scary wolf because that’s how our brains are hard-wired. It’s now grown into this entire thing where throughout the entire class is a continuing callback.

Rebecca: It sounds to me like you must have some pressure every semester to have to have something new that you introduce into some sort of lecture so that there’s some anticipation.

Alex: I’ll be the first one to say that I shamelessly recycle all of my jokes…

[LAUGHTER]

… and so I’m always sort of terrified when I have a student who started the class last year but had to drop at the 6 week mark or the 8 week mark because whatever was going on their life, and then they’re back the next year. Like “oh man, you’re gonna get the exact same jokes with the exact same timing, the exact same patter.” It mostly seems to work fine, and I do continually try to incorporate new things. One of the nice things about teaching finance, although some of it is very static… the basic concept of time value of money is going to be there for basically forever and I won’t need to change the actual examples in those slides really ever… but a lot of the other material changes very rapidly. So, the notion of what does market efficiency mean? who can beat the market and win? what is the evidence? But for topics like market efficiency, for topics like financial markets, for topics like investment banking, those areas transform rapidly… and so I’m continually changing those lectures year after year because one of the big topics that I cover in my course that is not really traditional for an undergrad business finance class is how firms raise external capital. The reason why is because it’s one of my main research interests, and so I have lots of ideas of what I want the students to know, and lots of research that I can tie into the lectures that I give. For the past five or ten years financial markets, the regulation of financial markets, firms’ ability to raise external capital, has changed tremendously as regulation has changed and so I’m continually revamping that portion of the course basically every time that I’m breaking it out.

Rebecca: One thing that I can’t help but think is that you and John have some similar backgrounds in terms of content, so the time value of money that you keep referencing probably makes sense to you but I feel like our listeners might not know exactly what that is so maybe we should just take a minute and give a quick cap of that so people know.

Alex: Yeah, absolutely. So, if you have $100 and you invest it earning an interest rate of 5% per year, in a year how much money will you have?

Rebecca: One hundred and five?

Alex: One hundred and five dollars.

[LAUGHTER]

Rebecca: It’s not a trick question, right?

Alex: No, No, it’s not…

[LAUGHTER]
… and so, as you invest money it earns some rate of return and so then money in the future you have more, because it earns some positive rate of return, and conversely, would you rather have $100 today or $100 in a year? Well, the answer is $100 today, because you could invest that for that year and have the hundred dollars plus some additional return. So, you’d have a hundred and five dollars one year from now instead of the: “oh, I’m going to give you a hundred dollars a year from now” and that’s all the time value of money is.. and then it’s application of that over multiple cash flows and multiple periods where people start to lose track that it’s just math.

John: That’s the fun part.

Rebecca: Yeah. Yeah.

John: I also ask my students have they ever burned their mouth on a slice of pizza, and why don’t they just wait? ..and it’s one way of introducing the notion that we prefer things now to later. It’s a fairly important concept in economics and finance and it’s at the basis of finance.

Alex: Oh, absolutely… and I like the way of framing it as how patient you are. Are you willing to wait or are you impatient? …and so the way that I sometimes describe that in class is if you have a low discount rate that means that the future is worth about the same as the present and so you are patient. If you have a very high discount rate well then the future and the present are very different, so high discount rates → impatient, low discount rates → patient.

Rebecca: You realize that the irony of this whole thing is that I have these conversations with my husband all the time because he’s in finance, right? Yeah….

[LAUGHTER]

John: Ok, well, does that help explain it?

Rebecca: Yeah, yeah. I’ve had it explained to me many times.

[LAUGHTER]

I figured our listeners would need it.

Alex: Well, since we’re talking about the things that our students do, I’ll just share one example that I use in class that I continue to use even though it completely does not work with undergraduate students. It’s when I’m trying to introduce the concept of sunk costs… and so a sunk cost is some amount that you have paid… a cash flow that has happened in the past perhaps… and so once that is paid, you can’t get it back …and so it’s like the notion of “should he throw good money after bad” is another way of phrasing it… and so, what I used to say is, “Well, you go to a movie, you pay your fee to go in, and you decide it’s a terrible movie, should you keep watching to get your money’s worth or should you leave? and then a student pointed out to me one year that well, actually, if you go and you complain you can get your money back from the management.

[LAUGHTER]

Okay, different example. This is the one that never works on the undergrads. You go to an all-you-can-eat buffet and you have a choice of do you buy one plate for this amount or do you pay a little more to get the all-you-can-eat buffet and you decide to go for the all-you-can-eat. You pay the $10 for the all-you-can-eat instead of the $7 for the single plate… and you go and you fill up your plate and you eat… but you decide you’re full after just your first plate… but you’ve already paid that all-you-can-eat price. Should you go back for more? …and the intellectual scholarly answer is “Well, no, because the marginal benefit of eating more food is negative at this point because you’re full. Even though the marginal cost is zero.” But, for the students, the undergraduates, the marginal benefit of eating more food is always positive.

[LAUGHTER]

They view it as a sport. Dinner’s not over until they kick me out.

Rebecca: I think that’s really important to consider your audience and what works. So, I think that’s a really good demonstration of knowing your audience and why something might not work for a particular audience.

Alex: Yes, nonetheless, I still use that example… but I tell them ahead of time “I know this is not going to make sense to you. You’re going to push back on me. Nonetheless, I want to talk it through with you so that we can think about who has marginal benefits and marginal costs of what amount.”

John: Just last semester I used exactly the same thing of asking them how many of them would leave a movie theater if it was clear that they weren’t enjoying the movie and this time I had some people argue “I do that all the time.” So, yeah…

Rebecca: Movie theaters? Don’t you just have Netflix?

[LAUGHTER]

John: Okay, that’s actually a relevant point. In terms of the Princess Bride, we’re moving into a generation where many of your students might not have seen that. Do you have any other movie references that you might substitute in the future?

Alex: You know, I haven’t found the right one yet and, honestly, even if I found a good substitute, I’d really want to keep the Princess Bride, just to give them the incentive to watch that movie.

[LAUGHTER]

It is such a classic and such a gem. I really would like for them to all see that movie.

Rebecca: It’s an investment you’ve made.

[LAUGHTER]

Alex: That’s right.

Rebecca: How have faculty responded to the methods that you’ve been using?

Alex: Faculty are not surprised that I do the things that I do.

[LAUGHTER]

John: They’ve known you for a while.

Alex: They’ve known me for a while and I really lean into the corny pun bad jokes big time in class…. and I really I’m quite shameless about it.

John: That’s not new… I seem to remember that back at Duke when you were still in college.

Alex: …and it’s just a question of do I try to suppress that innate desire or not?

[LAUGHTER]

…and I’m at the stage in my career where “nope, not anymore.” You’re just gonna let it all out. My colleagues are not really surprised, but interestingly, one of my colleagues has adapted her lectures quite a bit to embracing this notion of adding images into the slide decks… and that’s my wife. She teaches business law and regulation of business. She was an attorney by training and I described her as a born-again economist because when she started teaching regulation of business she had to teach herself basically all of public choice economics. So, she got a couple of high-level textbooks and worked through them all but she and I regularly discuss teaching techniques and so she now has gone down the rabbit hole of finding that perfect image to highlight the point that you want to make… to have that really stretched metaphor that you can then call back throughout your course. So, it’s been a lot of fun to have her as a sounding board to go back and forth with.

Rebecca: I appreciate that you’ve brought design into the process.

Alex: Yeah.

John: Excellent. It’s something we all should probably do more of and think about more and certainly much more effective than those PowerPoint bulleted lists that are so common.

Alex: Yeah, a little bit of both actually goes a long way.

Rebecca: You’ve won some awards for your research. How do you maintain a balance between teaching and research?

Alex: It’s difficult. There are some ways in which research can feed directly into the teaching… and so my research that touches on market efficiency, some of that can come into the classroom. My research that touches on how firms raise external capital, some of that can come into the classroom. Now when I teach my PhD courses, those are heavily flavored by my research interests and preferences, but when I teach the undergraduate core finance class there’s not a whole lot that can flow back from that teaching into the research. So, that’s one of the downsides of teaching that class as opposed to perhaps a class that is more specialized or an elective or something that’s a little bit further downstream from the core class.

Rebecca: But at the same time, if it’s something that you teach routinely then the prep isn’t as difficult.

Alex: Absolutely. That’s true. It is hard to overstate how useful it is to teach a class a second time, or a third, or a fourth, or twentieth. Nonetheless, every single year, even though it’s the same class and mostly the same topics, I go through every slide, every lecture from the beginning every year.

Rebecca: But, I remember teaching a bigger selection of classes and one of the things that I’ve liked about my position at Oswego is that that suite of classes has gotten smaller… and then there’s a little less I’ll keep on top of to make sure that you have all that fresh information and what-have-you for classes.

Alex: Absolutely, and I’ve done the same thing. I’ve taught a variety of courses over the years and it’s been nice… as you describe it that suite of classes narrows, so that you have the same core group of classes that you’re teaching over and over again and you can start to specialize. You can really invest the time to get over the fixed costs of finding all those right images for the slides… to going through and taking time to invest in the design aspects of the lectures. That if you had four different courses every term, it would be incredibly difficult to find the time to do that effectively.

John: While you’re working with PhD students, you must do quite a bit of mentoring of them. How do you see the role of a faculty member as a mentor for graduate and undergraduate students?

Alex: Faculty vary widely on their views of how much mentoring PhD students should have… and so you have one model where it’s sink or swim… the PhD students are some of the smartest people in the world… they’re good students… they’ll figure stuff out… just point them in the right direction and let them go, and they’ll get there. That’s not the view that I subscribe to. Because I think we frequently overestimate just how much the doctoral students know, particularly about how the profession works. There’s no book for that. They can’t just go down to the library and find a textbook on how to be a good assistant professor or… there are books on how to write a dissertation but that only gets you so far, because it really needs to be very field specific. So, I tend to go very much the other way, which is a lot of sort of high-touch mentoring. I write co-authored papers with many PhD students, one of whom is now your colleague there at Oswego in the Finance Department in the School of Business, and it’s enjoyable for me. It is a good learning experience for the students and I think it helps them to learn how the profession works much more efficiently, because when it comes time to write a paper and they might put together some tables and say I want to structure the introduction this way. Oh, no, no, no, no, we can’t do that… because if you do that then it’ll make people be concerned about this issue here. So, instead, we need to twist it around this way and start with this… start with the big picture, not what your paper does but what your paper’s about. That sort of thing. That’s hard to learn on your own.
PhD students are PhD students because they are extraordinarily good students and they’re really good at learning. Though, that’s not the job for academics. The job is not the learning, the job is creating knowledge… and the transition from being a consumer to a producer of knowledge is scary, and it is the road that has very few signs or roadmaps to help them get down. It’s a transition of going from a consumer to a producer of knowledge… is very profound for a lot of people.

Rebecca: What’s interesting about what you’re hearing is you know my field the terminal degrees in an MFA a Master of Fine Arts, and the undergraduate degrees are really professional degrees. But, it’s the creators of cultural content ,and so that struggle happens at the undergraduate level too, of going from being that consumer of culture to a producer of culture. It’s really not that much different… just what they’re creating is a bit different.

Alex: That’s right.

John: I remember when I was working on my PhD, one time, where up until that point I had been meeting with my advisor every month to talk about my research, and at first I was just asking him questions… and then he was asking me questions and I realized suddenly that I knew more about the topic than he did… and that’s I think that sort of transition that’s sometimes difficult… because when you’re working on your research you’re mostly going out and finding all these earlier studies and so forth but you get to some point where suddenly you become the expert in the field and that’s a tough transition to make. It’s scary, as you said.

Alex: It is. It’s quite the watershed moment when you realize… when you are presenting your research to a room of 30 presumably learned scholars that maybe collectively they know more about the topic than you, but you know more than any individual person in that room… and becoming that expert and then owning it, so that you can write confidently is, I agree, a very tough transition

Rebecca: I like the emphasis on the owning it part. I think that’s key.

Alex: Yeah, imposter syndrome is real.
[LAUGHTER]

John: Everywhere.

Rebecca: So, we usually wrap up by asking “what are you gonna do next?”

Alex: Well, that’s a good question. I really wanna keep doing what I’m doing, making my class better year on year, teaching PhD students and training them year on year and working on research, hopefully research that people will actually find interesting and useful.
I have a sabbatical coming up.

Rebecca: When’s your sabbatical? What’s the countdown?

Alex: Well, the sabbatical is a year from now and in between then, one of my colleagues with whom I co-teach a doctoral seminar, he’s on his sabbatical. So that PhD course, that normally I teach half of, I’m now teaching the entirety of next fall. So, I think the first thing I’m gonna do is prep the rest of that class.

John: Excellent.

Rebecca: Deadlines make a difference.

Alex: Yes, indeed.

John: That procrastination thing… and that time preference…

Well, thank you!

Rebecca: Thank you so much.

Alex: Thanks.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

29. Learning about learning

Learning is hard work. The most commonly used study techniques often provoke the illusion of knowing. David Parisian, a member of the Department of Curriculum and Instruction at SUNY-Oswego joins us in this episode to discuss how he helps students overcome their misperceptions by introducing them to the science of learning.

Show Notes

Transcript

Rebecca: Learning is hard work. The most commonly used study techniques often provoke the illusion of knowing. In this episode, we discuss one faculty member’s success in helping students correct misperceptions by introducing them to the science of learning.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today our guest is Dr. David Parisian from the Department of Curriculum and Instruction at the State University of New York at Oswego.

Rebecca: Today our teas are…

John: Barry’s Irish tea.

David: I’m not drinking tea today.

Rebecca: No?

David: Nope, I’m trying to cut back. [LAUGHTER]

John: With some people, it’s an addiction.

Rebecca: I have a problem. I’m drinking Prince of Wales tea today. Could you tell us a little bit about what you teach and how you became interested in incorporating evidence-based practices in your classes?

David: Well, I teach a few different courses. When I’m on loan to the Computer Science Department for the CSC 103, which is “Computer Tools and Informational Literacy for Educators” and then in the teacher prep program, I do ADO 394, which is “Interdisciplinary Methods” and then EDU 303 and oversee the “Block One Practicum” students, and do the online EDU 430, which is taking concurrent with their student teaching.

Rebecca: What is the 303 and the last one you just mentioned?

David: 303 are the “Block One Practicums.” So, their first semester junior year, when they entered the the block sequence for education, they have to spend time in a classroom. So, Field Placement secures their placements and then they spend a semester observing and helping where they can, but they’re getting their feet wet of being in a classroom. And the EDU 430—the online course—is “Professionalism and Social Justice” and that’s an online course that they take concurrent with their student teaching. So, a lot of the modules that they’re based on actually helps them in preparation for the edTPA that they have to submit for certification.

Rebecca: So how does the evidence-based practices fit into these courses that you’ve been teaching?

David: Well, one of the things that happened is the CSC 103 is designed for freshmen or transfer students that come to us for teaching certifications. So, one of the things we look at ishellip; approach that class from how technology is integrated in education. And one of the thingshellip; there was the book-read we had a few years back by Peter Brown, Make It Stick, kind of reaffirmed my doctoral work as we went through it… and I created a little quiz… a little matched-pair quiz… so that if they had to learn effective means… “does repetition build memory or does retrieval build memory?” and taking the material from the book and organizing this quiz… and what I found out was that all my students failed. So when it came to the 12 Principles within the book that we were testing on, less than one percent passed… which meant that a lot of the students we see coming to us, even though they can come to college, don’t have the knowledge or foundation of what strategies are most effective to learning. And that, in a teacher preparation program, my thought was, “do I continue that propagation through the pipeline or do I try to stop this stop that flow of students who are not effective or aware of the most effective strategies as they go out to be teachers or do we want them to have those effective strategies so they can implement them once they start teaching?”

Rebecca: So you staged an intervention?

David: Yes, it really becomes an intervention at that point and it’s really interesting because they have so many misconceptions coming in. And what we try to do is… they change their relationship with how they view sleeping. We set it up within the course so they take Barbara Oakley’s MOOC Learning How to Learn. So, they’re taught how sleep flushes out the chemicals overnight that are built up in the brain. They talk about procrastination and the Pomodoro techniquehellip; the benefits of flashcards and how to integrate that in spaced practice… and that’s one of the interesting thingshellip; I was just talking students today because they’re preparing presentationshellip; but a hundred percent of the class knew about spaced practice but no one knew how to implement it, or what it might look like.

Rebecca: Right.

David: So we provide opportunities for them to explore those areas, whether it’s creating flashcards where they’re using a flashcard app or have them making flashcards to build in the spaced practice to know when to practice and how to interleave and all those components that research has shown to be most effective.

John: Many of our listeners are familiar with all of these, but could you go back and talk just a little bit about the Pomodoro Technique.

David: The Pomodoro Technique is really fascinating. What it is… is for those who procrastinate, you block out 25 minutes …and actually the Pomodoro Technique is based on a tomato kitchen timer… where you set it for 25 minutes and then at the end of 25 minutes, you take a five-minute break… and then you set the Pomodoro timer and you go through it again. And what happens is you can begin to measure your workload in terms of how many Pomodoros it’d take. For the students, what I notice is they start changing their perception of assignments from a product driven “what’s the end product?” to a process of “how to get to that end product?” So many have commented on that and you can get apps on your phone that are the Pomodoro Technique, that will set up a 25-minute block. There are apps that will stop any notifications for that time so you don’t have to be keep looking at their phone because that’s the distraction that I’ve noticed, is that people are attached to their devices and have to be aware of every beep and every notification and everything that comes as opposed to blocking time. So the Pomodoro Technique has been pretty effective and most students have implemented that in terms of working on their procrastination skills or adjusting… changing behaviors.

John: So why do students have these misperceptions?

David: Good question. I come from a K-12 environment. I was a secondary science teacher for 18 years. I was a district administrator for another 12. So, coming from the K-12 environment, what my message was to them is that you were taught by loving, caring educators who were passionate about what they did and did the best they could with what they had… based on the information. That might not have been entirely best practices of what we now know about neuroscience and how the brain learns and the effective study strategies on the materials from Make It Stick. So, I think they grew up with whatever progression they went through. One thing I did notice is that students who struggled… that worked hard in school but just got into collegehellip; still struggle with the workload. What surprised me was talking to some studentshellip; is the students who were the bright ones… the ones that got it quick in school… that went through high school with no challenges… come here and all of a sudden, they’re placed in a situation where now they have to study and do all those things and they don’t have the skill sets to study. Just coming up through, I think our assumption is we feel that students know how to study because they’re in college and really what’s taking place is students aren’t really taught how to study. I think the assumption that students know how to study probably backs all the way down into fifth, sixth grade. I think we had some earlier comments where we were talking about us being in school and teachers made us write flashcards. We didn’t understand why and they might not have understood why, but we made flashcards. Now, as that is an appropriate practice, or one that works that you know, that can be integrated into a spaced practice and it’s really just teaching them a little bit about neuroscience… how the brains learns… how that all we do is encode, consolidate, and retrieve… and how do you build those principles and practices using technology… using skillsets… managing their time and trying to put that into a package where they can begin to see it. Because once they see it, then they want to improve their studies so they may begin doing it… so they go to bed earlier, they get better sleep and they begin to change their behaviors. I didn’t expect that but it was an awesome sidebar from that.

John: Part of it is, as you said, they haven’t always had much practice or training in learning how to learn and partly that may be because many of the teachers didn’t have the same…

David: Exactly.

John: …people have been just doing the same things that seemed right for an awfully long time… and one of the problems, though, with some of the evidence-based practices is that it doesn’t feel quite as good… because when you try to work on retrieval practice, after you’ve been away from a topic for a while, it doesn’t feel as good as perhaps repeatedly rereading something until it looks so familiar that you think you understand it. So there is that fluency illusion that people get that feels really comfortable and when you do some low-stakes testing or when you do some attempts at retrieval, you realize you may not understand it quite as well and it doesn’t give you the same sort of reward immediately. So it takes some training, I would think, for students to be convinced that these methods really do work. It sounds like you’ve been able to achieve that in your classes.

David: One of the advantages is it’s a setup so that we use the class and the content within the class as a training mechanism. So, they’ll practice working on a flashcard app to learn the app and then create flashcards for the content. One of the things they do have to do because the modules I set up are in a worksheet form so the initial encoding part is our instructional part. The consolidate part is them now going into another class and applying it in another class and showing the evidence in mine. So that they’re forced to, not only just use it in mine, but now go into a Psych 100 or a LIT course or whatever they’re in to create flashcards, create mind maps, begin to incorporate that… and we map out, When are your tests,?” So, how far do you already have to back up. So it’s really just coaching them on how to utilize those strategies and to get them so they can begin to internalize that to meet with their success.

Rebecca: Do you find that the students are a little resistant at the beginning or do you think because you’re providing the evidence and the science behind… the reasons behind… why you might use a method, is that what’s helping?

David: Well I think it’s a combination of both. Part of it is they have to go through the MOOC… so it’s not me presenting the science. Then we have a series of videos. There’s a gentleman by the name of Thomas Frank, who has a series of videos on studying and using it, so he has a more modern twist to it, being late twenty-something or just through college so it’s a more animated video. And then I’m in class to give suggestions, those type of things, but once they saw that on the first day school, they take a quiz and fail it and then realize that they don’t know what they think they know, then they’re pretty open to “this is what we’re gonna do for the semester.”

Rebecca: Sounds pretty motivating.

John: It is a motivating technique, yeah.

David: Well, I had one student once say like, “first day in college and I’ve already failed a quiz.” I’m like, “rather fail it now then in 15 weeks.”

Rebecca: Yeah.

David: …that’s the poise. It’s really just redefining their tool sets and introducing new ones… and not everyone takes to everything. Like, mind mapping is a classic example. whether it’s Tony Buzan’s method, but I’m very mind map oriented, but a lot of students have a negative experience towards it and I said, “if you want to be a teacher you may never do a mind map but you will know how to do a mind map, because you may have a student that needs to know how to do it and you have to be able to teach it so even if it’s not for you and you don’t like it, you darn well better know how to be able to help someone else learn it.” It’s redefining a different role for the teacher as our candidates come through as be more knowledgeable base, more pragmatic, and more understanding on what effectively works because they can then share that information with their students.

Rebecca: So what do we do for all those students who aren’t in your class in their first year, right, who aren’t becoming teachers but they’re, you know, trying to exist in this system, who might also have the same exact struggles that your students have? What’s your recommendation?

David: Well again, Thomas Francas has great videos and whether it’s how to read a college textbook, how to take flashcards, how to do any of that. That’s one aspect– knowing that you need help is another aspect. We have the success tutors that that have those skill sets, are using the same information to help other students. So, I think there’s resources on campus to support that.

John: …and our Student Success specialists or I do are very familiar with all these techniques and they work with students that are struggling to help build that up.

David: Right, and I think looking at it from a instructor side of the house, just as people become familiar with the strategies… is making those known..making the strategies known. If you’re teaching an Economics course or teaching whatever, if you say there’s a set list of vocabulary or concepts that you need to know, you know, put them in flashcards, go to these links, it’s not like you have to prepare everything because you can’t make meaning for someone else… they have to make it on their own… but you have to provide those initial concepts or the information because the student has to be able to take it in, but telling them how to take it in or just pointing them to links can facilitate that process. I don’t think anyone really comes here with a desire to flunk out, you want to be successful, but some of the times, they get paralyzed by the paralysis of the whole situation.

John: Some of it can be done through course design too, so that they are designed to include some level of retrieval practice,

David: Absolutely.

John: … some amount of interleaved practice,

David: yeah

John: ….and some spaced practice.

David: Yup, yeah, absolutely. And those type of structures in the redesign isn’t that large of a jump to do for the professional side of the house. The biggest thing I can tell for any student, the simplest thing to do if you’re feeling that stressed is breathe. It’s kind of funny… whenever you’re upset, what do people tell you to do they tell you to breathe and just from a physiological standpoint, taking deep breaths… slow deep breaths… once you start breathing about six breaths a minute, there’s a branch of the vagus nerve of the parasympathetic system of our physiology that automatically slows the body down. So, when you’re taking a test, what do you do? Breathe slowly. If you get stressed out on it, draw a circle around every fifth question so that every fifth question, you’re just taking three deep breaths just to say, “this is all right, we can do this.” And you that will calm the physiology and reduce that stress response of seeing an exam… the fight, flight, or freeze… and we’ve all experienced those… with varying degrees of success. [LAUGHTER]

John: But we want to set up a system where students generally be successful.

David: Absolutely… Absolutely.

Rebecca: So you’ve talked about the Pomodoro Technique… you’ve talked about breathing… you’ve talked about flashcards. What are some other key things that instructors could help students think about as strategies to be successful?

John: Well, you mentioned sleep, too.

Rebecca: Sleeping’s good.

John: And that’s something students often have trouble with.

David: Well it was interesting because there was a presentation done once and one of my students… I commented that she would stay up till 2-3 in the morning texting her friends and then they got that part of the MOOC and talking about sleep and she’s the one that started to go to bed at a more reasonable hour to be able to sleep. I think if one hasn’t read Make It Stick… that’s a great book and I think for a lot of us on the professional side of the house… as instructors we’re like the great white shark of our content, we’re the rogue, we’re at the top of the food chain. But one of the things, I think, the book does is lays out for you this way in which people encode information, consolidate, and retrieve it and I think having that as a foundation and then, reaching out to the center here for like, “how would you design something? I’m struggling with this” and just say “I want to reach the students” because I think a lot of professors do want to make those connections with the students. I think there’s help and support on campus to help people in designing those strategies. Personally I start with trying to give the overview… try to give the gestalt of the course… “what’s the wholeness gonna look like” and then just lay out the clear objectives and then integrate and make the assignments, being those flashcards and studying and there’s just so many ways to communicate, I guess. Even taking Barbara Oakley’s MOOC Learning How to Learn and just learning some of the basis from the neuroscience, even from the students. Have students get together informally and go through it. It doesn’t really take a lot, all the information’s out there.

John: And if I recall, it’s offered every month isn’t it? It’s a four-week MOOC and it’s offered very regularly.

David: Well it’s a four week…, like every week now, I think. For those who aren’t aware of the MOOC through Coursera, a MOOC is a massively open online course, Coursera is a vendor for that and Barbara Oakley’s MOOC. I think last year i read in the New York Times where it was the most popular MOOC on the planet.

John: It is i believe still the most popular MOOC…

David: I know when I took it, I took it with two hundred and seventy-five thousand other people at the same time. That was a few years back. Integrating that type of information and layering it into a course just gives a different feel for the students. And as I tell them even though they’re freshmen and transfer, I said “if you’re going to be a teacher, down the road, some of your professional development is going to be delivered through these.” So, we’re just getting them ready orientating them. Well there’s just you know some really simple information that people can do to design, redesign, and be more effective, more engaging and have students be more successful.

Rebecca: I’ve had students respond very positively when you explain why you might do a particular technique or a method. At first it seems like “why are we doing this? why are we doing quizzes?” but if you explain why and how it helps they’re much less resistant and actually embrace the idea. I’ve had students ask for more quizzes because they’ve learned how it’s helpful…

David: That just reminds me of, in our lesson plan development, as the students are going through it… one of the parts of the lesson plan development I have after activating prior knowledge is setting the purpose… and setting the purpose is “so, why is this lesson important? why do you need to know it? How many courses have we ever been in that no one’s ever told us why we’re there and you’re absolutely correct. Once you say the why and give the explanation, it creates an association or connection with the student and that’s one of those underpinning things… as the more associations you can connect to them, then the more apt you’re gonna have their buy-in to engage you in the content.

Rebecca: I think a lot of times students see assessments like quizzes and tests as some sort of penalty…put you in the penalty box or whatever… and that’s a faculty member’s way of torture or something, but as soon as you say that the purpose of doing this is to help you recall information and to make sure that you have that foundational knowledge, you can continue building in these more advanced classes. They stop seeing it that way and it’s pretty amazing that it doesn’t take much of a conversation… it takes having the conversation.

John: And the more frequently you do that, the easier it is for them to get past that because high-stakes testing is stressful, but if you replace it with lower stakes testing and more regular testing, it’s easier for them to see that they’re learning from this experience and it doesn’t hurt them as much if they screw up… that it’s an opportunity for them to improve and continue…

Rebecca: However your students responded after your class like moving into these upper-level classes because you’ve been doing it long enough now that probably some of them are now in those advanced courses I would imagine.

David: Part of it is coming up through the pipeline and having viewed or patterned assessments or quizzes as the “gotcha” and getting hammered. Ken and Rita Dunne, one of the things they stressed that I always kind of stress is “high content, low threat.” So, whenever you can engage and have your material be of high content but a low threat, students are more apt to engage because they don’t have that fear reaction going on and I think it’s changing that patterning by having the conversations that these are beneficial for you. So, that’s my thought on the quizzes. Some of them continue that patterning and that’s one of the things where I try to keep it going for the semester in hopes that they’re gradually continuing to do those processes, those strategies to continue their learning. The one thing about teaching the freshmen… I’m probably one of the few faculty members that have this interesting purview because I see them as freshmen or transfers coming in, I’ll see them as their block one junior EDU 303 practicum. I’ll touch base with them again for the ADO students that take 394 as their block two, so I have them as a class and than in block 4 when they’re seniors going through the the end of the pipeline. So I can see their growth along the way from that and…

John: And ADO is adolescent

David: Yeah, the adolescent, yeah… and that one they’re constructing the lesson plans, they’re trying to see how this all fits together, trying to pull on multiple layers. We revisit Make it Stick because in all my courses that’s just part of it. So, they’ll read the book as a textbook in that section… so then they can begin to refresh like “Oh, I remember when we did this” and “how’s it going?” and then we have conversations on when they’ve been using it and how effective it’s been. I just keep trying and plugging away and stepping up to the plate…. every day you get to take a new swing

John: And the more people who do that in their classes and certainly the more people who are trained to do that at lower levels in elementary and secondary school, the better off students will be.

David: Yeah… especially nowadays when you you look at the various challenges in a k-12 environment. If you can begin to seed the ground with what works and just focus on that then we’ll be okay.

John: Do students go on and use these practices in other classes after they’ve been exposed to them in your intro level course?

David: Part of it, once they leave me is trying to continue that propagation based on the courses they’re in. Some of the strategies are more effective in some content areas than in others. Math is always a struggle in terms of looking at applying the flashcards while you can do color coding or dual coding where you’re including images or multicolored in the equations as you follow different variables through an equation sequence, those type of things. The other thing is that the strategies, and this might be one of the misconceptions students have, learning isn’t easy. Some people comment like “well if I just sit here I should get an A”…… but learning is messy… learning’s organic… learnings dynamic… and learning takes a lot of work and sometimes, depending on the student’s course load and what they’re taking… if you’re taking a 4-credit science course, you got three hours of lecture and a three hour lab somewhere in the week… having been a science major and having a lot of hours on a lab. So part of it is finding time to create the materials… the mind maps… the flashcards. Those students who have a better time management… work ethic, those are the underpinnings I think that makes this a successful student and that they put forth the time and schedule that to do all those things that are necessary, whether it’s creating a mind map or whether it’s creating flashcards or creating the time intervals for the spaced practice or when to do the spaced practice. I was talking to a friend who used flashcards and whenever she was grocery shopping for her family picked the longest line because it wasn’t about getting out of the grocery store fastest, it was about being in the line the longest as she pulled out her set of flashcards and reviewed them in the line at the grocery store. You can find intervals to do those type of things.

John: I always wondered why there were always more people in the longest lines. Maybe…Maybe they’ve been in your class [LAUGHTER].

Rebecca: Yeah… everybody’s doing flashcards.

David: Then again yeah, flashcards is just one aspect of that but you can integrate that from a quiz standpoint… from a retrieval…. and one thing that could be interesting is when you look at the research on flashcards, or how to create them, there isn’t the level of “how do you create” going down Bloom’s continuum of higher processing from a flashcard aspect. A lot of the information we see is low level…. vocabulary words, or those type of things… but how do you all of a sudden take two flashcards and compare them and say “compare and contrast these two concepts of something…” and so how do you get a bigger cognitive load going from using those… and the designing of flashcards… that’d be a great study for someone to do. I’lljust put that out for anyone who’d want to. [LAUGHTER]

John: You get to work with students a bit later in their academic careers that you worked with earlier. How do they respond when they come back in upper-level classes? What do they say about their experiences?

David: Usually, semesters later they forgot about me. [LAUGHTER]

Yeah, but one of the things we try and do at the end of the course is they have a five-minute presentation they have to do… and I give them the slides and so it’s like “how has this changed you?” and they go through and reflect on that. So, I give them a template of what the presentation is and it’s their five minutes of fame where they get to begin to find their teaching voice, and it’s the first time they’re in front of the class talking about it. So, you talk about how they how they did it. In terms of seeing them later on, the people who use flashcards and grew up continue using them and then you have various levels of people who took the buy-in to create those processes. The other thing is you try to encourage them to use it, so as they’re developing their mini unit you have them do the flashcards to go with the unit… you have them do the mind map. So, you have them go through it, and I think from that aspect, they recall fairly quickly what it is they needed to do to generate it… and then it reminds them like “oh, yeah.”

John: But, if we do this in more classes and we use them or we structure our classes so that students naturally adapt some of these practices, it’s going to help reinforce these things… and the more people are reminded of how important this is and how their usual practices may be really helpful in cramming for a test the next day but aren’t going to allow them to remember the things much past that day.

David: Right. In the real world you have to remember those things past that day.

Rebecca: A couple of the themes that you were mentioning most recently is about time management and the work ethic component fitting into this and so it seems like that’s the next discussion. How do you make sure that students know what is a good work ethic? Do they even know? Do these conversations happen? I’m not sure that they do.

David: You can see those students that have work ethic pretty quick …and I’m just trying to flashback through courses and images through my head of students… from a freshman those that are asking questions… those that are getting work done and turning things on time… those who are turning it in early… those who who show up to class earlier… and sitting there… they have a certain level of comfort within themselves… where they’ll ask a question. Truth be told, I didn’t feel comfortable asking a question in college till after seven years of college… not that it took me that long to get a bachelor’s degree… it wasn’t till after I did my student teaching a long time ago… and after I did my student teaching I learned how to study. I did the outlines before the chapter. The following semester I went for a neuroscience certification taking our geology courses. So, I outlined the chapters before I walked in there. I pre-taught myself the material. I laid out all my notes and stuff before the lecture… had the conversations because then it was internalized to me that this was important… and I think until someone gets that into their intrinsic fabric of themselves… where they want to take this as being important… and I still don’t know what I want to be when I grow up. I think that’s the big thing. People who are serious know to get the job done. They’ll do the work and they’ll do it to what they need to do… and if you give them criticism and feedback, they correct it.

Rebecca: I’m hearing a growth mindset described.

David: It is.

Rebecca: …you’re open to feedback… The the real challenge is how do you get students with the fixed mindset, who maybe don’t have that work ethic. or good time management skills. to get on board.

John: …and you mention that case about some students earlier who had done really well in high school and then suddenly struggle when they get here. Those, empirically, tend to be the students who have a fixed mindset… where they’ve been successful with the techniques they’ve used, but once they have to move into a new environment or they have to engage in more transfer than they had to earlier… suddenly face some struggle and it’s a shock to them and they often give up.

David: Well, and one of the interesting things, though, and this they’ll need to be reminded of, but within all of us we have growth and fixed mindsets.

JOHN. Yes.

David: So, I could have… I’m not picking on math but I’ll pick on that…. Most students… math is not a strong aspect, because really the only time you do math is in math class. No one asks you when you’re walking down a street to factor a quadratic equation, right?

John: Well, rarely… [LAUGHTER]

David: …rarely… but you consider a musician. They know with practice they get better. In sports, you know with practice you’ll get better. In math, if you practice you’ll get better. So, part of that is understanding the context of which it is. I may struggle in math but I might be a musician in the band, and so I know. So, you have to kind of transfer where they have been successful in showing them, in this part where they’re not being successful, how they can be. Because, if you can tap into a person’s growth mindset… and it could be in a K-12 environment. You have somebody I saw students riding their skateboards… phenomenal skateboarder… can do all these tricks… will spend hours learning a trick, right? That whole idea can be shifted to their studies as well. How long did it take you to learn this trick?

John: How many times did you fall you know along the way.

David: …and how many times did you fall? The culture is changing where, not only are we imparting the information to the students, but we’re also being their coach. We’re trying to nurture them. We look at them as adults coming here… without the parents for the first time. The baton we’re handed is actually trying to nurture them into the adult working life and understanding how we learn… how we process information… how we interact… the building of rapport. how do the rapport aspect is all part and parcel… I feel… what we need to do… or what we do… I know it’s what I do.

John: How did you integrate the MOOC into your class?

David: I build the class around modules that last two weeks. So, with the beginning of class, I took some of the material from the MOOC or what the topics were and then I created worksheets based around that. So, if it was procrastination then there’s articles that they were reading in, and picking up on, and getting their takeaways… So, that part was teaching the foundational aspects of of how these strategies work, and then giving them time to practice and doing them within the two weeks. The MOOC they can view offline. They can take the quizzes. The other thing I did, is with Barbara Oakley, she had Coursera set me up as an administrator for my course. So, then I could just upload my class list to it and then it would keep track of the quizzes…

John: Oh, nice.

David: …and then I could download the the grades or whatever. So, she was giving him the content on one side; we were building in related practiced and article support on the other side. Then the consolidation part… and I broke it down into it an encoding section of the worksheet, a consolidation, and a retrieval part basically patterning our learning process. So, watching videos and reading we’re encoding… applying the material was the consolidation… and then their reflection and the reflections based on making a video, responding to an interview question, or reflecting on their experience over the two weeks… and they were able to communicate those. So, part of it is just finding out what the MOOC is doing, getting materials that kind of pattern that (that’s where I brought in you know the Thomas Frank videos and other support materials).

John: The learning scientists also have some good ones.

David: Yeah, exactly, and I use a lot of the Learning Scientist’s material and McDaniel’s site deals with retrieval practice, so there’s a variety of things. We try to overlay the MOOC with Make it Stick and strategies there to create an environment that over the first four weeks they’re experiencing… they’re learning… and they’re beginning to apply… and then as we build out the other modules, we still keep repeating… For example this is module four we’re in and one of the things I’m training them in is advanced search strategies. What are the topics they’re going to be searching? Well, there’s eight setups within the lab so… elaboration… retrieval… spaced practice… those are the deep research things. So, each group has to now prepare a presentation, but they have to do the research. What’s the research that supports this? and what are strategies to help? So, now there’ll be eight presentations of the 8 strategies that they’ve learned. Trying to deepen and make touch points through the semester and keep reminding them… constantly reminding… constant… constant…[LAUGHTER]

John: So, what are you going to do next?

David: Well, next will be a continuation with the Computer Science…. Looking for more in-depth application across the content areas to help students… and then professionally, this summer, working with Educational Administration Department and their Project Blend Symposium. We’ll be doing the third installment of Resiliency and Leadership working with the Institute of HeartMath. So, outside of that, we’ll continue to work in those areas of heart-brain synchronicity and just working and having fun.

John: Thank you for joining us. It’s been a pleasure.

David: It’s been my pleasure. Thank you very much.

Rebecca: It’s always great to hear what you’re doing in your classes and the results and thanks for sharing that for everybody else.

David: Very good, thank you .

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

23. Teaching with comics

Looking for ways to increase student confidence in their ability to learn? Or their ability to see themselves as professionals in the field? In this episode, Carly Tribulli, a Biology Professor at SUNY-Farmingdale, joins us to discuss how comics may be created and used to meet students where they’re at, draw them in, and help them develop mental models of complicated processes and concepts.

We discuss Carly’s plans to create an OER biology textbook in which biological processes are represented using comic strips, her planned research on the effectiveness of instructional use of comics, as well the positive role model that she provides in Carly’s Adventures in Waspland, an instructional comic that Carly created for the American Museum of Natural History during her graduate study there.

Show Notes

Carly’s Work

Topics mentioned in the podcast (in order of their appearance):

Economics comic books:

STEM web comics recommended by Carly:

Transcript

Rebecca: Looking for ways to increase student confidence in their ability to learn? Or their ability to see themselves as professionals in the field? In this episode, we’ll explore how one faculty member uses comics to meet students where they’re at, draw them in, and help them develop mental models of complicated processes and concepts.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Carly Tribull, an assistant professor at Farmingdale State College, where she mostly teaches general biology for non major students in entomology. Her interests include bugs, biology, and of course, comics. Welcome, Carly.

Carly: Hi, nice to meet you guys.

John: Welcome. Today our teas are…

Carly: I’m actually drinking… a kind of cold coffee. But, but it’s good. I like it.

John: Yeah.

Rebecca: …and it used to be warm.

Carly: It used to be warm. I got it about an hour ago, so I knew this was going to happen, but I was like “You know, this is my only opportunity to get coffee, and I know you guys like to talk about what we’re drinking…”, and I was like “ooh, yeah a coffee, cool… I could have lied…”

Rebecca: That’s true. I have a Paris tea.

John: and I have blueberry green tea.

Rebecca: So, Carly, can you tell us a little bit about how you’ve been able to combine your interests in art and biology in your educational and career paths?

Carly: So, I’ve always been interested in both art and biology ever since I was a little kid. I grew up in a very science-forward family. There was a lot of interest in me becoming a biologist and my parents were both very encouraging, and my dad always sat and watched those sort of Wild Animal channel, Discovery Channel shows when I was a kid, with all like this farming animals and stuff like that. So, I was always interested in the animals and eventually that led to drawing animals. By the time that I was in high school, I was taking formal training in art and doing AP art and things like that, but also very much maintaining my biology education. By the time I was later in high school, I was drawing comics. I had discovered comics around early high school. I read a lot of manga, and then I started reading more graphic novels, never a lot of the superhero comics, but more of the weird offbeat stuff like the Sandman, and a bunch of manga series. So I started drawing comics, and I drew a bunch of weird comics and then I entered college at UC Berkeley, and I was a double major in art and biology, and I just continued that path all the way through. And I was really stubborn about not giving up art, despite the fact that I had chosen not to go to a traditional art school. I knew at that point I was going to go into biology, but I was very much stubbornly holding on to art, and so what happened when I was at Berkeley, is that I was actually able to do biological illustration as an undergraduate researcher. And that was the very first research experience I ever had, doing biological illustration for a paleontology lab. This has always made sense to me as a biologist, because there’s a really, really huge history of biology and art meeting together. Especially in entomology, when you consider the work of Maria Sybilla Marian, who is one the famous female entomologists of her time (probably the only major female entomologist of her time) and she was really the first person to study metamorphosis. And much of the way she shared that information, since this was obviously way before photography, was by these really elaborate illustrations that were shared with other entomologists at the time. So to me, it’s always made sense that there is some sort of crossover between biology and art, and I think while I was in college I was very stubbornly imagining myself as becoming that type of natural historian. And then when I was in graduate school there was a lot of encouragement for me to continue doing comics, weirdly enough.

John: Could you tell us a little bit about your graduate program?

Carly: I went to the Richard Gilder graduate school at the American Museum of Natural History, and that’s a pretty long name, but historically the museum has always funded graduate students from the City University of New York, and from the Bronx Botanical Gardens, and from NYU and Columbia, but only within the past seven or eight years or so, did they decide to start their own in-house PhD program. So, we still have all of those students that are coming from other institutions, but only recently where we like, we’re going to create our own graduate program. It was very, very, very different from your standard evolutionary biology PhD program. Usually the big state public schools, and a few of the private schools that are strong in the sciences, have an evolution in ecology, biology grad program that you spend five to six years and that you TA undergraduates to support your stipend. But at the AMNH, because it’s a museum, there are no undergraduates for you to TA. and you also have to finish in four years. So, because you had no formal TAships, and the funding was very good so you didn’t really need them, you were very much encouraged to do these informal teaching assistantships, and to find your way into the outreach education side of the museum, or working on exhibits and making yourself part of the contributing community to the museum. That is basically how the grad school ran, and I did my PhD in the evolutionary systematics of these parasitoid wasps that I study.

John: It sounded like a really natural blend of your interest and a superb educational path for you, in terms of giving you a way of continuing your earlier interest.

Rebecca: Before we jump forward I’m really curious, Carly, as an art faculty member, if you could talk a little bit about that first project, that first opportunity you had as a student and how you got that opportunity to combine your interests. Was it something that you pursued or was it something that your faculty helped to nurture?

Carly: Kind of a combination of both. My freshman year at Berkeley, I took an undergraduate symposium with Kevin Padian, who is a vertebrate paleontologist, and it was very much your standard freshman seminar. It was actually very small, it was only about 10 students. We did some readings, we did some talking, and around that time I think I was looking for research opportunities, and so I started talking with him and I started trying to get myself into the lab as an undergraduate researcher for future semesters, and it came up that I’m a biological illustrator, or that I was interested in biological illustration, and I think at some point he was like “okay, show me what you got, go draw the T-Rex,” because there’s a big T-Rex in the center floor of the Valley Life Sciences Building at Berkeley. And I went down and I drew it as best as I could and apparently he was pretty satisfied with my work. So, I joined the lab, and I was assigned to a current PhD student at the time named Katie Brakora, and I actually drew some of the images that were used in her dissertation. And that was excellent. I didn’t become Kevin Padian’s biological illustrator, but I was working with grad students that were going through grad student life, finishing their work… and at the same time I was taking the core art classes, because I was a double major and I knew I was going to be a double major for my freshman year. So, I was doing all of your standard intro to drawing, intro to painting, techniques classes, and things like that and it actually worked out really well for me to be a biological illustrator, as sort of a side biology undergraduate researcher, because Berkeley’s art program isn’t really focused on illustration or comics. It’s actually much more of a fine arts program. So, sometimes I was actually butting heads with the other art faculty, because I was very illustration focused and they’re very studio fine arts, and I was like not all of us are going to become studio painters. So, illustration seems like a skill that I should be investing in.

Rebecca: What a great story. Thanks, Carly.

John: While you’re in grad school, one of your projects was developing Carly’s Adventures in Wasp Land, and we’ve looked through that and it’s superbly drawn and fascinating. Could you tell us a little bit more about some of your work with illustrations and developing comics while you were at the American Museum?

Carly: I guess this goes back to how Carly’s Adventures in Wasp Land started, which by the way is not the title I came up with it, that was the title that the museum folks came up with it, I was just like, “okay.”

John: Did you have a title?

Carly: No, I did not have a title, that was probably an error on my part. I was opening myself up there, I think it’s a fine title. It’s a little bit goofy that it has my own name in it, but, whatever. In my interview to get into grad school, I had actually brought my portfolio in biological illustration, which was very unusual. Of course, evolutionary biology does attract people who can draw, but I think I was the first person who had come to that relatively new program with a portfolio. [LAUGHTER] I was kind of a scrappy undergraduate. I didn’t do that great in my courses. I’m a terrible memorizer, which allows me to sympathize with other students that aren’t doing that great in intro biology, especially my own students, because I actually didn’t do all that well for the first two years. And part of making myself an attractive student to graduate schools, was actually building up my research curriculum. I did a lot of research with Marvelee Wake at Berkeley after the Padian lab, and then also building up this biological illustration thing early on. I interviewed with Jim Carpenter, he accepted me to his lab, and I think he was very impressed with the fact that I did illustration and apparently it stuck with him enough that when he got a grant from the NSF, he came to me about helping him out with the broader impact section of that grant, and broader impacts is where you actually have to make your grant meaningful outside of academia. So, it’s where you would have outreach education. He remembered from my interview that I like to draw, he came to me and he was like “do you want to work with the digital outreach education side of the museum, and create a project with them? “And I was like “yeah, sure,” and as long as it was about teaching kids about wasps, and the different types of wasps, I pretty much had free rein. I started working with Ology, which is the digital outreach section of the museum, and a lot of what would happen is collaboration between me, Jim, and the Ology folks, especially when it came to writing the script for that comic, because the Ology folks have way more experience in writing for middle school readers than I did. So there was a lot of modification of my script but mostly I had free reign when it came to the illustration side of things, and I also mostly had free reign when it came to the creative decisions, like the decisions to make the wasps anthropomorphic and have them talking with you, that was something I decided on, even though it isn’t truly a hundred percent scientifically accurate. It was something that both the Ology folks and Jim signed off on.

John: I liked it.

Rebecca: I thought it worked well for adults too, I don’t think it’s just for middle schoolers. I’m just saying… I know way more about wasps now than I did before I read it.

John: Me too, and it was much more engaging than reading a textbook description of those things.

Carly: Thank you so much!

Rebecca: I also just really love that you’re like a superhero in the story. What a great way for little girls and boys to see a strong female scientist… taking on the wasp. I just thought it was a really great way to frame the story.

Carly: Yeah, and I think the first chapter in Carly’s Adventures in Wasp Land doesn’t actually talk about wasps but it sort of talks about me and how I became an entomologist. That wasn’t part of the original plan, but me and the folks at Ology, and eventually Jim was totally on board with this, felt that it was important that part of the broader impacts, should be showing young girls that they too could be an entomologist, this field that is commonly associated (at least by other people who are outside of entomology) as being male-dominated and being a career for boys… showing them that, that’s not necessarily the case. So, that’s when the strengths of comics especially when it comes to showing girls and underrepresented minority students that they can envision themselves also as scientists. That’s one of the things you can do with comics that I find really engaging… is that, in your choice of narrator, you can make those decisions.

John: I believe you’re releasing some of your materials under an OER license. Is that correct?

Carly: Yes, not Carly’s Adventures in Wasp Land. That is an OER in that it’s freely available, but it’s going to stick with the museum’s website for the time being (as far as I know). What I’m putting on an OER license is actually the comic textbook that I’m going to be eventually making for the Farmingdale State general biology students, but it’s certainly going to be available to any SUNY professor or any professor anywhere.

John: Have you requested an grant for that or are you doing this on your own?

Carly: So, I’ve requested a summer stipend and I should be finding out about that soon. As you might guess drawing and writing comics takes a lot of time, much longer than say a written textbook would take, and there are certainly many professors that are working on written OERs for their class. So, I’ve requested a summer stipend and SUNY Farmingdale has recently announced that there’s going to be an OER incentive grant, so I’ll be applying for that too.

John: Excellent.

Rebecca: You’ve also done some writing about using comics for science specifically, can you talk a little bit about the research that you’ve done in this area?

Carly: Yeah, so I think when I accepted the job at Farmingdale, I knew that I was going to be very, very, interested in making comics and researching the impact of comics… part of the research that I do for my tenure decision… and luckily the faculty here have been very supportive of that. Farmingdale is a primarily undergraduate institution, so there’s actually lots of professors that are also not only researching their scientific field or their artistic field, but are also researching educational techniques in their field. Part of preparing for that was actually some work that I did last year. I was actually invited to an open access issue from the Entomological Society of America, on educational communication in the sciences …and they had known over the years, because I kept presenting on comics, that my interest really lied in the use of comics as outreach education. So, I began actually searching through the literature because this was something I wanted to continue doing as a professor once I moved to Farmingdale, and it was also something that I just wanted to continue just as someone who was going to keep making educational comics regardless. And so what I found in doing this big review paper called “Sequential Science” is that there is much research in how comics impact the interest and attitudes towards the material, at students at a variety of levels, but there isn’t so much research in actually measuring their gains in content knowledge. So there’s lots of research to show that comics makes students at all levels more interested in the material, but not a lot showing and quantifying how much more they’re learning and retaining. So, I think that’s an area that I actually want to put more research into myself… but yeah I spent a lot of time for that paper reading a bunch of other papers about studies that had been conducted.

John: Have you started this research or is this a plan for future research?

Carly:This is definitely a plan for future. So the development of the OER textbook for gen bio is just happening right now, and anecdotally I’ve certainly seen students are more interested, so I do incorporate comics into my slides right now. They’re not my comics necessarily, they’re comics from a lot of different sources like Beatrice the Biologist or Your Wildlife, those are popular webcomics that are biology focused. I also make some drawings for them for the slides as well. In reality, any comic is just a set of sequential images. So, I can draw a set of sequential images that are explaining mitosis and meiosis. My students might not necessarily read those as comics or recognize them as comics, but they’re still comics because they’re telling an ordered set of events. So when I do that, anecdotally, I can tell you that the students are more interested… especially if there’s just been a slide with the textbook image and some complicated information, if I can show them that slide and then be like “oh let me break it down into these steps that I’ve drawn out” it seems to help them. But have I actually started measuring the impacts? No, not yet.

John: So do you have a research plan on that?

Carly: Yeah, so as the OER textbook is going to take some time to make. It’s probably going to take a couple of years to finish in its entirety, but there’s no reason that I can’t start exposing the students to the chapters as I complete them. So, until the OER is finished in its entirety, and given that I usually teach multiple sections of gen bio, I’m going to start setting up testing control groups just looking at small chapters, as I complete them. So, one class will receive the comics, the other class won’t receive the comics, and since both classes have the same test, I can actually see if there’s any improvement. Now, once the comic is finished in its entirety, that’s when I’ll actually begin the full-scale research… and what’s going to happen there is… again I teach multiple sections of gen bio… I can set up a test group and a control group. The test group will get the comic textbook and then the control group would get a traditional OER (probably the OpenStax gen bio textbook) and I can give them the whole textbook at that point and measure what their differences are in terms of performance using their midterms and their quizzes and their homework assignments. But I also plan on surveying them on interest, because although the interest and the attitudes might not seem as strong a topic as actual performance, I think when you’re teaching non major biology students (many of them who feel like they’re just there to check off a box), many of them who have prevailing biases against science… many of them who don’t feel like they can connect to science… I think it’s so important to measure those attitude changes.

Rebecca: Why does a sequential format work so well for a topic like biology? What do you see the benefit of being sequential in that way? This sequential art form.

Carly: So even in general biology, intro biology for non-majors, there’s still lots of processes that are multiple steps. So, I don’t know if either of you remember learning the Krebs cycle or photosynthesis. These are very complicated multi-step processes where something has to happen and then there’s a result… and then another thing happens and then there’s some sort of result. So, there’s plenty of stuff, even in the entry level biology classes, that lend themselves really well to a narrative. Comics really are any progression of images that build a narrative… now, that narrative doesn’t have to be fiction. The point is that there’s an order of events and together that order of events makes sense. You actually don’t have to add words for it to be considered a comic, but obviously the words help in the context of a biology class. I think given that there are so many multi-step processes whether you’re studying the Krebs cycle… or photosynthesis… or mitosis… or meiosis… or even natural selection or ecology… sequential comics… so these images, where you have processes that are laid out in order and broken down into steps, really help intro students.

John: Do you have an anticipated timespan on your textbook project?

Carly: I suspect that it’s going to be this summer. I probably have two and a half months that I’m not actually teaching, but I’ll also be doing research on my scientific stuff (on my wasp studies) at the same time. I suspect that I’ll be able to draft out the first half of the textbook and probably be able to complete about three to four chapters of it. So, I’ll have those chapters ready for the fall semester and then I’ll try to get some work done during the fall semester and keep building that project. I suspect in total it’s going to take me at least two summers and also the semesters between, where I’m actually doing much more work on sort of my regular school requirements to actually finish it.

John: Do you have any people who’ll be working with you on reviewing this and giving feedback?

Carly: Not yet, but I recognize the need for that. I want to have this textbook be one of the contributions that I have for getting tenure. Making a textbook is a common contribution for the tenure package, but to make a textbook you actually have to have some form of peer review if you’re going to go through a publisher. So, when you’re making your own OER and you’re publishing it on your own website, you might lose some of that aspect of peer review. The plan right now is to actually enlist a set of beta readers who are also science educators in their own field and have criticism from them. This isn’t quite the same as having peer review, but I think for now it’s the very best that I can do, but I’m certainly open to suggestion and open to constructive criticism and changing things up. One of the challenges of creating your own OER is that at some point you might lose the more rigorous aspects of submitting a textbook to a standard publishing company.

Rebecca: Will you have an editor working with you for this project?

Carly: Currently no one is lined up, but that’s a valid suggestion, to actually pay an editor… probably someone who works in science textbooks. But, I think before I can even get to that point, I actually have to have a fairly large body of material to show them in the first place.

John: I would think that one thing that would be useful is, once you have this material, adoptions and response from adopters could be used in place of the peer review.

Carly: Oh yes, certainly. And when I put it up on the website there’s definitely going to be a forum for educators to be like “You know what, this didn’t make a lot of sense. Can you change the wording on this?” So, treating this as a living body of work instead of: “oh, I published that, it’s done…” because there’s no cost associated with changing and the material outside of my own time cost because there’s no physical version. So, it actually wouldn’t be all that difficult for me to have those changes be something that’s constantly happening, especially as we find out better ways to teach say homologous chromosomes, or mitosis, or things like that. But even before I launch it, I still want to have beta readers that can give it a read-through even before that, but having the ability for educators to constantly give me comments would be something that’s on the main website.

Rebecca: What software are you using to manage the process?

Carly: The website build itself is through SquareSpace and that is because I have absolutely no training in making a website, whatsoever. So that’s the actual platform that I’m building the website through. In terms of drawing, I start a lot of stuff out by hand and then I usually draw it in Photoshop on a tablet. Certainly, there are times when my tablet is down and I have to draw it by hand, and then scan it… that’s also a possibility. There is something else I’m interested in and this is more of a conversation about OER versus publishers. On the major publishers textbooks right now… so, McGraw-Hill, Pearson, Cengage, stuff like that… I think they’ve recognized that students can get OERs for free, professors can get OERs for free. So, what these publishers are doing now is that they’re offering adaptive learning systems, where you have assignments that get harder or easier as the student does better or worse, where the grades go automatically into the professor’s LMS which (if you’re at a school that doesn’t have grad students) is great because you don’t have a TA to do your grading. The publishers are offering these adaptive learning systems that go seamlessly into your Blackboard or your Angel (or whatever you’re using), but if you’re developing an OER you don’t have that capability. You can make standard multiple choice quizzes on Blackboard and give them to your students, but that’s not the same thing as an adaptive learning system that tracks your students progress. So, I would also be interested in working with someone (or maybe even SUNY at large) to develop platforms that actually make these adaptive learning systems… because then I think they’ll actually be able to convince more professors to adopt OERs.

John: Some of the publishers do have that. I know that Cengage, Pearson, and McGraw-Hill have been putting together packages of OER materials, where they add other resources to them (including some adaptive learning tools) that they release under a fairly inexpensive license. Another option might be to investigate Lumen Learning. Lumen Learning works with OpenStax and they package OER materials with some other materials they’ve created through a variety of grant-funded activities. But that might be worth doing and SUNY does have a contract with Lumen Learning on these things.

Carly: Yeah, I would like to work with someone that is not just SUNY…

John: Right..

Carly: I’m a SUNY professor but I would like people at the University of California system to be able to use my comics.

John: Lumen Learning is not restricted to SUNY.

Carly: OK.

John: SUNY happens to have a contract where they get a discounted price on the bundles when colleges adopt the Lumen Learning platform, but it’s basically a bundling platform that works with OpenStax and other OER materials.

Carly: Yeah, so that’s worth considering, because not only do I want to make the comic, I also want to make assessment tools… so that whenever professors are using my comic they also have a test bank… a way to create these adaptive learning assignments and things like that. So, this is something I’ve talked about before in my presentations at the Entomological Society of America… that you can’t just make a comic and put it out there for educators, you actually have to provide study tools, study guides, teaching plans, teaching lessons, to actually make it useful for educators.
I really like the idea of there being a platform where a professor could create their own test bank and then assign levels and topics to those questions and then just be able to import those into something that is automatically going to make adaptive learning assignments.

John: I don’t think we’ve got that yet, but there are a couple platforms out there: CogBooks and Acrobatiq. Both are do-it-yourself platforms for creating adaptive learning solutions and based on the Carnegie Mellon system,… which they’ve been doing for quite a while there. But it’s a lot of work, and it automates some of the process so you don’t actually have to do the programming, but you still have to work through most of the structure yourself. I noticed that you give students the option of making their own comics for extra credit. Could you tell us about that? how have students responded? and how has that worked?

Carly: Sure, so this has really come out of a desire to actually start generating and using comics in my class while getting the OER ready… because I have people who are asking me “What results do you have already? How have students responded?” And I’m like, “I haven’t finished the comic yet.” So, I’m aware of that and so that’s where incorporating comics into the classroom right now, while I’m preparing, comes from. General biology is a very difficult course for most incoming freshmen (which is the vast majority of the students I have). What it feels like to me is that I give all of my students the benefit of the doubt… I assume that they’re all studying… and when they do poorly on their first test I don’t say to them “Oh, it’s cause you guys didn’t study enough.” I say to them “No one has taught you how to study.” So a lot of my students, when they do poorly on their exams and they come to me during office hours, I ask them how did you study? And inevitably the answer I get is “I reread the PowerPoint notes, I reread the slides,” and so I’m like “No, no that’s not how you study, that’s just reading”. I try to emphasize that studying is the active reorganization and recontextualization of all of the information sources I’m giving you, not just my PowerPoint slides, but the lecture notes your hopefully taking in class, the textbook itself, the homework assignments. There are all these different forms of information that I’m giving you, and what I’m hoping you’re doing is actively reorganizing it. So, we talk about rewriting your notes. We talk about how to actually make flashcards that are effective. We talk about making flowcharts… and really from that last one… making flowcharts… that’s kind of like making a comic already. With the making comics as an extra credit, I’m really just encouraging to do another form of studying, where they have to take all this material for a midterm and they have to draw their own comic. So, usually what I do is I start the first couple of pages for them. So, on my Twitter right now I can actually send you an image of this first page I’ve made to kickstart their own process. So, spring break is coming up and they have a midterm, not the day after spring break that would be cruel, but the Thursday after spring break. That midterm is going to cover mitosis, meiosis, inheritance, and DNA transcription and translation. And these all seem like different topics but in reality they’re all very interconnected topics. You really can’t talk about mitosis until you can talk about alleles, and genes, and Mendelian inheritance and things like that. So I’m trying to encourage the students to conceptualize that these are all interrelated things because I think that I’ll actually help them memorize things better than just treating them as separate slides that they’re just reading through. At the end of next week’s Thursday lecture, the one right before spring break I’m going to introduce this project and hopefully I get some results from it. Previously I had done this at my last teaching position, which was at Sam Houston State University. I was a visiting assistant professor there, and for extra credit, I offered students the opportunity to make a comic on the same set of materials and I get responses… but the problem is that I get responses usually from the students who don’t need the extra credit. I think this is something that’s a common problem with offering extra credit… that inevitably many, many, many times it’s the students that don’t actually need the extra credit that turn in the extra credit assignments. Now. I still enjoy reading them and they still say that it was helpful and it’s a new study technique that they’re going to do, but reaching out to the other students is one of the challenges I’m facing as a young professor.

John: We all do that, it’s not just related to age. I know in my class I give them lots of chances to retake tests… the people who do it the most are the students who are already doing best in class. So, it increases the variance in the outcomes quite a bit when that’s not entirely the goal, you’d like to have everyone rise up but not necessarily spread out further on that continuum.

Rebecca: So, I’m curious with a project like this, do you use the opportunity as being a scientist who also as an artist to sneak in some art teaching as well? Do you use things like Scott McCloud’s Understanding Comics or anything as a tool to help students understand how to put together a comic and the medium of a comic?

Carly: I love Understanding Comics, it was like one of my foundational books when I was an undergraduate taking my first comic drawing class. I actually tried to avoid situations like that because I don’t want to discourage the students that feel like they don’t know how to draw. Which is a silly thing because everyone can draw… drawing well is a different thing. I don’t want them to get hung up on how good their drawings are. I want them to get hung up on how much conceptual sense that it makes. So certainly Scott McCloud talks about this, about how you can still have a comic that’s just stick figures. And so for me, I don’t want them to freak out about the fact that I’m an artist, and that I’m pretty decent at drawing, and that I expect them also to be pretty decent at drawing. But the funny thing about teaching non-majors is that inevitably some of them are art majors. So, that’s that’s always fun, they’re always surprised to find out when they come to my office and they see that I have paintings that I made as an undergraduate up on my walls and things like that. I would love to refer classic comic making literature, but it’s just something that I don’t have enough time when I’m just spending five minutes to introduce something. But, certainly… the students that come to office hours… we do talk about you know what makes a comic because I also have students that read a lot of comics. I have lots of students that are going on the Manga reading websites and a lot of students that talk about superhero comics with me when they find out that I like comics. So it does come up, but it’s usually not something I have time to make part of my already jam-packed lecture.

John: Students often have this perspective that they’re either creative or they’re good at quantitative skills in STEM fields, and it’s really nice that you’re modeling the possibility that you can be both.. That they’re not mutually exclusive.

Rebecca: That’s also why I like McCloud as a reference book too, because it’s not really about fine art in the traditional sense but rather about how to tell a story. Which is interesting and helpful and doesn’t really necessarily emphasize being able to draw.

Carly: Yeah, I think he has that… what does an expression look like, and it’s just like two dots for eyes, and then eyebrows, and then a line for a mouth, and you can get the full range of human emotion. And then I show students comics like XKCD, that is just stick figures and it’s really effective so, yeah. I try to avoid things where they feel like they have to be a professional artist, not to say that’s what McCloud does, you just pointed out that it doesn’t do that. But I try to focus more on the conceptual – like how does this help you study, you’re not just making this to impress me. And you get that a lot with extra credits, sometimes you feel like students are just doing those projects to get extra credit. Instead I’m trying to be like “Mo, no this is a study tool. This benefits you.”

Rebecca: Have you had any students follow in your footsteps and develop a love of both art and science and pursued you as a mentor?

Carly: At Sam Houston State, I certainly had students that like to come and chat with me and sort of explore those topics. But unfortunately, I had to leave there to start the position I have at Farmingdale, and unfortunately I just haven’t been here long enough to build those connections. One of the things I want to do, as I’m at Farmingdale a bit longer, and I get settled in, is actually propose a biological illustration class. So we have the ability as biology faculty to offer these topics in biology courses, and one of the ones I really want to do is biological illustration… especially since we share our building with art, or rather… I think it’s design communications… whatever the technical college…

Rebecca: Communications design… probably.

Carly: Yeah… but they’re still students taking drawing and watercolor and painting so..

Rebecca: How cool. That would be so fun.

Carly: Yeah, and you know what I actually kind of taught that course at Berkeley. When I was an undergrad at Berkeley, there was this thing where students could actually teach one-credit non-graded courses. So, I actually offered a biological illustration course. Sort of one of those things to build my resume and make up for my not-so-great GPA, but I actually really loved doing it and it seemed like as long as you can get some specimens, and you can sit down, and you have a studio space, you can come up with some amazing work, and luckily I’m still a research associate at the Museum of Natural History, so hopefully they’ll let me borrow some animal mounts. But there’s also insects. Insects are great… they’re cheap and I’m also the entomology professor so it could just become entomological illustration and then of course Farmingdale also has a huge Horticulture Department and botanical illustration has always historically… much like art has been a big part of biology… art has been a big part of botany for a long time. So I think we have the ability to do this, and that there would be interest, and it’d be a cool collaboration with these two departments that are both in Hale Hall.

John: How have your colleagues responded?

Carly: I would say positively… extremely positively. I’ve been thinking more about transitioning into… not fully being a pedagogy researcher… but having it be a large part of what I do on the research side. So, I still plan on doing my usual wasp entomology taxonomy research, but I also want to do a lot of research that’s in comics and the use of comics. That was something that came up in my interview and I think it overall was a helpful thing, and even while I’ve been here I’ve talked about it a lot with my chair and she’s been extremely supportive, and my other colleagues have also been supportive. I haven’t received any negative pushback… which I think was something that I was expecting… because when you look at the literature about educators… whether they or not they want to use comics, there’s this fear… that comics have this bias against them. And so a lot of educators at the primary and secondary levels are kind of afraid of assigning them, and they’re afraid they’re going to be looked down upon by parents and by other educators. But I’ve been extremely fortunate, and I have faced none of that and largely the faculty have been very supportive.

Rebecca: I wonder if some people maybe perceive comics as just being not very rigorous. Which is crazy,because you can provide so much more information… because there’s a visual element as well as a text element. So they might actually be more rigorous.

Carly: Yeah. We talk about lack of rigor and lack of detail in textbooks anyways. If you look at a non major biology textbook it’s obviously not going to be as detailed as a major’s introductory biology textbook, and there’s a reason for that. You’re not teaching people who are going to continue in biology for the most part, so there’s less detail. But, still people harp on the lack of information and the lack of rigor. So, I feel like that’s going to be an argument that comes up no matter what assigned reading you’re going to use. Certainly with comics there’s another bias and that there’s a bunch of superhero comics… but comics are actually a lot more diverse these days.

Rebecca: Comics are probably a really great way to help students understand those basic concepts so that they can build their mental model because they probably come with all sorts of assumptions and things that are not correct, and I could see how demonstrating visually could help overcome some of that.

Carly: Yeah, certainly, and for me it really comes down to what is the point of general biology? What am I aiming to do? I still want my students to learn about photosynthesis, and the Krebs cycle, and mitosis, and meiosis. But I also want them to come away with an appreciation and a sense that they are able to understand it. I want them to walk away from the class with positive feelings towards science and not just- it’s a collection of facts I had to memorize.

John: I wish I had had a class like this when I was in college. it seems like a fascinating way of addressing this

Rebecca: It goes back to what we were talking about earlier. I didn’t know I wanted to know about wasps, but maybe I want to know more now after reading your comic.

Carly: Yeah. So these are all like my lofty aspirations as an educator, but I’m pretty sure I’m still making common mistakes and it’s still a bunch of facts that they have to memorize, sometimes. But I feel an awareness of these of these issues is helping and hopefully I only get better at that process.

John: …and there’s nothing wrong with it being fun for them to learn those facts. ..

Carly: Yeah.

John: … they do need to learn facts but there’s nothing saying it can’t be engaging.

Rebecca: Well, providing those sequences might make it easier to remember, because you have a clearer understanding of how the things connect. The visual representation can help provide those connections that words don’t always help because it’s too abstract.

Carly: I think with biology, especially at the introductory level, especially when you’re a professor that doesn’t have graduate student instructors or TAs, you don’t have a lot of time. So we always talk about wanting to have critical thinking questions and essays, but inevitably just because of time constrictions it does largely become scantron multiple-choice questions, and in that way it does become a lot of memorization. Now I still think that memorization is valid. I still think it’s important to know the steps and the processes and be able to call up that knowledge. But for me, the struggle is making that memorization easier. And if comics make that easier then I’m accomplishing my goal…

John: One of the things that really impressed me, though just following you on Twitter recently since I saw your work, is how engaged you are in the scholarship of learning and teaching in your discipline. It’s nice to see people starting their careers doing that. What got you interested in doing research on teaching and learning?

Carly: I think it actually comes down to who professors are. Professors tend to have PhDs, and in my case, I didn’t take any classes about how to teach. So I think most of us are just kind of thrown into this process and we learned slowly along the way. I was like “Well, there’s a whole body of research out there…” and I started reading some papers about how to be a more effective teacher. We have our own center here for teaching that has workshops and stuff like that, and I think recognizing my lack of formal training, I have no teachers certification or anything like that, made me more interested

Carly: I’ve got the list of questions.

Rebecca: One of the things that I think is kind of interesting is the way that we started this whole conversation… and it ties nicely back to the scholarship of teaching and learning… is that your first research position was doing illustrations. And I think that in academia, we don’t often see those sorts of actions as being research. So I really love that that role was called researcher and brings all this sort of together. It doesn’t have to be traditional to be effective or useful.

Carly: Now certainly that first position as an illustrator in the Padian lab… I still wanted to do traditional types of research, but that experience (as someone who is already sort of hanging around on the graduate student level and hanging around the research labs) made me a person that was visible in a crowd of something like 2,000 undergraduate biology students. So from the Padian lab, I was actually able to transition into a more traditional research role that actually led me to parasitism, to studying parasitism, and that was in the Wake lab with Marvalee Wake, who is one of my most important mentors as an undergrad. But yes, my first research position I was called an undergraduate researcher was actually just doing illustration. And I learned a lot about vertebrate anatomy because that was what Katie Brakora studied.

Rebecca: People don’t realize that when you’re doing that kind of illustration work, what kind of attention to detail you need to pay, and how much you can actually learn by just looking at something very carefully.

Carly: Oh yeah, being able to measure something… getting proportions down correctly. There’s a lot of math that goes into biological illustration and serve a lot of rigor. And then you just spend hours stippling, and that was my life.

Carly: Yeah, I would just say if this sounds like something that a faculty member is listening to this podcast and they’re like, “Ah I want to either start making comics or I want to incorporate comics even into a STEM class, I have lots of resources and I can sort of talk ad nauseam about that. You know like, “What are some good comics if you’re teaching biochemistry? What are some good comics if you’re teaching literature?” So certainly if there’s anyone who’s interested in either making comics or choosing comics for their classroom, I’d be happy to talk to folks.
I think unless you’re a comic book reader you probably don’t realize just how much comics have grown outside of what you might have imagined they were twenty years ago, and you’d be surprised by the amount of some relatable materials… especially in the social studies classes… especially in history, there’s a lot of memoirs… a lot of historical memoirs right now in comics.

John: Actually right now, I can think of at least a couple of examples in economics of comic book series that were created for instructional purposes. The Federal Reserve Bank of New York created a series of comic books to help provide middle- and high-school students with information about the monetary system and the role of the Federal Reserve Board; the other, a series of comic books created featuring Captain Euro… this was originally created to provide support for the introduction of the Euro and for the European Union in general.

Carly: Medicine has really moved with this, especially when they’re thinking about “How do we make information that transcends language barriers?” I follow a Twitter that is just medical graphics and there are conferences on medical comics as well. So I think that’s a field that’s really sort of latched on to making comics as a way to share information with patients, and there’s actually been some research showing that it’s more effective.

Rebecca: It’s used a lot in areas where there might be outreach for really low income or people in poverty who need important information about health or resources and things, and that’s where literacy might be an issue, and so sequential images are often used in those contexts as well. When I was doing a project in India, I discovered all of these really interesting graphics that were used… sequential graphics… to get people to do all sorts of things because there’s so many different languages… to kind of overcome that barrier. It was really interesting.
So we usually wrap up our interviews with the question of what are you gonna do next, you’ve already talked about a number of things that are on the horizon, but is there anything specific you want to share as your your next step, whatever it is that you want to research or do?

Carly: Yeah, so we’ve talked a lot about comics, but I can tell you a couple of other things that are on the horizon for me. My field season… the actual going out and studying wasps that I do that’s going to start up in the summer… and I’m hopefully going to bring an undergraduate or two with me, and then hopefully bring that undergraduate to present at the Entomological Society of America. So, that’s sort of the science side of my life, but sort of the swing back I’ve been talking a lot at the Entomological Society of America about using comics in entomology research… and sort of more in line with what you guys do generally, my next thing is actually proposing a symposium on education for undergraduates. Since most entomologists that are at a university don’t just teach entomology, we also generally teach any biology courses. So, kind of swinging more strictly into undergraduate education instead of the broader community outreach education that I’ve been doing with comics outside of academia. So, that’s exactly next on the horizon for me outside of just keeping working on comics.

Rebecca: So, where do your wasps take you this summer?

Carly: They’re going to take me hopefully to Puerto Rico for about a week, down to Florida for probably a week or two, and also local collecting. There hasn’t been a lot done around the Northeast, so going out to the Pine Barrens on Long Island and then probably making it up as far up as you guys and things like that and further up and down the East Coast.

John: Well if you do get up here let us know

Rebecca: Yeah.

John: … and we’ll take you out to lunch or dinner.

Carly: Oh, thank you. Yeah. This has been great guys, thanks for having me and inviting me to this.

Rebecca: Yeah, thanks for sharing all that you’re doing it sounds really exciting, I can’t wait to see it all happen.

John: It’s great to have you here, and you’re doing some wonderful work.

Rebecca: And you have two fans here and two advocates here.

Carly: Oh thank you, that’s important. I want to like tour all of the centers for teaching and learning excellence, however it’s called at every university, and you know be like “Comics, comics, comics, comics!”

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.