81. Intentional Tech

Some faculty try to use each new educational technology tool they find. Others are reluctant to try any new tools. In this episode, Dr. Derek Bruff joins us to examine how to productively choose educational technology that will support and enhance student learning.

Derek is the director of the Vanderbilt University Center for Teaching and a principal senior lecturer at Vanderbilt Department of Mathematics. He’s the author of Teaching with Classroom Response Systems: Creating Active Learning Environments. His new book Intentional Tech: Principles to Guide the Use of Educational Technology in College Teaching will be available from West Virginia University Press in November 2019. Derek is also a host of the Leading Lines podcast.

Show Notes

Transcript

John: Some faculty try to use each new educational technology tool they find. Others are reluctant to try any new tools. In this episode, we examine how to productively choose educational technology that will support and enhance student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Derek Bruff. Derek is the Director of the Vanderbilt University Center for Teaching and a principal senior lecturer at Vanderbilt Department of Mathematics. He’s the author of Teaching with Classroom Response Systems: Creating Active Learning Environments. His new book Intentional Tech: Principles to Guide the Use of Educational Technology in College Teaching will be available from West Virginia University Press in November 2019. Derek is also a host of the Leading Lines podcast. Welcome, Derek.

Derek: Hi, I’m happy to be here.

John: We’re happy to have you here. Our teas today are…

Rebecca: Do you have anything that you’re drinking Derek?

Derek: So I do, I have some coffee here. [LAUGHTER] I’m not a tea drinker. But there’s a bit of a story. I’m drinking a coffee called Kaldi’s Dog from a local coffee vendor called Bongo Java. And a couple years ago—I work here at the teaching center—we had been serving Folgers coffee in our coffee machine in the break room for several years. And some of us claimed that it was terrible and others of us claimed that people can’t actually tell the difference between coffee brands. And so we actually had a taste test at one of our staff meetings, a blind taste test. [LAUGHTER] From Folgers and several other kind s of fancy coffees and I have to say, I was justified actually. It was very clear that that some coffees were more alike than others. And this was actually the winner, Kaldi’s Dog… the winner of our taste test.

John: So there was no p-hacking or anything going on there? [LAUGHTER]

Derek: No. No.

Rebecca: The nice thing about our tea selection is that we just make hot water and then you can have any of the many varieties that we have in our office.

Derek: That’s true. That’s true.

Rebecca: So speaking of which, what do you have John?

John: Ginger Peach Black tea.

Rebecca: And I have Gold Monkey still.

John: Okay. Mine is nearly empty. [LAUGHTER]

John: We’ve invited you here to talk about your new book. Could you tell us a little bit about the new book and what prompted you to write that?

Derek: Sure. So my work at Vanderbilt involves working with a lot of faculty around their teaching—much like your work—helping them kind of think through the choices they have as teachers, what kind of objectives they have as teachers, and what are some teaching strategies, activities, tools that they could use to try to kind of reach those objectives with their students. It’s a really great job, I get to work with faculty all across the campus, lots of different disciplines. And, in recent years, it’s taken me to other universities as well—and colleges—to kind of talk to faculty there. And so my area of expertise and kind of specialty is around educational technology. And I kind of feel like a lot of faculty come at technology in their teaching from kind of three different areas. Some faculty are told by their administrators that they need to use more technology. And they’re not always sure why. [LAUGHTER] Like what’s it good for? Why do I need this? How can it be helpful? And then kind of at the other end of the spectrum, we have all these faculty who are easily distracted by shiny objects and they see a new technology, and they’re like, “Oh, Pokemon Go, how can I use this in my teaching,” right? [LAUGHTER] And they’re great, these folks are great to work with, they’re all great to work with. But there are also a lot of faculty kind of in the middle who just want to teach well, right? They want to connect with their students, they want their students to succeed, and they want some sensible tools to help them get there. For all three faculty, sometimes they struggle with figuring out how to match technology with learning goals and teaching principles. They know kind of what they want to accomplish, but they’re not sure how to select or use the technology that helps them get there. And so the example I often give in my talks is, I’ll say that my favorite teaching technology is actually wheels on chairs. [LAUGHTER] When I walk into a classroom, right, I have stuff I want to do with my students, I have learning experiences I’ve constructed for them, and I want the furniture in the room to be flexible enough to support what we need to do. Maybe it’s small group activities today, maybe we circle up and have a whole class discussion, maybe we use a debate. I want the technology in the room to support my teaching choices. And so it’s pedagogy first, but then we find tools that help us accomplish those pedagogies. As I’ve talked with faculty at Vanderbilt and elsewhere, I see a lot of patterns in how they use technology and what I’ve done is I’ve tried to distill these patterns down into seven teaching principles, because it’s a book and you have to have seven principles, right? That’s the rule.[LAUGHTER] So seven teaching principles that kind of give you a reason for using technology. And so the intent is to guide faculty to say, “Oh, here’s why I would use technology,” and then each chapter explores one of those principles and has lots of examples of actual teaching practice from faculty in a variety of disciplines. What does it look like in English to use technology to accomplish this goal? What does it look like in biology? What does it look like in communication studies? That kind of thing. I love telling stories and one of the reasons I’m excited to be a part of the Teaching in Higher Education series of West Virginia University Press, it’s edited by Jim Lang—who is a fantastic writer—and he takes this kind of storytelling first-person personal approach to his writing and I was really excited to be a part of this series, because that’s how I like to write too.

John: And you start your book with a chapter on a time for telling, speaking of narratives. Could you tell us a little bit about what the focus of that is?

Derek: This is a little counterintuitive, sometimes for faculty, but it’s really one of the most useful principles I found when working with faculty around designing especially—I mean, to some degree, it works at all scales—but it’s really helpful in kind of a lesson-plan scale, like one day in the classroom. And so I think sometimes there’s this impulse that faculty have to explain the thing, and then have the students do something with it, right? Here’s what it is, here’s the background, here’s the context, here’s the theory, and then let’s have the students do something with that. But the example I give actually comes from my daughter’s preschool. This was 10 years ago now, her preschool had science day and they asked the parents to come in and do sciencey things. And so I was the dad who brought the Mentos and Diet Coke. [LAUGHTER] So, you’ve seen this, right? You take a Diet Coke two-liter and you put some Mentos breath mints in there and then half a second later, you get this huge geyser of Diet Coke. It’s rather dramatic. Mine only got maybe seven-feet tall, but I’ve seen them much higher on YouTube. [LAUGHTER] And so I do this kind of fun thing in front of the five-year olds. And then they ask me “Why did it explode?” And so one could lecture—maybe not to five-year-olds—but you might lecture to a bunch of chemistry students for fifteen minutes on the physics and chemistry behind this and then do the demonstration. Or you could start with the demonstration and have students conjecture. Why is this working? Why does this explode, right? Bless her heart, my daughter asked, “Why is it Diet Coke?” I was like, “That’s a good question.” Why is it diet and not regular Coke? So this is the idea behind time for telling—this is a term from the literature Schwartz and Bransford wrote about this back in the 90s—that if we can create these times for telling with our students where they’re ready to learn and they’re interested in learning, then they’re going to get a lot more out of it, they’ll learn more deeply. We can use technology to do this. One of the stories that I share in the book is a grad student in English at Purdue University, Alisha Karabinus, and she’s teaching a first-year writing composition course and she has her students play this text-based online game. It’s all text, and you’re typing in commands and telling your character where to go. You kind of wake up in this apartment and you’re not sure what’s happening and you have to navigate and walk through the apartment. And there’s this kind of sequence where you need to take a shower and so you walk into the bathroom and you try to take a shower and the game’s like, “You still have your clothes on, you have to take the clothes off.” And then they’re like, “You probably don’t want to walk into the shower with your clothes in your arms.” And so you’d have to put the clothes down and you have to take her watch off, right? Like there’s all these kind of step-by-step things. I’m of a certain age where I played games like this back in the 80s…

John: The old adventure games, yes.

Derek: Right. And so there’s a kind of, you know, interface here that you have to master and you have to learn. It needs to be very explicit about what you’re doing. So she has her students play this game outside of class and then they come in and they debrief the experience. And it’s really lovely because they get so frustrated with the interface, then she makes this nice, clever little pivot where she says, “Well, when you’re writing, when you’re trying to express yourself, you’ve got all these ideas in your head. If you’re not explicit with your reader, they’re not going to know what you’re actually saying.” And so she uses that to talk about transitions and topic sentences and things like this. And so I think it’s a really lovely example of using a little bit of technology that was not at all designed for teaching to give students an experience that then prepares them to learn this lesson about how they communicate and how they write.

Rebecca: Do you have any advice on how to find some of those key little demonstrations or technology pieces that could lead into particular ideas?

Derek: Yeah, I’ve got some more examples in my book. I mean, part of it is that I think—especially for the time for telling—there’s this kind of experiential piece that’s pretty great…

Rebecca: Yeah.

Derek: A lot of faculty will show a video clip. This is one way to kind of do it. The Office is very commonly used to introduce various topics in different courses. I have several examples in the book of games, either video games or board games, and so I think there’s some real value in this experiential piece. And so, there’s no silver bullet. I think a lot of this involves being open to taking something outside your area and bringing it in. In this case, part of it was the interface. It wasn’t the content of the game that was interesting, it was the interface of the game that really helped. And so those are things to look for. Is it the content? Is it the interface? Those are helpful. I also talk about what tools are designed for teaching that can help create this time for telling. And so my first book was all about teaching with classroom response systems—which used to be called clickers—and now in most places students bring their own devices and answer on their phones. But the idea is that you can pose a question to all of your students and they all answer and then you can show the distribution of answers up on the big screen. And if you’ve asked a question that really taps into some type of misconception that students have, and they get the question wrong, the technology is important here, because you want everyone to answer so they all have that experience of grappling with the question. So you need a way to hold all the students accountable for answering in a way to collect all their answers. So you need some tech for that. And then by displaying the distribution of answers and the wrong answers on the board, you let everyone know, “Hey, this is a hard question, right?” It’s not like everyone got this right? You’re split across these two different answers. I share an example in my book of a colleague here at Vanderbilt in the law school, Ed Cheng, and he’d ask a series of questions of his students about Carl and his rhinoceros. So this was a situation that was perhaps prone to disaster when Carl keeps a pet rhinoceros and so he plays out these different scenarios of things going wrong. And then he basically asked these multiple choice questions about who can sue whom for what. And so the first two questions are actually really straightforward, right? The rhinoceros escapes and runs into a car or something, and Carl should have known that was going to happen. And there’s clear cut answers to the first couple of these clicker questions that my colleague asks. But then the third one, it’s a little bit different. And the students, when they respond to the scenario about Carl and his escaped rhinoceros, they’re actually split across three different answer choices. Ed has this great move in class where he’s like, “Well, you’re kind of all right. There are parts of law that are really clear cut and there’s a clear answer, and we just talked about a couple of questions that fell under that category, but we’ve moved into this area where there’s actually not a clear answer and a good lawyer could argue any of these.” It’s critical for his law students to know when they’ve moved into that part of law because that’s when they have to really do the hard work and marshal the resources and make the arguments and work with evidence. And so he’s using this short technology exercise to create this moment where they’re like, “Oh, right, I need to really pay attention here.” It’s that time for telling moment that I think is really lovely and having the bar graph on the screen that has the three-way tie is really important to creating that moment.

Rebecca: I think those are really good examples that I think will help faculty get started.

John: Actually, we did talk about one in an earlier podcast where we had Doug McKee on and he was talking about using this technique in his econometrics class, where you give students a problem that’s just a little bit above what they’ve been working on and it forces them to recognize the need to develop new tools, and then they’re primed to be receptive to a solution if they don’t quite make it all the way there themselves.

Derek: Absolutely. And again, this is a little bit counterintuitive. I think some faculty are hesitant to give their students a problem they know they can’t finish, or they can’t solve, or they haven’t been fully prepared for. But by having that experience, starting class with this hard problem that they can’t quite finish and getting stuck and recognizing the limits in their mental models or their need for additional resources, then they’re ready for the second half of class when the faculty member’s like, “Oh, here’s the resource, here’s the concept, here’s the tool.” And again, very non-intuitive and one of the things I think that’s important about my book is that my focus is on using technology to accomplish these things. But all of these teaching principles are true regardless. You don’t have to use technology to create a time for telling, but it is an interesting and useful way to think about certain types of technology and how you might bring them into your classroom.

Rebecca: I think that’s an important point to hit. The technology is supposed to follow the pedagogy like what you said earlier.

Derek: Yeah.

Rebecca: So remembering that you need to make good teaching choices first, and then finding ways to support.

Derek: Yeah, and sometimes tech is not the answer, right? Or low-tech is a better choice, right? So I have a chapter on knowledge organizations and so this is the idea that when we all organize the information in our head in various ways and so you can kind of imagine in your head like this concept map of ideas and examples and facts, and there are connections between all of them and novices in a domain, right? When our students walk into our class, their knowledge organizations are not as robust, they don’t have as many nodes, they don’t have as many connections, and connections are not as meaningful, and part of our work as instructors is to help them develop more robust knowledge organizations. Well, if we just leave them to their own devices, they’ll do okay, but we can actually help them learn and see the big picture in our course, if we can give them activities that help them develop, construct, represent, and visualize their own knowledge organizations. And so I teach a first-year writing seminar at Vanderbilt—and I talk a little bit about my own teaching in the book, because I think it’s important that I’m using these tools myself and trying to figure out how they work—my first-year writing seminar is on cryptography. Codes and ciphers. And we talk about privacy and surveillance and the role of encryption in today’s society. As part of this, I teach a novel, even though I’m in the math department. [LAUGHTER] It’s not something I was trained to do in grad school, to teach a novel, but I do work a teaching center so I’ve picked up some ideas. But there’s this book called Little Brother by the author Cory Doctorow and it kind of imagines a terrorist incident that happens in San Francisco and then the kind of surveillance and security apparatus that comes after that and the lead character is this teenage hacker who’s kind of fighting against this. And so what I have my students do is they read the book, I have them blog about it in the course blog—so that is a digital technology that they use—but when they come to class, I ask them to get into small groups, I give each group a couple of large Post-it Notes—so these are the kind of five-inch by seven-inch brightly colored Post-it Notes—and some markers. And I say, “Your job in the group is to find two arguments in the book in favor of surveillance and two arguments in the book in favor of privacy.” And so they have to kind of page through the book, they’re looking for arguments made by specific characters in favor of one of those two things. And so the privacy arguments go on Post-it Notes of one color and surveillance arguments go on Post-it Notes of another color. And so then, once they’ve done that piece, the second phase is they in turn go up to the chalkboard and they put their Post-it Notes on the board one at a time, and they have to do two things here. One is they have to put practical arguments towards one end of the board. Like, “If we monitored everyone’s subway movements, are we actually going to catch bad guys?” Like that’s a practical argument. And then have to put principled arguments at the other end of the board. So one of the characters says that, “Hey, it’s about life, liberty, and the pursuit of happiness in that order. If you’re not alive, you can’t be happy, so we got to keep you safe.” That’s a principled argument. And so the students have to put their their Post-it Notes along this axis and then they also have to use the chalk to connect their argument to something already on the board. Because basically, there’s this really complex debate space around safety, and security, and privacy, and surveillance, and I want my students to know how complex that is and to start to see the relationships between some of those arguments and ideas. “This argument is a counter to that argument, or this argument is a support to that argument.” So by the end of class, they’ve constructed this debate map on the chalkboard out of Post-it Notes. And they have, I think, a much better sense of the complexity of this debate. They’ll do more with this. They’ll write about this topic throughout the semester. And so that debate map, that knowledge organization that they’ve constructed collaboratively, can then inform the arguments they make as they take positions within that debate later. And in this case—as I said—this is pretty low-tech, it’s Post-it Notes and chalkboards. I actually tried it once using some software, but having students build this map in a collaborative digital space at the same time was just too chaotic and so I needed to kind of slow the process down, and the Post-it Notes were really great for that. And so this is something I’ve used several times in my course and I think it’s a really great way to help students see how the ideas in an argument space are connected.

Rebecca: One of the things that I’ve liked about using Post-it Notes in some of the kinds of things that I do in my classrooms is that it is easy to change your mind too. You can easily pick it up and move it and it doesn’t seem as intimidating as trying to navigate software to make a decision or something. It somehow lowers a whole bunch of barriers and then it’s a little more flexible and fluid.

Derek: Yeah, absolutely.

John: And you’ve got them thinking about it among multiple dimensions and also making connections, which really, I would think would help them develop a lot of scaffolding there and a lot of connections that they wouldn’t necessarily do with their own reading. So it’s forcing them to develop better close reading skills and analytical skills, and so forth. It’s a wonderful exercise.

One of the principles of learning is that it helps for students to have lots of feedback opportunities and lots of practice, and I see you have a chapter on that. Could you tell us just a little bit about what you focus on there or some of the points that are made in that chapter?

Derek: Sure. So I actually start the chapter of the story about how I learned how to ski a couple of years ago, because learning to ski as an adult is a challenging process, as I found. [LAUGHTER] It involves much falling down. And every time you fall down when you’re learning how to ski, your body is getting a little bit of feedback about what works and what didn’t. And so in a very kind of physical motor skills way, to learn to ski, you have to practice skiing. You fail a lot, your body gets feedback, and then hopefully over time you get better at manipulating your limbs and controlling your muscles so you’re going kind of where you want to go. And so our students need this too, this is actually so key to learning is that we have to practice with the stuff that we’re learning, we have to do stuff with it and we have to get feedback on that practice. It’s a key part of learning. In the chapter, I use this as an opportunity to talk about the so-called flipped classroom because I think there’s a traditional model in some of our disciplines where students get an introduction to information during class. And then after class, they go and do something with it. They do the practice, they have a problem set, right? And the problem is that the practice and feedback part, it’s really important and also really hard. And so to have students do that when they’re left to their own devices, is a lost opportunity actually. And so the flipped classroom model says, “Let’s take some of that activity. Instead of doing it later on your own, let’s do it together collaboratively during class.” And so in the chapter I talk about some ways that some faculty have used technology creatively to help students practice the skills of their discipline during class. I mentioned the classroom response systems as certainly an option for this. I think sometimes when I talked to faculty around technology in the classroom, there’s sometimes an assumption that you’re talking about AV tech. We have a projector, we have some speakers… and that’s great, we need that, that’s helpful. But all my examples involves students using the technology because I think that’s really important. One of my favorites, actually is Kathryn Tomasek from Wheaton College. She’s a historian, and she wanted her students to practice doing the kind of close reading that historians do. When they get a primary-source document, a lot of that reading they do is looking at it line by line, word by word, figuring out who is that person? What is that term? What does it mean? Looking at the very building blocks of this primary-source document, because especially if you’re separated in time by one hundred or two hundred years, you got to do a lot of this close reading to kind of make sense of what it is. And so she had her students work with… she started with historical documents in her library’s special collections and asked her students to do what’s called a TEI. It’s a text-encoding initiative, it’s a way of marking up the text—kind of like HTML a little bit if you know web development—where you’re actually kind of tagging things in the text and labeling them as to what they are. So this is a date, this is a noun, this is a person, this is a location, or in her case, this is a theme that comes out in the sentence. And so her students, they would do this together in class. Like she’d take a piece of it and walk them through it collaboratively on the big screen and then have them take their own pieces and do this markup. And the neat part is the students were actually contributing to a larger digital history project because their markup would be kind of incorporated in this bigger database and shared online. And the work that they did with the primary source documents would then inform the writing and the argumentation that they do later in the semester. But in this case, she wanted to target a very particular skill that’s kind of close reading in history and she found a technology that digital historians use actually pretty regularly to create these opportunities for practice during class to help our students do this kind of work. I share another example of Richard Flagan from Caltech and he was doing chemical engineering. Very different course. But he used little mini projectors to kind of turn his lecture hall into an active learning classroom so his students could work in groups and do some coding—they were doing MATLAB coding in this case—and he found that when he introduced the coding in class and had them work on it after class, they would get hung up on these really small errors around grammar and syntax in the code. So he shifted that work into class to do group works in class and so then he’s able to kind of circulate among them, see what they’re doing on their little projector screens, and intervene and ask questions and help them. And so again, it’s kind of targeting these very particular skills that students need practice with that will inform often bigger projects later in the semester, but creating some time and space in class through technology to give them a chance to practice those skills and get feedback either from each other or from the instructor.

Rebecca: A lot of those examples I think are opportunities for faculty to also see where misconceptions are happening because it’s happening in class soyou can address them one-on-one, but you can also address those bigger themes that bubble up as well as a bigger group rather than having the same conversation 20 plus times.

Derek: Absolutely. Yeah,you may walk over and talk to one student or a small group or you may see a pattern across the students and then kind of take a moment to kind of gather everyone’s attention and try to kind of walk them through together as a whole class.

John: Doing some just in time teaching type of techniques, which is much more efficient use of class time.

Rebecca: That seems really tied to the knowledge organization that you were talking about as well because I think those same kinds of things happen when you’re doing those sorts of activities in class too, right? Like, “Oh, I didn’t realize that you thought this was connected to that,” right? And you can help negotiate that. [LAUGHTER]

Derek: Yeah, and these teaching principles overlap, right? So when I had my students do that debate map activity in class, we were doing practice and feedback. We were taking class time, it was just that this kind of analysis level where they were making connections across topics as opposed to Kathryn’s example of the close reading. That wasn’t necessarily a big picture kind of practice and feedback, it was a very skill focused practice and feedback.

Rebecca: I also really like that these are examples that don’t necessarily make feedback more work for faculty. It’s embedded in the practice in the classroom and it’s just when they need it. And it makes more sense because they’re getting it while they’re doing something so they’re probably more apt to listen to said feedback rather than getting it on an assignment that you hand back and they put it in the garbage or something.

Derek: There’s this book by Walvoord and Anderson called Effective Grading that you may be familiar with and I remember the first time I read about what they called light grading. L-i-g-h-t grading. I thought, “Oh, this makes so much sense. I don’t have to grade everything the students do very rigorously, I could give them a grade on the effort or I could give them a zero, one, or two if the work that they’ve invested in that little piece then shows up later.” So if I have them write a blog post before class to get ready for class, I don’t have to grade that very intensely because we’re going to talk about that material in class and that’s where they get the feedback. I may need the grade it enough to motivate them to do it, but I don’t have to give them detailed feedback at that stage, it will happen during class discussion. And I think that’s kind of freeing for instructors to know I don’t have to grade the heck out of everything. I can kind of design a sequence where students get the feedback they need apart from the grade itself.

Rebecca: I think faculty always appreciate those opportunities. [LAUGHTER]

John: And in the podcast that’ll be coming out a week before this, we talked briefly about specifications grading, which is a variation on the same theme.

Derek: Oh sure.

John: One of your chapters is entitled “Thin Slices of Learning.” Could you tell us a little bit about that?

Derek: I am so glad you asked. [LAUGHTER] Yeah, so this may be my my favorite chapter in the book just because I think the creativity that faculty brought to their use of technology in some of the examples I share, it’s just really amazing. And I also get to quote, one of my mentors a couple of times. Randy Bass is, as I like to say, the Vice Provost of Awesomeness at Georgetown University. [LAUGHTER] That’s not his actual title, but he gets to do some really amazing things there. He’s also really active in the Scholarship of Teaching and Learning world and I think he’s just a really deep thinker about how learning works and so there’s a couple of things that he talks about that I’ve heard him talk about before. He was doing some video projects with his students in an American Studies class a while back and he would look at their finished products—these short videos that they put together as a class assignment—and he realized that he wasn’t seen all of their learning. That—as he said it—there’s a lot of learning that’s left on the cutting room floor. And actually, in the book I talk about how my daughter wrote—she created a short film a couple years ago just because she wanted to—and she filmed, I think three hours of footage for a two-minute film. And so to decide what footage to use, what footage not to use, which angle, which take, even kind of which characters she wanted to include in the final product, there was a ton of decisions that went into those final two minutes. But if you just look at the two minutes, you may not know what those decisions are.Ttechnology, though, can be really good at making visible student learning, and in particular, thin slices of student learning. The kind of choices and sense making that they’re doing in the middle of learning, or creating, or designing, or producing something, and the more we can learn about how our students learn, the more we can kind of get those thin slices of learning in front of us. We can be responsive, we can be helpful, we can guide, and we can direct. And so the examples in this chapter about using technology to make visible (or sometimes audible) kinds of learning that students might not kind of share with us naturally. My all time favorite example of teaching with Twitter is from Margaret Rubega. She is a biology professor at the University of Connecticut. She’s also I think, Connecticut’s State Ornithologist, and she teaches a course on ornithology. So it’s a course on birds and so she has—wait for it—she has her students tweet about tweeters. [LAUGHTER] She’s so articulate about it, like students come to this class—and it’s a fairly large class, I don’t know, 40, 50 students—and they’ve seen really cool birds on National Geographic or YouTube and they think of birds as doing really amazing things in the Amazon and in Africa and far away. And she wants her students to know that birds in rural Connecticut also do really interesting things. Their ecology, their behavior, their biology is all very interesting. And so what she has them do as they’re learning about birds in the course is several times a semester they’re asked to tweet about their observations of birds as they go about their lives. So they’re on their way to work, they’re on the way to school, they see a bird, they see it do something or behave in a way that connects with what they’re learning in class. And she asked them to tweet about it. They have to kind of include their observation and where they are, and they have to connect to the course material, and they’ve got now 280 characters to do that on Twitter. Sometimes they get photos of birds that they see, which Twitter is really good at handling that. The thing I like about it is that it’s leveraging the field observation device that they carry around with them in their pockets—also known as their smartphone—or their regular phone, whatever it is. When they’re in the moment when they see that bird doing something, they’re able to kind of capture that and then share that back with her and with the entire class actually. One of my favorite tweets is a student who was walking by a golf course and he noticed the bird song sounded different in different parts of the golf course and in his tweet he conjectured that the golf course itself was dividing bird territory. [LAUGHTER] I was like, “That’s genius,” right? I don’t know if he’s right, but he’s really paying attention. And what Margaret’s doing is she’s helping her students practice transfer, taking what they learned in the classroom and apply it to real-world situations outside of the classroom and that’s one of the hardest things about learning, is how to transfer learning to new context. And she’s giving her students explicit practice in this, but then also making visible that practice by having this class hashtag on Twitter, #birdclass. And if you go search on Twitter for bird class you’ll see some of the tweets from her students because they’re sharing their observations with her, with each other. I just think it’s a really beautiful use of a very particular technology for a very particular reason. I love the bird class example.

Rebecca: That’s really fun.

John: Yeah, that’s a great example.

Rebecca: I couldn’t help but think that if my two-year-old could tweet, she’d be really into it right now. [LAUGHTER]

Derek: Does she have a lot of observations about life that she tries to share?

Rebecca: A lot about birds lately. [LAUGHTER]

Derek: But you think about all the learning that students do when they’re not with us that may be really important. I have another example from Mark Sample at Davidson College, where he had his students live tweet the film they were watching in his sci-fi class. They watched it on their own time in their own rooms or whatever, but it was Blade Runner and they would live tweet their observations about the film. And it’s one thing if we have our students read something or watch something and then write a response paper and bring that to class and then we discuss it, right? That’s great. But he was getting kind of a next level down. They are kind of immediate in the moment reactions to what they were seeing in the film and kind of surfacing that and making that visible. And this is a great use of technology. There’s other ways to do this but technology can be really good at this kind of thin slices of learning piece and that’s one of the connections I want folks to make in the book is that, if you think about it, “Hey, Twitter seems really useful. What can we do with Twitter and our teaching?” Well, there’s a lot of things you can do. But one of those is to surface thin slices of student learning. And that provides some focus for thinking about how you might use Twitter in your class for a very particular purpose.

Rebecca: Sounds really fun.

John: It does, and you can see this in other areas of biology or botany. I think Michelle Miller was on a while back and she talked about a class where students went out in the field to identify plants and tweet back photos and so forth.

Derek: Yeah, and I shared this with a grad student here in civil engineering and she has her students tweet about structural things they see in the built environment that connect to the material that they’re learning. I’ve ran into teacher educators who have their students—while respecting some privacy—but they’ll tweet about what they see in the field when they’re in classrooms and they’re observing teachers in action. They’ll tweet those observations and so yeah, I think there’s a lot of different uses for this kind of application.

Rebecca: This also moves a little bit into the idea of learning communities, because this is a community practice using a hashtag where you’re kind of seeing things outside, but you also have a whole chapter on just learning communities.

Derek: Yeah, and I mentioned the bird class, because there’s a lot of things going on with the bird class piece. And part of it is that yes, by making the students tweets visible to the other students, you have this other dynamic going on which is that the students are starting to learn from and with each other. And students can learn a lot from their professors certainly—and we have a lot of expertise and authority that we that we use in the classroom—but if you think about the places where you learn naturally when you’re picking up a new hobby, or you’ve got some interest of yours that you want to pursue, you’re often connecting with other people who do that too. So like, I have a friend who just went to a quilting conference in Nashville because she loves to quilt and she’s going to connect with other quilters. And that’s how she learns how to quilt, it’s this kind of peer-to-peer learning that she does. And so we can leverage that in the classroom. It’s always a little authentic. Students in a statistics class aren’t going to just get super passionate about statistics and learn from each other necessarily, they’re going to bring their own levels of intrinsic motivation to this. But if you think about all of the different perspectives and experiences that you have in the room with your students, they have a lot to bring and they can actually learn a lot from each other and you can learn a lot from your students. But you’ve got to create some mechanisms for that, it’s not necessarily going to happen naturally. And so bird class is a great example because as the students, I mean, they’re all in Connecticut, right? But other than that, they’re going to different places, they’re seeing different parts of town, then they go to different locations for spring break, and see different birds, and so they’re all bringing their kind of different perspective on this. And by making that visible, they can start to learn from each other. In my chapter I talk about, I use a social bookmarking tool called Diigo, which allows basically students to share links with each other. And so we create a group for a class and I give them these assignments, in my cryptography class especially. So, find an example of cryptography in the news or find an example of military cryptography or let’s find out something about the National Security Agency. And so what’s really cool is—especially for a course like this—students bring a lot of different interests into this topic. And so I’ll have students who have kind of like a literature interest—I had a Sherlock Holmes buff in the class once and so she was always finding really interesting examples of cryptography in literature to share—I had students who were always interested in kind of cybersecurity and computer science and so they’re bringing in kind of modern news and technology. I had one student—bless her heart—she loved Russia, she found a way to find a Russian connection to everything that she did. And so it was really great, right, because she found all these examples of cryptography, especially kind of Cold War espionage stuff that we wouldn’t have seen if she didn’t have this passion for Russia and then found resources and shared them with the class. And so by having students share these resources, in a shared space and then talk talking about them in class, I can really leverage the fact that we’ve got a number of individuals in this room that have different experiences and perspectives and if we can make advantage of that, we can actually all learn more deeply.

Rebecca: I’ve used Slack in my web design classes to do troubleshooting and technical help but I use the same Slack channel across semesters. And so what I found is that people who have graduated who are out in the field will sometimes randomly pop in and answer questions, and it’s really cool, but I remind the students that I’m not the only one that can answer questions. They can help each other out. But sometimes—you never know—some other lurker might pop in and help out. And they have some sort of vested interest, you know, because they were also in that class at one time.

Derek: Absolutely, and that’s one of the advantages. One thing that can happen when you shift away from a course management system, course management systems are good for a lot of things but they’re not good for semester to semester continuity. They kind of put courses in little boxes by semester and the students can’t get out of those boxes. And so once you move to Slack or social bookmarking—like my Diigo group, we’ve been doing it for like seven years. We’ve got hundreds of resources collected by students over time. Course blogs are really good for that too—and so that’s really exciting when you can use some technology to make some student work public and persistent in a way that invites future or past students to participate. It’s still a learning community, it’s just expanding beyond the time and space of this one particular course offering

John: One of the issues with learning management systems is—as Robin DeRosa and other people have called it—is that the assignments often take on the nature of a disposable assignment, that they’ve done the work and then at the end of the semester, they even lose access to it unless they keep it outside. And there’s a lot of advantages to having a sort of persistent work that you’re describing there.

Rebecca: It seems like you’re moving right into another chapter of Derek’s book on “Authentic Audiences.”

Derek: I mean this is the other thing—and again, the book is not a critique of course management systems—but I will say course management systems also make it hard for student work to escape, to be seen by anyone not in the course, and often that’s appropriate. When students are first learning a topic or a discipline, they need a private space to practice, and screw up, and say dumb things, and get feedback, and get better. And that’s true for the assignments, right? Sometimes we have students turn in an assignment to us and we’re the only one who looks at it because they’re still learning the skill set and they need some good practice on that. But when we have students construct work or produce work for authentic audiences outside of the course, that can be hugely motivating for students. Hugely motivating. I’ll quote Randy Bass again, he’s got this white paper where he coins this term social pedagogies. These are pedagogies where we’re asking students to construct their knowledge by representing that knowledge for an authentic audience other than the instructor, and it could just be each other. That can be really powerful as well. But when students see that the work that they’re doing is not disposable, it’s not going to be gone. Students often do write a paper and there’s only one human being on the planet who ever looks at it, but if you can build toward some assignments where students are writing or constructing or producing for an external audience or an authentic audience, there’s a lot of motivational benefits to this for students and they start to take their work very seriously and invest in it in ways that they don’t sometimes in the disposable assignments. One of my favorite examples, Jonathan Rattner teaches cinema and media arts here at Vanderbilt and he had connected with a colleague of his from grad school who was teaching a writing course, Bridget Draxler, she was at another institution. Jonathan was teaching students how to create short experimental films and they needed an audience to share that work with. Bridget was teaching her students to critique media and she wanted her students to find media to critique where they could interact with the creator, and so they just set up a course blog for the two of them, these two courses. It wasn’t public to the world but Jonathan’s students were creating media for her students and her students were critiquing it and then they would have this conversation. And this idea of connecting your course, with just one other course—somewhere else on your campus or maybe at another institution where you have colleagues working—all of a sudden, you have this really authentic audience for the work. And in this case, this was really intentional too, this wasn’t just a random pairing “We want to share stuff with someone,” but there was this kind of synthesis that worked well across the two courses. But that’s a fairly easy way to create some authentic audiences for your students. I also talked about Tim Foster who used to teach Spanish here—he’s out at one of the University of Texas schools now—and he had his students write for Wikipedia. This is actually becoming increasingly common in higher education where you have students write for Wikipedia. There’s certain standards that you have to follow and it’s kind of hard to get content to stick on Wikipedia because of that. He was actually teaching an introduction to Portuguese course—so this was first semester Portuguese language learners—and what they realized was that the Portuguese language page for Nashville on Wikipedia was very skimpy. And what his students didn’t know at first is that Portuguese Wikipedia is not just a translation of English Wikipedia. Portuguese speakers create their own Wikipedia. And so the national page was kind of skimpy. So as a class project, he had his students create content for the Portuguese language Wikipedia page for the city where we are. And so it was great as a language production task for them because they could focus on writing two or three sentences, first semester language learners, but they knew that actual people are going to look at this so they took it very seriously. Some of them went above and beyond. I think this is just a really powerful pedagogy. And again, you don’t have to use technology but this is something technology is actually good at, is connecting people across time and space. Having students use some technology to create something for an authentic audience can be really powerful.

Rebecca: I think you have one last chapter that we didn’t quite get to yet and that’s…

John: “Multimodal Assignments.”

Rebecca: Which, you know, technology is also really good at that whole multimodal thing, right? [LAUGHTER]

Derek: Sure. I was so close to calling this chapter “Beyond the Five-Page Paper.” [LAUGHTER] Because again, the five-page paper has its place where students write a thing, and it’s just text, and they give it to the professor, and they get feedback. Again, there’s a lot of practice and feedback that happens in activities like that. But there’s a lot of research that says, not so much that learning styles exist. The research actually doesn’t support this idea that I need to match my teaching modality to my students learning preference. So the matching hypothesis would say that I have some visual learners and some verbal learners and some kinesthetic learners and so I should do visual stuff with the visual learners and verbal stuff with the verbal learners and kinesthetic stuff with the kinesthetic learners. There’s no research that supports that actually. So there’s kind of this learning styles myth that I like to debunk when I can but where the research does support is multimodal learning. Now, we all learn better when we encounter stuff in multiple ways. And so I think this is the reason the learning styles feels so compelling to a lot of instructors, is that if they’re doing that, if they’re thinking about their lesson plan and saying, “Oh, I got to have some visual stuff, I got to have some verbal stuff, I got to have some activities.” It’s not that they match those with individual students, it’s that all students are benefiting from those three different modalities happening in the same classroom. This chapter is all about multimodal assignments, ways to tap into this dual coding that our brains do where we take in information in verbal ways and in visual ways and kind of put that together and it’s stronger. We’ve done a lot of work at Vanderbilt. We call it students as producers. This is kind of a course design and assignment design approach that we work a lot with here through our course design institute and elsewhere. It’s helping faculty move away from some of those traditional text-only assignments and moving to more open-ended assignments, more multimodal projects, student projects that have authentic audiences. And so actually, this chapter is kind of all Vanderbilt examples which sounds a little self-serving, but I just happen to know a lot of faculty here who are experimenting a lot in this area. I share an example for my own classroom about infographics in a stats class where I’m asking students to represent quantitative information visually. There’s an English grad student here, Kylie Korsnack, who has her students take a paper they wrote, and revise it in a different medium. So it starts off as a traditional paper but they have to revise it as a Prezi or a concept map or choose your own adventure novel, or one student did a Pinterest pin board. And so by kind of re-seeing their work, moving into this other sort of medium, the students are often able to see their work in new ways and realize, “Oh, that’s what my argument really was,” or, “Oh, my transitions are terrible. Now I know how things have to be connected.” And so there’s a lot of value in having students move into different media than straight text as a way to help them make sense of things. I’ve been experimenting a lot with podcasting. So I got this idea from Gilbert Gonzales, a colleague of mine here in health policy, who had his students create podcast episodes instead of research papers. And he really wanted them to be fluent with the language of healthcare policy. HMOs and PPOs and all this kind of stuff. And so an audio assignment made a lot of sense, actually, for the students. And he founded it with a lot more fun to listen to a few podcasts than grade a few papers [LAUGHTER]. And podcasting is a low bar, right? Not to say that what we’re doing isn’t super challenging here, but you can, you know, create a pretty decent podcast with your phone, right? It’s not going to be super high quality, but you can record and you can edit using some free software and so I now have my students do a podcast assignment in my cryptography course. And with about 25 minutes of in classroom technical training, they’re able to produce some interesting things and then we can focus on “How do you tell a story through audio?” Or in my class, how do you explain this kind of technical mathematical stuff that they’re studying through audio only without pictures? And so, again, all of these are about kind of moving to different modalities and shifting between modalities to help students see and understand the material in different ways. And if you keep doing that, they’ll start to kind of triangulate and, and make a lot more sense out of it.

John: I would think it would force them to think about it a bit more deeply to make connections that they might not otherwise. Just seeing things from a different perspective seems to have a lot of value in it.

Derek: Yeah. I would also add that when you move to a non-traditional assignment—this is something that I realized kind of late in writing the book—is that we asked you to do a podcast they walk in and they don’t know how to do podcasts. So Gilbert and I were like, “Okay, so we have to listen to some podcasts together and critique them, and then maybe come up with a rubric together, and they need to outline it and maybe even turning the script and get it approved.” We have this whole scaffolding process around preparing students to do this type of work. Well, some of our students come in and they don’t know how to write a five-page paper either. We assume they do, we assume they’re good at it, that they’ve learned that in high school or something. But for some of these traditional assignments, we have students who really struggle and so one of the things that non-traditional assignments do for faculty is help them realize, “Oh, I really have to get in the head of my students and figure out what’s the scaffolding they need.” And we really should be applying that to more traditional assignments as well, because a lot of our students struggle because we don’t have them submit a proposal, or get feedback on a rough draft, or practice how to find a credible source. These are all things that we that it’s easy to assume our students can do, but they can’t always actually do that. And so moving to a non-traditional assignment often then helps faculty move back to more traditional assignments with a new lens with greater intentionality.

Rebecca: So we have to wait until November to get this book? [LAUGHTER]

John: Yeah, I know I saw when you posted this on Twitter, I said “I’d like this now.” [LAUGHTER]

Derek: Well thank you, yes. The book production process is a long timeline as I’ve found.

John: Yeah.

Rebecca: It’s a good tease though, right? [LAUGHTER] So you were also just talking about how you’ve been experimenting with podcasts and you’ve been the host of Leading Lines since 2016. How did you get interested in doing all this podcasting stuff in the first place?

Derek: Part of it was that—at the time—I had a 45 minute commute to work so I was listening to a lot of podcasts and appreciating podcasts and wishing I had more podcasts like Tea for Teaching that talk about teaching, learning, and higher education. And so that was part of it. I think, also, I was involved in a pretty big online course project that involved a ton of video work. And I saw how powerful that was, but how much work it was to put together really high-quality video and I thought, “What if we had a podcast on educational technology?” There’s folks that I run into here on campus and elsewhere who are doing really cool things and I would just love to kind of give them a bigger audience for the innovative teaching that they’re doing, and producing a podcast seems way more tractable than producing a YouTube series. [LAUGHTER] So I mentioned this at a meeting here, we were having this meeting on campus with some other folks who deal with educational technology. One of our Associate Provosts, John Sloop, for digital learning, he’s like, “I would love to do a podcast.” We kind of both had been thinking about this idea for a while and so we combined forces. And so it’s the Center for Teaching, it’s our libraries, it’s our Institute for Digital Learning. We created Leading Lines, we’ve been doing it for a few years now, each episode is an interview with a faculty, staff, or grad student who’s kind of doing something interesting in the educational technology space. We call it Leading Lines because it has this kind of connotation of looking into the future. So leading lines in a photograph, are those kind of straight lines that draw your eye into the frame. And so we’re not really trying to predict the future—because I think that’s a fool’s errand—but I’d rather kind of shape it and influence it and so we’re looking for folks who are doing things that are kind of one or two steps down the road with technology. And it’s been really great, I mean several of the examples in my book are drawn from interviews I did for Leading Lines. It just gives me this occasion to talk to interesting people who are doing interesting things.

Rebecca: Now you know our secret. [LAUGHTER]

Derek: Right.

John: We’ve gotten this opportunity to talk to all these people doing some wonderful research that we wouldn’t be able to be in contact with so many of them otherwise.

Derek: Yeah, absolutely. I’m always referencing these people that I’ve met and the work that they’re doing and my other work here, connecting faculty, and it’s been a lot of fun. And it’s been fun to work here. So we have about six of us who do interviews for Leading Lines and so we have a little bigger team then you guys have and they go in different directions. Sometimes I’m like, “That’s really not an interesting topic,” and then they do an interview and it’s an interesting topic. And so it’s been really fun to kind of work with my colleagues here and having this collaborative project across multiple units at Vanderbilt, that’s been pretty great too.

Rebecca: Cool.

John: On one of the recent episodes, you had a discussion of the VandyVox project. And in particular you had a podcast from there. Could you tell us what this project is and how that came about?

Derek: So I had this idea actually just last summer. We were running a course design institute here at the teaching center and we had several faculty who are really interested in doing podcast projects because I think I had shared Gilbert Gonzales’s Health Policy Radio podcasts with them. And several faculty started thinking, “Oh I could really use this in my teaching,” and I thought, “This is great, but if we have Vanderbilt undergrads, especially, who are producing really interesting audio for class assignments all over campus, wouldn’t it be fun to curate that to have a podcast of podcasts?” So then I reached out to my colleagues at Vanderbilt student media and they’re like, “Yeah, that sounds great. We love to help students make media and share media with the world.” And so they were able to do all the heavy lifting on the technology piece, all I had to do was reach out to some faculty members and ask them to recommend some student produced audio for this and so this spring we launched VandyVox. It’s the best of student produced audio from all over campus. It’s a bit of a fudge, right? Like if some student has a Sports Radio Podcast, we’re not covering that. But if there’s an academic component to it, if it’s curricular or co-curricular, we’re happy to feature it on the podcast. And so it kind of serves two purposes. One is to kind of shine a spotlight on some student work, show this great stuff that our students are doing to provide some inspiration maybe for faculty and students to have students engaged in this kind of work. In the show notes for each episode, we have some background information about what the assignment was, or how the faculty members worked with students around this. So there’s a faculty development piece to it as well. But it’s been really fun to see what students are doing all over campus. You know, I highlighted some Health Policy Radio piece, we had a student from an anthropology course on health care politics. She created a 10 minute speculative fiction audio story…

Rebecca: Oh, cool.

Derek: …dramatized it as her project where she kind of imagined what would happen in the future with gene editing and baby selection. It’s just a really great sci fi kind of look at the course topic. Well researched, right? Like she turned in an annotated bibliography with all this so it’s all kind of backed up by the latest research. We had law students who were doing podcast episodes on immigration and refugee law talking to some immigrants and refugees. For that audio. We have Robbie Spivey in our Women’s and Gender Studies class teaches a course called Women Who Kill [LAUGHTER]—which is a great name for a course—and so she had her students do kind of true-crime podcasts about women who kill and how we make sense of that as a society. And then our last episode of season one, which came out recently, featured some work by Anna Butrico, who was a senior here last year, an English major. She did her senior honors thesis on podcasting and kind of connected it to ancient Greek rhetorical forms, which is really great. But her senior thesis had audio pieces to it. It’s hard to do a senior thesis on podcasting without creating a podcast and so we featured the audio introduction to her senior thesis, which I was really excited because Anna actually did a lot of work with podcasting while she was at Vanderbilt and her technical skills and her composition and storytelling skills are really strong. So it’s been really fun to kind of see something of a critical mass here at Vanderbilt around student podcasts and to be able to kind of highlight that a little bit. And I’m really excited, we’ve got some good stuff lined up for season two this fall as I’m kind of reaching out to more faculty and students about the the audio work that they’re doing. And again, part of it is getting started with a podcast is not hard. Doing it really well is still very hard, but the bar for entry is pretty low actually. And so if you want to have your students kind of move into a different modality—and again, you need to kind of be intentional about why you’re doing it and how it connects to your course goals—but podcasts offer a really great option for that. And I’ve just seening more and more faculty start to embrace this as a kind of creative output for students.

John: Going back a little bit, you mentioned that video project or the the video intensive project, I’m assuming those are the two MOOCs you have on teaching in STEM courses. I participated in the first one when it first came out.

Derek: Oh, that’s great.

John: It was a lot of fun, it was really useful. I didn’t do the second one. I think we both recommend those to a lot of faculty and encourage more people to take those. I believe they’re still running on Coursera?

Derek: Oh, yeah, we’re running one every semester. They’re not on Coursera anymore, they’re on edX. But you can always go to stemteachingcourse.org and you can find out information about those courses.

Rebecca: So you’ve already talked about the podcast that you’re working on and your book, the editing process and such that takes a long time, so you’ve got a lot of things in the cooker but we always wrap up by asking, what’s next?

Derek: Short run, we’re running a couple of course design institutes in the first of May and so that will occupy the next several weeks as kind of prep for that and those are always fun because I get to work with faculty. It’s on the theme of students as producers so we’ll be working with faculty around these creative nontraditional assignments. That’s always pretty exciting. There’s also—this is just an idea right now—but I keep running into faculty who are teaching with games or having their students design games, board games especially, as course assignments. I mentioned this text-based game that Alisha Karabinus uses and so I just keep finding examples of games and simulations that have a learning goal or learning purpose and so I’m hoping maybe this fall to put together a little one day symposium on campus on games for learning, games for social change, that kind of thing. I think that’d be a fun space to explore. And the other thing that I’m seeing—and I talked a little bit about this in the book—is this move towards active learning classrooms. I mentioned I like to walk into a classroom and see wheels on the chairs, because we can move them around and make them do what we want. The affordances that our classrooms have really matter for the choices we make as teachers. And so classrooms that are designed to facilitate small group work, student collaboration, active learning, this is a strong trend in higher education. I’m really kind of shocked how even from like three years ago, where I was having to tell people about this idea for the first time, now and here on our campus, our campus planners have decided this is a standard classroom configuration going forward. And so I see a lot of campuses moving towards active learning classrooms. Again, digital and analog technologies that support learning and so I want faculty to use them in intentional ways. And so I think we’re going to be doing a lot more with active learning classrooms here on campus, probably starting a learning community on that in the fall and I’m excited to dig into that work too.

Rebecca: Sounds like a lot of exciting things going on.

Derek: I try to stay busy. [LAUGHTER]

Rebecca: It’s going to be hard to keep up with all of them.

John: Well, we appreciate that and we’re looking forward to the book coming out.

Derek: Awesome.

Rebecca: Well thanks so much for joining us. It’s been really exciting and I know we all have a countdown now.

Derek: Thanks so much for having me on. This has been a really fun conversation. I’m happy to get the chance to share a little bit.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

80. Self-Regulated Learning

Most students arrive at college with serious misconceptions about effective learning strategies. In this episode, Dr. Linda Nilson joins us to examine what we as faculty can do to help students develop their metacognitive skills and become self-regulated learners.

Dr. Nilson is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning. In this episode we focus on discussing one of her books: Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-awareness and Learning Skills.

Show Notes

  • Linda Nilson—Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2013). Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-awareness and Learning Skills. Stylus Publishing, LLC..
  • Nilson, L. (2014). Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time. Stylus Publishing, LLC.
  • Professional and Organizational Development (POD)—Network in Higher Education Conference
  • Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77(6), 1121.

Transcript

This transcript has been edited to improve readability.

Rebecca: Most students arrive at college with serious misconceptions about effective learning strategies. In this episode, we examine what we as faculty can do to help students develop their metacognitive skills and become self-regulated learners.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson, the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. Dr. Nilson is the author of many superb books, book chapters, and articles on teaching and learning. Welcome.

Linda: I’m very honored to be here. Thank you.

Rebecca: Today’s teas are…

John: Are you drinking any tea?

Linda: Yes, I am drinking tea. I am drinking Lemon Lift.

Rebecca: Oh that sounds like a great way to start the day.

Linda: It is. It’s a very good way. Well, I also started it with coffee, but… [LAUGHTER]

John: And I’m drinking Ginger Peach Black tea.

Rebecca: And I have my Golden Monkey tea today.

John: We’ve invited you here today to talk about your book, Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-Awareness and Learning Skills. Could you define what it means to be a self-regulated learner?

Linda: Yes. Self-regulated learning is the conscious planning, monitoring, and evaluation of one’s learning for the purpose of maximizing it. That’s a very fancy way of putting it. It’s that voice in your head that asks you questions about your learning as you’re involved in some sort of learning task, questions like, “Okay, I’m going to do a reading now, what strategy works best for me?” Now you just might brush over that because you’ve done readings of this type a dozen times, a hundred times, whatever, but you’ve asked yourself that question along the way. “What’s my best strategy? What kind of a task is this? And monitoring: are my strategies working for me? Am I getting it? Can I paraphrase the last couple of paragraphs that I just read?” It’s a reading thing, but it works also in lecture. And then at the end, you evaluate yourself. “Well, let’s see. I had a goal, being able to recite five main points from this chapter, let’s see if I can do it,” without looking at the chapter of course. [LAUGHTER] So you evaluate your abilities, you evaluate your strategies. That’s really what it’s all about and it involves a great deal of talking to yourself.

Rebecca: So how did you get interested in talking to yourself? [LAUGHTER]

Linda: Well, I heard voices. [LAUGHTER] Well, how did I get interested in this topic? Actually, it was an accounting professor at Clemson that got me interested in it. This is 2007, right, this is a long time ago. And so she said to me, “What about giving a workshop on self-regulated learning?” In my head I said, “Huh? What’s that?” I’d never heard of it. And so I decided to go find out about it and it took me a few years to really get a workshop together on it and I decided, “Gosh, this is wonderful. This is learning how to learn. This is familiar to me,” because I’ve been talking to myself for years. [LAUGHTER] So I thought, “Okay, I’m not crazy. This is a learning strategy, a major learning strategy, one that you can use throughout your life.” And so I gave the workshop, I started giving workshops, like at the POD Network Conference—which is made up by people like me who go to this conference every year—and then I decided—well, I didn’t decide—a book publisher came up to me and said, “Please write a book on this, I will publish it.” Since I was in love with the topic anyway, I decided to do it. And so I did and delved into it deeply.

John: As you’ve described it, it sounds like part of this deals with improving student metacognition, but you note that it goes a bit further. Could you talk about the additional aspects of it?

Linda: Metacognition is the cognitive part of self-regulated learning, which is a major part of it. However, there are a couple of other elements to it that I don’t know that you could say are really focused on cognition. There is the emotional element to it, which involves getting yourself to be motivated and interested in the topic. Remembering, reviewing what your professor told you about the relevance of this topic, and thinking about it yourself. We can motivate ourselves, we can reframe a task for ourselves, and we can certainly reframe what is going on in terms of a learning experience. That’s a major, major part of it. The emotional part of the end is: “If you didn’t reach your goal, what do you do about it?” Do you give up, walk away, and say, “Well, I wasn’t born to do engineering,” or whatever the topic is. No, what you should say is, “Let’s try another strategy. Let’s look into possible strategies.” As instructors, we need to familiarize students with various strategies because they come to us—I like the phrase—“as feral children” in terms of the life of the mind and what they know about learning. We don’t have cognitive psychologists—unfortunately—teaching first grade or fifth grade, and so we need to equip them with how their mind works. There is one other element, a physical element, and that involves planning, monitoring, and evaluating your physical setting, where do you study best. If doing a reading or writing assignment, is it in a coffee shop, or do you have to essentially be in a soundproof booth where you don’t have any stimulation? How much coffee should you have? Or tea? [LAUGHTER] What kind of an environment should you set up for yourself—perhaps putting your digital distractions in another room. How should you schedule your breaks? Other things that you might want to consider is the amount of sleep that you have had because that can be a very important element of learning and writing. Some people study better to mild background noises as long as they are familiar music they’ve never heard before. You’ve got to try out these different things and find out your best setting.

John: In your book you describe how you became a self-regulated learner. Could you relay that story?

Linda: Yes, it was based on fear and terror. [LAUGHTER] As a child, I went to a private Catholic girls school—great education, but not in the sweetest of ways. From about fifth grade on, we had what was called recitation every single day in English and history classes. The nuns would ask a question and would randomly call out students’ names—we were in small classes so it wasn’t an absurd thing—and we had to get up and we’d better have the answer to the question. Now, not all the kids did, but I needed to be Little Miss Perfect because I wanted to get into college. Somehow I thought that the answers that I gave in fifth and sixth and seventh and eighth grade would or would not get me into college. So I learned to quiz myself while I was reading chapters or essays. Constantly quizzing myself. And another thing we had to do, usually later in the day, was called “exercise period.” We had 35 minutes to write an answer to an essay question that was generally related to the readings. And we couldn’t look at our books. It was out of desperation that I was trying different strategies so I could perform very well every day, and I was really quite successful. So when I was reading about self-regulated learning, I said, “My God, I was doing this as a child.” That’s why it sounded so familiar to me.

John: But that’s not the experience I think that the students we have entering our colleges have.

Linda: You are so right. You are so right. And we don’t do this to children anymore, okay? There is a good side to it—there really is—because you do buckle under and get very serious about your homework, very serious about your studies, or you look like an idiot the next day. Anyway, the sport I’m trying to think of is that sport where you get the ice or any kind of ice particles out of the way.…

Rebecca: Curling?

Linda: Curling! Curling, yes, we are curling teachers and curling parents. We try to clear the way for our students. We don’t want to put them in stressful situations. We don’t want to ask Johnny to read if Johnny might not be able to read—and I mean, get up and read well—and the problem with this is that students are denied the opportunity for achievement. And there is no achievement without the possibility of failure, there just isn’t. So students have no idea what fear and terror in school might be. There’s bullying and all that, but I mean from the learning experience. So no, they don’t have any kind of the experiences that I had.

Rebecca: How do we start coaching students then to become self-regulated learners if they’re coming out of this really different environment that’s much more supportive and doesn’t allow for failure seemingly?

Linda: They start failing in college. We’re still sort of, you know, curling them a little bit, but they are really facing a much greater challenge. They get insecure really quickly because they’ve been told how special they are and how smart they are, and then they begin to question that, because they’re not doing as well as they were in high school, where you could get an A relatively easily. Now, “Oh my, it can be really hard,” and then they start getting C’s, and then you have their attention. That’s a way that you can tell them that, “There are ways that you can get A’s, you did not learn how to study, here is a way to learn.” Does it involves a sort of effort? Sure, but it’s really just talking to yourself and deciding what strategies would be best for you, testing out strategies, seeing how they work, and you will be more successful. And there have been studies of students—like in developmental courses—that show that the students who are struggling the most tend to know the least about self-regulated learning strategies and start to do better if they use these strategies.Of course we’ve got to get them to use the strategies, we’ve got to explain these strategies. It can be life changing for them in the most positive way.

John: I think part of the issue is that faculty generally haven’t been taught these strategies themselves. They somehow found ways to be successful, so they become self-regulated learners, but faculty are the exceptions. They’re not the typical student, and they’ve never really been trained to teach students how to become more effective learners, in part because they never learned that directly themselves quite often. What can faculty do to be more effective in this way?

Linda: Well first of all, faculty have to realize that they’re the weird ones, and everybody else is normal. [LAUGHTER] So we have to stop projecting our learning abilities, our strategies, our interests in the life of the mind onto everybody else. We have to not only sell our material, but we have to equip students to learn our material. We don’t want to do that. We say, “They should know by now.” Well, guess what? They don’t. So what are you going to do about it? You’ve got to start from where they are. Teaching students learning strategies takes a couple of sentences every class. Now, if you really want to get into self-regulated learning activities and assignments with them, that might take a few minutes per class period, but you don’t have to do it every class period, and a lot of self-regulated learning activities can be homework, in which case they take no class time at all. This is so easy to do. This is why I think faculty have really been attracted to my book and why I’m asked to speak on it so often, because there are so many little things you can do that don’t take away from the content at all—rather, they reinforce the content—that make this huge difference in the performance of most students. You can’t always bring everybody along with you. There are some people—some students—who’d just as soon shoot themselves in their foot, but most do not. They find these activities so easy to do. They don’t take a lot of time and they get to know themselves and start doing better, so students don’t complain about this.

Rebecca: Can you describe what a couple of those activities might be?

Linda: Sure, absolutely. Well, let’s consider the different parts of the course and I’ll just give you just a few, some of my personal favorites. For starting a course for instance—starting and you can also end it with these sorts of activities—but one of them is a goal setting activity. You can assign this as homework, you can have students do it in class with students write on “How I earned an A in this course.” Now, you would be surprised and students will be surprised—C students and B students will be surprised—that they know what it takes to earn an A in a course, and they will come up with, “Well, I’ve got to come to class every day, don’t I? And in class I can’t fiddle around with my mobile device, and I have to start a paper sort of early and I have to keep up with the readings.” For many people, writing this down is goal setting for them. They think, “Well, you know, maybe I could do this, maybe this isn’t so absurd.” If you make a discussion out of it afterwards, the A students will say, “Yeah, I do these things, it’s not unrealistic.” And then the C and B and F students will say, “Well, let’s give it a whirl.” Then at the end of the course, you give them another little essay assignment, “How I earned an A in this course—or not.” [LAUGHTER] True confessions time, right? And so students assess how well they met their goals. Goal setting is definitely a part of self-regulated learning, the planning and then self-evaluation at the end. Another thing that you can do is you can give your students essay questions. If you give an essay final—or have any essays on it at all—you can give them the essays on the final to take the first day. This will not take much class time at all, because students will know very little, or they’ll try to BS an answer. So they will try but they can be really quite wrong. Now at the end, for the final, they correct their answers, and then rewrite these answers given the knowledge that they have gained throughout the course. This can be really interesting for faculty—for not just faculty… well, it can be interesting for them, too—because they can see exactly what they learned. So it is a measure of learning. Faculty will never get that comment on the student evaluations saying, “Well, I didn’t learn anything in this course.” Never again, that’s gone. So anyway, those are a couple of things that you can do. Little assignments you can make on the readings. Little reflection exercises like, “What did you think was the most important point in this reading? What surprised you the most? What connections can you make between what you read and your prior knowledge, what you already know? Or to your life? Or your emotional reactions to it, if the material is amenable to that?” So those are little reflections you can give on the readings. Another exercise, a self-testing exercise, is called “read, recall, review.” This is the best way to do reading. Forget about rereading, that’s what students really do… It’s really a waste of time. What students should do is to read a portion of the chapter or the whole chapter, put their notes away, close the book, and then recall as much as they can and write it down. Then they should go back and look in the chapter for what they forgot and what they might have gummed up. And they know that, “I didn’t really get that point.” And so they go back and look at it, and then they recall again. Read, recall, review. Studies that have been done on this showing it is so much more effective than rereading. It really doesn’t take that long, and then you actually have the material in your head, even in your long-term memory. You get retrieval practice, you get deliberate practice, so there’s nothing as good as testing yourself except—well, the nicest thing we can do for students is to test them. In lectures—I should say mini-lectures—it’s a good idea to have students do this. You stop, let’s say, every 15 minutes or so and have students do the same thing. Write down everything that they can recall, and then work with their neighbor to fill in the blanks—their own blanks—and ask any questions they have. First they ask their neighbor their questions, and then they ask you. This doesn’t take very long at all, maybe five minutes, but then you know that the students got it and can remember it. Again, most effective… studies done on this, too. So this makes students aware of their learning or their lack of learning. You can give students what are called active learning checks. You give your mini-lecture, and then you stop—and by the way, you can warn students you’re going to do this so they’re listening—and ask them, “Okay, what are the three major points in my last mini-lecture that I talked about in the last 15 minutes?” Then they write those things down—and it could be two things or four things depending—and turn them in. They don’t really have to turn them in, but you know, you might want to see them yourself. Then you reveal the three most important points, and they monitor and evaluate their learning skills. Now, students are motivated to want to learn how to listen to you, so they want to improve. According to a study that was done, they improve really quickly. The first time, 45 percent of students got all three points correct. By the third time, 75 percent of the students got these correct. Remarkable progress, really remarkable. Then there are meta-assignments. In a problem solving field like chemistry or math, we are denying students learning opportunities when all we do is mark the wrong answers as wrong or incomplete and then drop the subject. Students should be able to correct their mistakes to get half the points back, let’s say. In other words, they’re going to learn how to solve that problem if it’s the last thing they do. [LAUGHTER] Again, you give them some sort of an incentive, then they learn. There have been studies on this technique as well. It’s extremely effective. And students can learn not just from you, but in peer groups. Peers can help each other very effectively. There is a wrapper—they’re called “wrappers”—for an exam, a reflection that students do after they get their exams back where they answer questions like, “How did your expected grade compare with your actual grade? How do you feel about that?” So they have to look at the exam and your feedback. “How many hours did you study? Was that enough? What did you do while studying? Might you want to change your strategies? Why did you lose points? Were there any patterns that you see here? How are you going to study more effectively for the next exam?” This has been life changing for students because they’ve never thought about this before. They’ve never really looked at their exams, their mistakes. They drop them too, right? They don’t want to see what they did wrong. Yet these are the best learning opportunities possible, and they will remember them. We remember our mistakes, we learn from our mistakes, and it’s sad that we don’t stop and use those errors. These are just a small sample of self-regulated learning activities. I can give you many more. [LAUGHTER]

John: And there are many in the book, which we strongly recommend to people.

Linda: Yes, yes, yeah…

Rebecca: A lot of what you’re talking about seems tied to growth mindset as well.

Linda: Exactly, and this creates, this generates the growth mindset because students learn that they can learn, they can do better. Otherwise they feel like their learning is like the weather. “Maybe it’ll rain on me, and maybe not. [LAUGHTER] There’s really nothing I can do about it. Because it’s all about you, professor, you are responsible for my learning, just like the fates are responsible for the weather. [LAUGHTER] And if I’m not learning, you’re not a good instructor, or you’re pitching the material over my head, or your teaching strategies are wrong.” And so everybody else gets blamed, and then they start to realize, “Oh, I can do this.” Now, this isn’t the best news for them in the world because then they have to start taking responsibility for their learning. And that can be, for some students, a hard pill to swallow. For other students it will be very empowering, and what we want to encourage in students is that sense of empowerment.

John: And that’s especially important, I think, in freshman-level classes, because students generally don’t come in with that type of mindset. They’ve often been able to blame it on the teacher and do things over and over again until they get the grade they want or get the extensions and so forth with a focus on self-esteem in many classrooms.

Linda: Oh, yeah, self-esteem without achievement.

John: But it’s an adjustment. So if they come in with a fixed mindset, and they’re confronted with failure, it’s pretty easy to give up. So we need to encourage students, I think, to see failure as a learning opportunity as you’ve mentioned. As instructors, I think we have to somehow convince them of that, because they don’t come in naturally picking it up, but the techniques you’ve mentioned are very good for that.

Linda: You know, our whole society makes them feel they’re not responsible for their learning. Look at what happens in K-12. Students have to take standardized tests and if they don’t do well, who gets blamed and who suffers? The teacher and the school, and that’s nuts. In the final analysis, we teach ourselves. We are responsible for our own learning. Good teaching can make a big difference because we can be motivated or unmotivated by teaching. We can acquire learning strategies through teaching. So it’s not that students are just left adrift on their own, we do have to help them. We do have to put them in learning experiences where learning becomes attractive for them, or you can’t help but learn, right? But they’ve got to pick up that learning and run with it themselves.

Rebecca: So you mentioned the idea of encouraging students to see learning and the self-regulation as empowering. What about those students who are a little resistant to that because it’s surprising to them that they’re not getting it and they’re failing and that it’s going to be more work? What are some things that we can do to encourage those students to see things a little differently?

Linda: Yes, first of all, if they’re failing and they subconsciously want to—it happens, it really does —[LAUGHTER] there is not a whole lot you can do about it. They might need some counseling and they might need to get some help from professionals like psychologists. But again, it can be difficult for students to realize that the ball is in their court because it’s a whole different gestalt for them. The only cure for that is success—a little bit of success—where they start doing a little better, let’s say, on the quiz on the readings or they start being able to solve more problems. That’s really the only cure. And we are assuming that they want to be successful. Again, if they prefer failure, then they are responsible for their own failure.

Rebecca: Right, they’re the ones that are normal and we are not, right? [LAUGHTER]

Linda: Exactly.

Rebecca: Maybe that should be the refrain of this interview, right?

Linda: Yes. They are the normal ones and we are strange. And we always have been strange. We were the strange kids in school, too. [LAUGHTER]

John: In your book you mentioned the Dunning-Kruger effect as being a barrier to some students, that the students who don’t understand things as well often overestimate their understanding. How can we overcome that?

Linda: Self-regulated learning helps when we give them activities and assignments where they do self-evaluation, because the only way to learn self-evaluation is through practice, practice with feedback. And that feedback doesn’t even have to come from you; it can come from peers or a computer program. We don’t give students a lot of practice in self-evaluation, and they certainly haven’t had much of it in K-12l. But the nice thing is that when we have students look back to see if they met their goals, or to evaluate their study strategies, or to assess their mistakes and the reasons for their mistakes, it makes all the difference in the world. After low-stakes practice, you can introduce higher stakes self-evaluation assignments and see more savvy self-evaluations.

John: What recommendations do you have for faculty who’d like to start building more self-regulation? Are there small steps that faculty could take to get started on this path?

Linda: Absolutely. There’s a sense in which most of the assignments, most of the activities, are little things. Here’s a little thing you can start off the course with, I was talking about essay questions, but you can just have students do a little reflection the first day and then again on the last day about the subject matter, as in, “What do you think chemistry is? Why is it a science?” You can find out a lot about students’ misconceptions just by looking at these reflections. And then of course, they’ve hopefully corrected a lot of misconceptions by the end. This could take like all of five minutes the first day. There’s so many little things. Here are some ideas for experiential learning. It’s so easy for students not to make a connection between a simulation, an interesting role play, a service-learning experience, or field work to the course. So it’s important that whenever you do an experiential assignment or activity, students reflect on what they are learning—for a simulation, to look back and explain what their goals were, to evaluate how well they met their goals, to assess their strategies, to explain how their strategies changed and their responses to other players. It’s very important that students become conscious of what’s going on in their heads. Only by becoming conscious can they remember the strategies? [LAUGHTER] And then they can write them down and articulate them. You can have students do short papers associated with papers and projects where they record, while they are doing it, the process they are following. If you’ve given them a process to follow, they even have a skeletal outline of what they should be doing. This is a place also for self-evaluation. If you have students do a revision, oftentimes you give them feedback on what they should revise, and they may or may not read your advice. So you can have them paraphrase your feedback back to them and write out their goals for the revision. What are they going to do? What are their strategies for revision? These are just little things. Students start to realize the value of this. And again, this is an assignment where you can’t screw up. It’s not a test, it’s just a reflection of what’s going on in your head. Students like to learn about themselves. And this is like the reading reflections. This is no stress. How do you mess this up? It takes less stress to just write an honest answer than it is to make one up that sounds credible. [LAUGHTER] I want to make faculty aware that the activities don’t have to be graded at all. The assignments don’t really have to be graded. You “grade” them pass-fail. Students pass just by completing the assignment. Let’s say, you had them answering three questions, three reflections. Did they answer three reflections? Is it vaguely on the chapter? Okay it’s not about football, it’s something about what’s in the chapter. [LAUGHTER] And did they meet the length requirement? It’s always a good idea to give length requirements on these reflections because for students, length means depth. So if you ask them to write a minimum of 150 words, you know, they’ll tend to do that. Those who don’t fail.You don’t count every word that students write. You eyeball the reflection. Essentially you are “grading” pass-fail at a glance. It doesn’t take much time. Plus, it gets students to do the readings in the easiest way and most productive way for them. It’s all about them, and it’s not about us. We just have to hold them accountable in some very quick way. Even the longer assignments that you might associate with a paper or project can be graded pass-fail. You have to make them worth some points if you’re still on a point system. But there are alternatives, that’s what specifications grading is all about. You don’t have to use points. In any case, you do have to at least eyeball the reflections and give some value in your course however you are grading. That communicates to students that this is important to you, that you put value on this meta-assignment or assignment wrapper, as you might call it. The same thing with the post-exam wrapper, these reflections on this exam. You make students do it because it’s worth 10 points if they simply complete it and hand it in—even though it’s for them—and they will realize right away that there is some value to this. Again, for some students, it will be life changing in the most positive way. And they will start to realize the way that they’ve been preparing for or taking exams may not be the best. They will realize what they tend to do when they’re taking an exam, such as to misread the question, or to be careless, or to not budget their time, or to not really thoroughly study all the material. Cramming is not very effective. You don’t have to spend time grading these exercises or giving any feedback at all. They can give themselves their own feedback. If they did it and they get the 10 points, okay, that’s plenty of feedback for them. They did it. You regard it as their meeting the requirements of the assignment.

John: And this is a topic you cover in another book on specifications grading, which is also another book we’d like to recommend. We’ll include a link to both of those on the show notes.

Linda: The title of the book is Specifications Grading: Restoring Rigor, Motivating Students—it’s been found to be motivating—and Saving Faculty Time, saving you time. If there’s one thing we don’t have, it’s time. Time is really more precious to us than money. Otherwise, we’d be working in some venture capital firm or something. [LAUGHTER] But time is really quite a precious thing for us. So, in terms of these sort of assignments, for self-regulated learning assignments, they’re all what we’d call “specs graded.” You set out the specs, they’re very simple, and you just grade them pass-fail.

Rebecca: I think pointing out how it doesn’t have to be complicated for faculty is important because I think we all want students to learn. We all want them to be self-regulated learners.

John: We all want to give students feedback, but we don’t want to make it impossible for us to keep up with our work.

Rebecca: Yeah. Or feedback that’s going to get ignored anyways.

John: Right.

Linda: You’re worried about students reading the feedback, and our feedback is valuable. We’ve given it, we’ve taken the time, and so you make them paraphrase it back to you. And this could be a learning experience for us because we might be “misread.” Students might not understand something that we’ve said. Awkward—that’s my favorite one—a sentence structure is awkward. What does that mean? That student didn’t set out to write an awkward sentence. That in itself will not help them because they don’t know what you’re talking about, and this is most unfortunate. But again, it’s a learning experience for us and we can learn to express ourselves somewhat differently. Too often, students get back a paper from us and look at the grade, read the paragraph at the end of the paper, and put it in their “circular file.” They dump it. They don’t read that feedback, so how are they going to get better? So paraphrasing our feedback back to us can be a very valuable exercise for them. And you can let them gain back some points for it. I just think that faculty should look at themselves as responsible for helping our students learn. They don’t come to us with those skills. We can be the finest instructor in the world, have the most interesting classes, hold their attention, and motivate them, but if they don’t know how to process that material in their own minds, it’s all for naught. Now, maybe, hopefully, seniors have learned to learn their material along the line. And by the way, there can be different learning strategies for different subject matter. There are different self-regulated learning activities and assignments for problem solving mathematically-based fields, and different ones for the social sciences and humanities. There can different kinds of assignments, different kinds of readings, actually different kinds of lectures. So we have to respect that. But we have to become conscious of study strategies, learning strategies, our strategies, and other strategies that are out there. But self-regulated learning strategies, to my mind, they’re the shortest distance between two points. Shortest distance between ignorance and learning because it’s all going on in your head, and it’s so powerful. The value of it to students becomes evident rather quickly.

Rebecca: And it’s a skill that can help through a whole lifetime, not just while they’re in college and I think helping students realize that is also really valuable.

Linda: Like no other generation before, these younger generations are going to have to learn to learn on their own. They’re going to have to keep up with their field, whatever their field is, and they might have to—will very likely have to—pivot into another field because their first field might run its course. They’re going to have to learn on their own. They aren’t going to have employers holding their hand. Not at all. They’re probably going to have to learn online, where you really are responsible for your own processing, more so than you might feel let’s say in a face-to-face class, and for your own motivating as well. There needs to be more motivation than simple fear that you will go hungry and won’t be able to get a job. [LAUGHTER] Yes, students are going to really have to learn how to learn. If they consider that a bitter pill, that’s too bad. This is reality, this is life, and most of them have not learned that life is hard. Many of them are wondering where their next meal is coming from, but a lot of students have not. Students need to learn along the line that life is not easy, that nobody does curling on their path. And they will face challenges, but if they have the strategies for facing these challenges, no problem. They needn’t be paralyzed. They needn’t freeze.

John: You foster some really good advice and I think our listeners will appreciate this and it’s really powerful.

Rebecca: Yeah, I agree. We’re all wondering, what’s next for you?

Linda: I’m actually Director Emeritus. I’m actually retired from Clemson University. But you know how academics are, [LAUGHTER] they don’t disappear, they just sort of like fade away. So I’m trying to ease into retirement because it’s not an easy thing to do? Not when you love what you’ve been doing. But I have sworn off writing books. That’s progress! [LAUGHTER] I’ve written some articles and chapters in other people’s books, so that’s fine. And I’m still traveling to give keynotes and faculty workshops. That’s hard to give up because it’s interesting to go somewhere else, somewhere new.” And I still give webinars and podcasts. But eventually I won’t be doing that anymore. Ultimately I want to work with animals. I do love animals but I’m still busy doing this and still loving doing this, but also loving just as much not having to do bureaucratic tasks for the university [LAUGHTER] and not having to stay up until two in the morning doing my email. When I’m traveling, not have to worry about, what’s going on back at the office. So I’m not complaining about retirement. I really like where I’m at right now, but I know that I will eventually fade into the sunset. That’s okay because then I’ll reinvent myself.

Rebecca: Sounds like some self-regulation was going on there, I’m pretty sure. [LAUGHTER]

Linda: Yes, I’m trying, I’m trying to retire but not too quickly.

John: Well, we’re glad you haven’t, fully yet.

Rebecca: This was really great. Thank you so much for spending some time with us today.

Linda: Well thank you for this opportunity. I hope that I have helped some faculty members out there to help them help their students to achieve more, because again, we all do want our students to learn. We’re all in love with our material, it’s worth learning, and we just have to help our students do that. So thank you ever so much, and thank all of you listeners for listening.

Rebecca: Thank you so much.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

68. Mobile Music instruction

There are apps for just about everything but choosing when to embrace them for instruction needs to be a careful decision. In this episode, Trevor Jorgensen joins us to discuss how the decision to use mobile apps in music instruction is affected by where students are developmentally, convenience, cost, and other factors. Trevor is an Assistant Professor of Music and the Student Learning Outcomes Assessment Coordinator at SUNY Oswego.

Show Notes

Additional Resources

Transcript

John: There are apps for just about everything but choosing when to embrace them for instruction needs to be a careful decision. In this episode we consider where students are developmentally in a discipline, convenience, cost, and other factors on the choice of supporting music instruction using mobile applications.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Trevor Jorgensen, an Assistant Professor of Music and Student Learning Outcomes Assessment Coordinator at SUNY Oswego. Welcome, Trevor.

John: Welcome.

Trevor: Hey.

John: Our teas today are:

Trevor: Pure Leaf Peach Flavor tea.

John: Iced tea.

Trevor: Iced tea. Sorry. Thank you.

John: …and Rebecca.

Rebecca: I have Chai today…

John:…and I have black raspberry green tea. We’ve invited you here to talk about how you’ve been using mobile technology in applied music instruction, specifically related to performance. Could you tell us a bit about that?

Trevor: Yeah, I think one of the most important things is to decide when to use technology where it benefits as opposed to be either a substitute for something you should learn in a different way, or whether you should implement it because it’s just convenient. Sometimes when it’s convenient, though, it takes away some of the learning behind it, and I’ll get into some of that later. So I think that’s the first thing. I have to make that decision on every app that I use, or every device that I employ. So, one of the main things that I do is decide what’s best for the student and what’s best for me. And a lot of it does substitute for other things, but in the end, it can’t. For instance, if I do a collaborative piano thing with my colleague, there’s a lot of tools that we’ll talk about today that will substitute for that collaboration. Except for, I cannot phrase with the app in the the same way that I can. We can’t make decisions on a moment’s notice, like I can with another musician. We can’t have talks… at least I shouldn’t be talking to my apps… about what the best way to do something is and then make that interpretation or change colors of stuff. But what it allows me to do is, instead of having a more rehearsals with somebody that’s really busy, it allows me to rehearse by myself. And then therefore, save some time when we’re actually in the rehearsal to talk about those musical decisions as opposed to just to try to learn notes or trying to learn one anothers parts.

John: What are some of the apps that you use with instruction or performance?

Trevor: One of my favorite ones is Amazing Slow Downer, and I use this for both jazz and classical, being a saxophonist, and also a multi-reed player…I play clarinet, oboe, and bassoon. I like to use it in every circumstance I can. And it’s really a great app because even though there’s things like YouTube where you can now click over the three little dots in the corner, and you can slow down to 25%, 50% and 75% and speed it up if you so desire. This tool is just something that I always have with me and I can always use. And I’m also more familiar with it, it allows me to do multiple things. So let me give you example if I can play musical selection here. The first two are not going to be on the music slower downer, it’s going to be on Amazon Music. But I want you to listen. These are Brahms’ Clarinet Trio. I performed this with a colleague and somebody from Symphoria recently. And one of the things you’re doing as a musician is you have to interpret the history of music, and that includes listening. So if I listened to multiple recordings of something, you know, we always ask “Why are there a thousand recordings—might be overstated—but of Beethoven’s Ninth Symphony?” That’s because the composer had a certain thing in mind. And then the conductor who prepares the music, and the individuals in that group have different ways of interpreting that music and every one is different. But for a lot of younger students, and sometimes myself when it’s really advanced players, I’m not exactly sure what they’re doing, either articulation wise or what they’re doing phrasing wise. And the slower downer allows me to think about what they’re doing and compare all these great artists together. So I’ll do multiple recordings on something like this Brahms’ clarinet trio in A minor and have my students listen to it. I’m going to play one of them for you, and then I’ll play another one, and then I’ll play the third one. And then the third one I’ll slow down to show you how we can listen for different articulations. You may or may not tell the difference between the three, some of them are obvious, like there’s a tempo difference between them. But as far as the phrasing and the articulation used by the cellist, and by that clarinetist, and by the pianist, is something to think about. What do I want to sound like? It’s great to imitate people, but it’s also great to make decisions between all these different things you’re, in a way, sourcing and decide what you want to do and why you’re doing those musical decisions.

Rebecca: Before we jump into listening, for our audience who maybe is not as familiar with the music terms that you’re using? Can you clarify the difference between phrasing and articulation?

Trevor: Sure. Great. So articulation is the front of the note, and thank you for your much for that. So as a saxophonist, it’s how I tongue the note, if I put an emphasis through air or through my tongue. Another word for that is “attack,” how it attacks into it. For a string player, it might be the direction—up bow or down bow—how fast they’re moving the bow. For the pianist, how they voice the chords and are there some fingers stronger than the others. A third comes out of the chord and the fifth comes out of the chord. For a chord you have multiple, you know 1-3-5s, or the seventh of the chord. And each one of those, as a pianist, when they’re putting down multiple fingers, they can decide which ones are the most important. Whether it be the melody, whether it be the bass part, whether it be the inner harmony. So, from a pianist’s point-of-view, they can listen for that, or how the touches on the piano, how much they pressurize it, or if they use pedal in this area that’s not dictated by the score exactly. So phrasing, the best way to describe it is like minute dynamics. Or just using what we normally do. We, as human beings phrase, if you like somebody you change your voice, or if you get excited like, “I have go to the bathroom,” you don’t think, “Hey, I should elevate the dynamic of my voice. I should make it sound strained, so that I show the excitement of me having to go to the bathroom.” But in music, we have to think about that. We have to think about where, based on the harmony that’s below the music, how much am I going to emphasize that note or more importantly, maybe even growth of that note, or that musical phrase. So phrasing’s how people interpret the music beyond playing the right notes in the right place, and doing the pianoforte dynamics; it’s how they’re going to go to a note and resolve a note. And not only that, they think about overall structure of a piece for phrasing and how everything fits together, if they’re amazing musicians. When we listen to amazing musicians—like the ones who found in these recordings, then what we’re going to do is interpret what they did. And then maybe in our own minds figure out why you think they did that, if that’s a choice that you like to do based on the harmony or just a different tone color they do. And that’s obviously easier to figure out when you slow down to 70% than it is when you’re listening to it at 100%. So before I play the examples, the actual artists will be in the show notes below in the description. First one is, I’m just going to name the clarinetist, Karl Leister, and this is them playing the Brahms’ Trio. And I’ll do the first 30 seconds or so, so you can hear how it sounds.

[KARL LEISTER PLAYING BRAHMS’ TRIO]

Trevor: The next example will be by the clarinetist David Shifrin, that trio that he’s in. I’ll play about the same amount of time for it.

[DAVID SHIFRIN PLAYING BRAHMS’ TRIO]

Trevor: The last one I’ll do on Amazing Slower Downer, the app. And we’ll actually play it in tempo first, since its Martin Frost as the clarinetist, playing the same piece.

[MARTIN FROST PLAYING BRAHMS’ TRIO]

Trevor: So what we’ll do now is we’ll actually slow it down to like 70% so we can actually hear a little bit better the articulations, the dynamics—as the artist gets louder, you’ll be able to hear it more individual. And we can even slow it down after that into something ridiculous like 25%, where you can actually hear the front of each note. It’s almost painstaking how slow it is, but interpreting different things, it’s interesting to see if they use a breath attack or a tongue attack, and some of them are so subtle, I’d have to listen to it 25 times without slowing it down where I can actually hear it at 25% right away. So let’s go back and do it about 70%.

[BRAHMS’ TRIO PLAYED AT 70% SPEED]

Trevor: Here’s 25%.

[BRAHMS’ TRIO PLAYED AT 25% SPEED]

Trevor: I’m just going to jump ahead a little bit so we get into the clarinetist and stuff. I’m just going to scroll ahead.

[BRAHMS’ TRIO PLAYED AT 25% SPEED]

Trevor: First of all it’s very unfair because I would never want to be heard at 25%—it’s just impossible to keep a phrase sometimes or keep a note perfectly that way—but what it does is for us is we can actually hear them the front of the note… we can hear the shape of that note even if it’s a quarter note if he—in this case it’s a he—de-crescendos or crescendos into it and we can look at the overall structure of the dynamics too. You can see if he’s actually growing on that note, if you’re taping the note as I said before… the overall picture or phrase too. I think 25% might be a little excessive, you have to find the happy medium based on the tempo of the piece. I think initially 25% for this one might have been a little painstaking at the beginning.

John: But for students, perhaps having the ability to slow it down to that level would be much more helpful than for someone with more experience?

Trevor: Right, there’s definitely that and then we’re looking into transcription later on too. There’s this great book called School for Cool: The Academic Jazz Program and the Paradox of Institutionalized Creativity by Eitan Wilf and I hope I’m pronouncing his name right. He wrote it in 2014 and he basically went to Berklee School of Music and then The New School in Manhattan. And he noticed that students were using this application even back then—and when I say back then is only five years ago, or four years ago, or three, because in the time of technology, that’s way back then—and they were actually interpreting, transcribing not only John Coltrane and getting his notes like I would have done when I was in grad school or undergrad, but also then listening to his articulation at the front of the note. So trying to make their tongue sound exactly like that slow, and then they’d be speeding it up. Also, in a real fast passage we use a fake fingering. What fake fingering is, is there’s a normal way of fingering D on saxophone which is like 123-123 fingers down and then there’s a way of playing the high D but not using the octave key so it comes out in kind of a muffled way, I could probably demonstrate it for you if you’d like.

[TREVOR PLAYS D NOTE 3 DIFFERENT WAYS]

Trevor: So you use three different fingerings for the same note, which I’d probably not do in classical, but for jazz, it adds a different color. A lot of people know certain people will play that way so that you can actually determine those notes when it’s really fast it’s hard to tell. Just that nuanced playing that you can actually interpret when you’re listening to John Coltrane, that I didn’t when I was that age of the students that he was, you know, surveying. But, slowing it down, it’s not something new. There’s another great book too, Thinking in Jazz: The Infinite Art of Improvisation by Paul Berliner who was published in 1994. He talks about how the old school players would get a vinyl record and they’d put their thumb on it and slow it down to like a third of the speed and then obviously transpose. Now they would be in the wrong key and way too low, and then they just transpose it up. So this technology, or using this technology, has been available as far as transcription. But if you did put your thumb on a record, it would probably distort the sound and the way that you can figure out articulation and you couldn’t figure out phrasing.

John: And the advantage of this is that students can preserve the pitch because all these apps maintain the pitch and you can hear the attack… just more slowly.

Trevor: This one is specifically and I’m sure a lot of things you can probably do this on YouTube. Also, a lot of my students have to in order to avoid buying a $15 app like this one is—at least when I bought it it was—they’ll find other sources, which I think is great. It doesn’t matter to me what they use as long as it does the same thing. I think on YouTube, you can do the same thing where you can loop it, and there is these three seconds or 10 seconds or whatever, and just loop it and here you can loop it. You can also save the file, save the section of the piece, call it what you want, and the next time you open up the app, it’s there. And you can also share your ideas with people. You can download stuff from things you buy on iTunes or you can use Spotify and there are other ways of importing into this app that’s really beneficial too. The nice part about this app is it can be used for classical and jazz. Most people use it mainly for jazz and transcription but I use it for everything and additionally you can actually slow your own self down which is always painful and hear that you weren’t phrasing that well or that your end of notes aren’t as tapered as they could be. And once you listen to amazing players you realize that they have control over everything. I’ve slowed myself down. I always know the differences between me and like the best person in the world, although I knew already but but it’s so evident when you slow it down to 50% and you hear the attack of your note is not as clear and not as shaped as the people that are just geniuses.

John: And you can also use that with students, though, to show them what they’re doing, right?

Trevor: Exactly. I mean, it’s a very revealing tool. So as far as students are concerned, I think it’s great tool for them but also can be somewhat depressing to hear yourself. It has to be a good balance and the student has to be at the point where we’re finding things not initially at the beginning, or maybe they’re even in their freshman or sophomore year, but at the end when they’re preparing for grad school or something like that, and it depends on the student as it does with every situation… what you can push them on and what you can’t. But, it’s just a great tool for that. So I also use the Amazing Slower Downer when I have a piece that I don’t have a—we’ll talk about Smart Music later where it is an intuitive accompaniment tool that takes away the saxophone or takes away or whatever it is you’re working on—but when it’s in pieces not in there or I don’t know the piece, I’ll find a recording of it and then I’ll play along with it. And this I can adjust a little bit of pitch if I need to and more importantly speed if I needed to get slower or I decided to take the piece at the end faster. Maybe now’s the time to talk about what the disadvantage of some technology is. I’ve got colleagues that don’t like using slower downers, not just colleagues but I’ve read articles and everything else, because you don’t have to go through the work and the persistence of doing so, and you should be able to hear it in the moment in the spot. And I think there’s something to be said for that. But I look at myself as one of the things where I get discouraged. I always think about one of my favorite reads, which is a psychologist by the name of Csikszentmihalyi and he’s got the flow theory, which is a book he published in 1990, which a lot of people are aware of. And number three of his flow theory is that there has to be a balance between the challenge and skills. For me, there wasn’t, my skills were low and my patience were also maybe lower when I was younger. [LAUGHTER]

John: And I think that’s true for most starting students. And a lot of people get discouraged when they don’t see that immediate progress.

Trevor: Yeah, that’s exactly it. So for me, I was able to use the tools that helped me a lot more. I painstakingly did it when I was my students’ age that was also because of the slower downers were $400 and I didn’t have the money. Now most students have phones but they use them for social media or actually using them as phones, or texting more likely. But now you can just download an app instead of buying a $400 machine for about $15, or you can go to YouTube and use that function or other tools that come with any kind of PC nowadays. Microsoft has their recording software and you can slow it down too. Not in such a certain way, but any of those tools are great too. Anyways I’ll slow my own recordings down of me playing along with other things. But the problem is trying to figure out that balance where I should be taking it to the tempo there I just took it or slowing it down. You just have to progress and move it forward so that you take it faster, faster each time. The thing about anything is once we do it more often, it becomes easier.

John: And the more they practice with this at a slow pace, the better they’ll be able to do it in real time as well.

Trevor: Right, and their ears develop immediately, faster and faster, and then therefore they can get to that point. I commend my colleagues and all the articles that I’ve read where you know, it’s better to do it that way. But for me, it was a wall, as it is for some of my students.

Rebecca: Sometimes it’s good to recognize when you are a beginner and that sometimes you need those training wheels just to feel like you got a little success.

Trevor: Right. That success as we all know it from pedagogy is that wields more success and it gets better and better every time. Let’s go on to a different app. Some of the obvious apps are metronome apps, every student can get a free metronome. I was excited in the mid 2000s when they dropped down to $20 as opposed to like $200, or the tuner is the same way, and then balancing that out. For me it’s I could buy a $20 tuner and put one on every one of my instruments but I always have my cell phone on me that’s what I love about it. I always have a tuner, I always have a metronome and I have no excuse, nor do my students. You know it’s a free app; you already own the phone. But there are some tuners and some metronomes that are better than others and I am not an authority, I don’t know every app that’s out there because there’s thousands. The ones that I know are the ones that I like… people have introduced to me and I’ll switch apps all the time if I find one that’s better. As far as the tuner or metronome it doesn’t really matter what my student brings in… whether she brings in the one that I prefer, like Cleartune for everyday apps, or if she brings in like TonalEnergy Tuner, but I’m going to talk about that one because it’s one that I found really interesting and it has a lot of functions on it. And it’s not only a tuner but a metronome. I use Tempo app for metronome primarily when I’m practicing but I’m getting into the TonalEnergy app for inside of it has also a metronome that I’m trying to investigate. So the app has so many different features to it, and I’ll show you and then maybe both of you can also chime in on what you think they look like, but…

John: I use the Cleartune app myself.

Trevor: Yeah, I loved that app until I saw this app. To give you a total side story, this one does analysis by waveform as it does it by another analysis and note names, too. This weekend, I judged a competition for high school students to get into all-county bands. And one of the other judges in the brass room said one of the students—and I don’t know the name of the student nor the judge—but the performance was really not great at all, where he actually pulled out his TonalEnergy Tuner to figure out what note the student was on to see if he could find out where he or she was was in the piece. That being the extreme of what the app is used for… nor do you want to use it for transcription because there are transcription apps where you plug it in and it’ll transcribe it for you. For playing our transcriptions—there’s worth to all those things—as far as actual transcriptions… a tool that you should have so that it ingrains in your memory and you have those phrases for jazz specifically. So this one here you can choose from waveform, you can do spectral harmonic analysis and note analysis. As you can see right now as we’re talking, sometimes I’m talking in tune and he gives me a happy face—it could be a she—gives me a smiley face, or if I’m really badly out of tune, there’s this little question mark with the thumb and finger or very sadness as you get way out of tune. But more importantly, as I do the waveform thing, it will record what I do.

Rebecca: I was also noticing that the color was changing as well. So, there was a lot of different identifiers to the user so if you had a certain kind of disability it’s actually providing that information in multiple ways.

Trevor: Yeah, and that’s great. And then over on the side it tells you pluses and minuses, I’m 13 cents sharp or whatever, 13.6 or flat. So, I can show the pitch, I can show the wave, and I can record it. Let’s go ahead and record it. You’ll see as we’re talking the spectrum, and then if I do with the notes, it’ll show me the notes, which ones they are, and how badly out of tune they are and we’re just talking. I could play my saxophone. But it would give you the same thing.

Rebecca: Right.

Trevor: So as far as I’m concerned, if I turn this away, and I can record it, and then go back and look at it, I’ll know that if I do nothing, or how much I need to do on the D. The D that I played for you before, nd gave you three different examples is the worse out-of-tune note on saxophone. So, I probably was sharp but this will tell me every note that I need to. You can even slow it down and stuff. You can also record your student visually and with audio. The visual thing is interesting because a lot of movement in the face will affect tuning. When I came to college, I chewed a lot, I had movements in my throat and my face and my teacher would just tell me what they were. But now I can like hold the video against them, show what their movements are, and how it’s affecting tuning and it’s, you know, just another another tool to get you there beyond just recording their facial features and hearing it.

Rebecca: It’s an interesting way to note take because it’s a visual note-taking method that you’re not just having to remember the conversation that you had with your faculty member or mentor, but that you can go back to and see again, if you’re able to record it and save it for later.

Trevor: Right. Yeah, I know, it’s great. It’s a neat tool. So, I’m starting to love this tuner and this metronome as much as the others. It does a lot of specific things where if I’m just pulling out a tuner like Cleartune, as we talked about, or Tempo, that’s more my functional as opposed to how badly I’m out of tune when I’m playing this passage. And the same thing I notice whether I’m using this app, TonalEnergy Tuner, or I’m using Amazing Slower Downer, or I’m using what we’ll talk about later, Smart Music, is that when I’m playing along with the pitch, in the case of Amazing Slower Downer, I know I can hear I’m out of tune, and then I can check it against this or if I’m playing along with Smart Music and you check it against here and see where I’m out. And I play multiple instruments so I’d like to tell you I’m perfectly in tune on every instrument but sometimes I’ll be playing A clarinet and I haven’t played that for a while where I normally do B-flat clarinet and I’m like, “Oh, that’s right. This B-flat is flat on the A but sharp on the…” you know, you have to adjust and unlearn and it allows me to do that prior to getting in there with my collaborative pianist and realizing that I’m out of tune. It’s a great tool. It’s got so many more features that I’m going to start looking into but there are funny features like… not really funny… but the band director or the the judge using it to try to figure out where they were, which hopefully we won’t have to use it very often for.

Rebecca: I wonder for a beginning student who has very little experience… maybe someone who’s not in the music program, but is learning an instrument. Have you worked with students using these kinds of apps with that population of students… like total beginners?

Trevor: Prior to teaching here at SUNY Oswego, I was teaching beginning students… so, seven to eighth in junior high program and then had fifth and sixth grade students. I haven’t used it and mainly because of physicality. Until you develop your armature, until you learn how to use the air, it’s almost impossible to use this effectively because if you don’t have the proper air you can’t make those minute adjustments. That being said, to get the initial tuning for every instrument, whether you’re in fifth grade or fourth grade, you do want to make sure the violin is in tune so students will start using that, you do want to make sure that your concert B-flat’s in tune, which only means after that all the other 11 notes are out of tune, but it’s a good reference point. And then students start to use tuners that way, and amazingly some of them like apps and stuff, and they get into and they show me different things they’re using for that I never thought. But initially, I try not… until those physical things are employed properly. It’s almost futile to actually do it all the time.

John: They need the fundamentals before you can work on tweaking.

Trevor: Right, yeah, especially for you know, with piano you don’t have to worry about tuning. Guitar, that’s got frets so it’s good to know how to tune it. You know that more…

John: You’ve never seen my piano, but… [LAUGHTER]

Trevor: Yeah, but yeah, no… We’ll talk about the next thing, Smart Music, and how it’s used more, or at least, I think more band directors in the public schools use it at elementary all the way through high school, then maybe as far as band music and material is than college professors. I use it for the solo literature, like clarinet concertos and sonatas ….but in the public schools they’re using this technology we’ll get into to not only evaluate their students—how they’re doing, to keep track of their practicing—but as a preparation tool, it’s great. And it can also be not as great for certain specific things and we’ll talk about that when we get there. The next tool—and we’re kind of focusing on my phone right now—the only app that is not available on a phone, a mobile platform that way, but is only on an iPad—is the Smart Music. As I said, we’ll get into that later. The next app I like a lot and I just got an email right before this, there’s a physics professor on campus who’s named Shashi and he’s a saxophone player. And he does exceptionally well, he’s way better of a saxophone player than I am in physics by any means. [LAUGHTER] He loves it. But he gave me an update “I used iReal Pro yesterday at an open jam session. I played these tunes” and went “Great, I’m excited.” It’s great to see that it can be used by by anyone. And as I said, he’s come miles and he loves this because for a physics professor he probably jams with people and he’s been doing a lot more of that. But initially, he’s like, “Who do I get to play with?” and if you don’t play music with somebody—it’s a social thing too… or people hear it—then you kind of like “Why am I doing this?” Even myself as a professional musician, my colleagues and I will prepare harder and more for either recordings or performances coming up then we will during the summer when there’s nothing to do. That being said, we’re preparing always for the next semester, but it’s always great to have that motivation. If I never played with anybody I probably wouldn’t continue playing, especially being a saxophone. A pianist is different, a guitarist is different, perhaps. Drummer is probably different too. I can’t imagine one person only being a drummer and doing that for the rest of life without collaborating with someone.

John: Did you tell us what iReal Pro is?

Trevor: In jazz, there’s these things called real books or fake books. Now, they’re legal. Back about 20, 30 years ago you can only buy them through illegal sources, and they were the melodies of famous popular music standards and jazz compositions, with chord changes behind them. And then you would get together with musicians and you’d play. Now back in the 70s, a guy by the name of Jamey Aebersold used professional musicians who recorded them on CDs—or back then I guess vinyl—and then gave a play along book with it. So he’s got the Jamey Aebersold play along. And what this is done is taking non-live musicians and put it into an app where you can play along with. You can see the chord changes. There are no melodies because you’d have to pay money and royalties for the melodies. But there is no royalties needed for chord changes, and so of the title of the tune and then the chord changes. It comes with no actual pieces on it until you go to the forum and then you can download thousands of pieces that are already programmed in that you can play along with. So I’m just going to pick up right now blues “Straight, No Chaser” by Thelonious Monk. Although it could be any blues until you hear the head and I’m just going to play you a little bit of how it sounds and then I’ll give you a visual description later about it.

[“Straight, No Chaser” PLAYS]

As you notice, when the music goes by, or the chord changes goes by, there’s a highlighted measure so that you know where you are, which is handy for students that are learning to improvise. Later on it’s not necessary, you could just look off a real book lead sheet and play along with it. There’s many features, one I can make the transposing instrument like I’m playing saxophone in the key of B-flat instead of C. I can see the chords transposed in front of me even though it’s playing the key of C. I can do that for any instrument. There’s like a balancing… if I’m a pianist, I can take the piano out or if I’m a drummer, I can take the drummer out. I’m not sure why a drummer would want to do that. And I can control the reverb… I can change the tempo… I can change the style—this is a jazz medium up swing—I can change it to almost anything—Latin, Gypsy jazz, trio guitar, slow swing, traditional jazz—and there’s pop and Latin stuff so I can change the beat behind. I don’t do that as much unless you’re working in the Latin situation but it’s nice to have and as I said, you can also program pieces in. So we just did a concert with this great alto saxophone player by the name of Dick Oatts down in the Village Vanguard Band—that’s where he is primarily and teaches at Temple University—a world renowned musician. And he sent us his charts which are really hard and don’t follow normal progressions. They’re really kind of tricky so I actually programmed them, or one of them, into here just so I could play along with it a little bit… hear the changes a little bit better. I mean I’ll voice them out on my saxophone like normal but it’s a really handy tool to have. And speed it up too like I can’t take it to the tempo he wants it so I’m gonna slow it down and feel good about myself and they’re not really that odd but they’re more unusual chord changes. They made sense but just needed more time to practice and I just programmed in it by…you kind of scroll up and type in F major and put a seven and you can do all the different things. It’s really interesting and you can save them and if you want and you can share them on the forum so other people can look at tunes. So this is an amazing tool. I love it for that because I can speed it up and slow it down, but there are other options. I think Aebersold things are still available online unfortunately for Aebersold ‘cause you know, he’s losing royalties and stuff on that. But then on YouTube, there’s “Learn to Play Jazz” I think if you type that in and whatever tune you want to do there’s that for the people that actually have iReal Pro copies on YouTube. You can’t slow it down, you can’t transpose it. You can adjust the balance and blend and stuff but it’s there. So my students will find those things too and use them. And I’m totally fine with that. You know, financially it makes sense. One of my favorite features is…and John, you have this app you said?

John: Yeah, I do.

Trevor: So you use this app, and have you have you looked at the tablature part of the app?

John: I haven’t been using it recently. Since we started the podcast I haven’t been playing so much music.

[LAUGHTER]

Trevor: Got ya. Give me a second here. So I’m going to show you…and do you play an instrument at all?

Rebecca: No. That’s why I was asking all those music questions.

Trevor: And please continue to do so. [LAUGHTER] In the corner of the app there’s this little thing that looks like a tablature and you can choose from multiple things. One is for guitar so you can select guitar chords. You can do piano, and the piano you can do one hand or two hands. You can do ukulele, which is a new feature and more importantly for me is the chord scale library which I use all the time. So let’s go back and add the chord scale library. So if I press play and then I pause it right away it’ll give me chords down at the bottom. So for that C7 chord I can do either dominant mixolydian chord if I scroll over. Actually for this one, there’s only one, but for a lot of times when it’s a different chord like flat 5 or something like that it’ll give you multiple options of what scaless you can use over top of it. Once you talk to a pianist—they’re not really fond of this app, because it’s not exactly like you would teach jazz piano—but if you have no skills, and you want to learn from an app, it can give you some things. It’s a really nice tool for myself and all of my students too. And as I said, our physics professor on campus uses it a lot and as do you prior to doing the podcast.

John: I used to use it more before we started this podcast.

[LAUGHTER]

Rebecca: Trevor, we found that students were using these tools—or when you’re using these as part of your teaching—that the students abilities have improved or that they’re playing has improved. Have you noticed a difference in the integration or?

Trevor: In some regards more about how they practice and how long they practice. But yeah, I’ve noticed an improvement. By having no one to play along with it’s difficult to improvise—you improvise in different ways—it’s a very valuable skill to be able to cut the changes… and me as a saxophone player, if I play and my colleague walks in the room they say, “Oh, you’re playing this tune” based on the solo I’m doing then I know I’m really cutting the changes. But when I don’t know a tune as well, it’s nice to have the harmonic progression so I can use my ear a little bit more than I can my mind and eventually, it’s a combination of both, I’d hope. But my students will then play along with the blues a lot more than they would if they were just doing it by themselves.

John: And much of the benefit is students can work and get up to a point where they’re ready to play with other students so that everyone can use their time more effectively when they actually do collaborate and meet at once. Rather than each person struggling just to do their own parts, they get to work out playing it together.

Trevor: Exactly, it saves so much time. In addition to that, when we think about rural areas. A lot of times you’ll be able to find a great collaborative pianist for doing your solo so it might be okay to use Smart Music—I hate to say that—or you might not have a chance to prepare so you play along with records—or you play along with, in the old days—and now you can play along with this. You know, even getting out and playing along with it at open mic night it’s just totally acceptable because it’s expensive to find musicians and especially in different places. Where in New York City I think would never see anybody using iReal Pro to play along with at a club, you know, just because…

John: …There’s a few musicians there, yeah.

Trevor: But I’ve heard of people pulling it out and it does irritate people. There is the idea that once you graduate from a jazz program—specifically, if you focus on jazz. I myself, did music education. I play jazz a lot and I also did classical clarinet, classical saxophone—but if you’re only a jazz major, or if you’re good jazz player, you’ll have like 150 to 200 tunes memorized. So you’ll call a tune, you’ll have the head, and you’ll know the chord progressions. But nowadays, a lot of the younger players are… from what I hear and what I hear from complaints from older generation people is that they’ll pull out the iReal Pro if they don’t know a tune and play along with it. And there’s pluses and minuses. Now you can actually play that tune. The minus is it’s not as ingrained as if you had to learn it and memorize it and ingrain in yourself, and that you won’t be able to interact with the high level players you’re playing with in the same way as if you did that work. So there’s, you know, again, pluses and minuses. It never should be a substitute for the ideal, but it should be a tool like you just said to get you there. Or in the cases of where there’s not a bass player to play with, at least it gives you a way to proceed until you can find one.

Rebecca: How do students respond to using these tools?

Trevor: They like them. They enjoy them. They think they’re really neat. This freshman student that came in—he was using YouTube, one of the play alongs he warmed up to it—but yet again, the disadvantage of this is all electronic and they sound electronic. So at least some of the “Learn to Play Jazz” are actually real jazz musicians. But you can alter the tempo—or you can’t—I guess you can with the 75%, 50%. I’ve never tried it, now that I think of it, on YouTube…but you know, it just it takes adjustment. You know, there’s nothing to substitute live musicians. The drummer can’t react to what phrase you just played. Nor can the keyboard voice omething different if you’re going there harmonically. But it is a tool. And yet again, a tool that’s not a substitute, but a tool that’s effective in preparing. The next tool we’ll talk about is actually on my iPad, and it’s Smart Music. Smart Music is an app which originally was on a computer and way back in the 90s when it came out it looked like the old Atari cartridges you’d buy with the machine for $2,000 and each cartridge would be like $80 or something and have one piece—Mozart’s Clarinet Concerto. But nowadays it comes on an iPad and you can also get it on your computer. The computer has more features on it but obviously is not as portable as the iPad. And what it does and what it’s being used for is amazing, but also can be negative and we’ll talk about that after we talk about the positives. It has all the band music that you’ll want to play in public school and what the band director can do—including myself as a band director—is that I can say, “I want you to play Holst’s ‘First Suite,’” is which we picked here, and gives you the saxophone part but I can also push the button as you can see, and I can pick whatever instrument I’m playing. So I’m playing bassoon, it gives you the bassoon part. It has the music in front of you and then once I play along with the rest of the band without that bassoon part or that saxophone part, it has little X’s and O’s and tells me what percentage of the music I got right rhythmically and everything else. So band directors will use this to say, “I want you to record this and get it to me and then that’s your percentage on it.” Now it does have some flaws. I’ve listened to students play and myself play and it’s no way that I’m getting 60% I’m getting closer to 85 or something like that. So some advantages and it’s gotten better over the years.

Rebecca: So just to clarify, it’s like Guitar Hero video game?

Trevor: To clarify, Guitar Hero has nothing to do with guitar.

Rebecca: Right. Yeah.

Trevor: It’s more of a rhythmic pushing buttons…

John: Pattern matching, yeah.

Trevor: Pattern matching and it has something to do with the music because it’s actually, you’re hitting it rhythmically along with the song, that’s about it. But you’re right, it’s like a play along, it’s like karaoke without the part you’re on. But this karaoke, in this case, records where your notes are wrong with red and green, where you’re behind, and it tells you where you’ve missed things, and then that goes back to the band director and they can grade you accordingly. So it saves them time.

Rebecca: The visual cues are similar is really what I’m…

Trevor: Oh I see what you’re saying. Yeah, I haven’t I haven’t played a lot of Guitar Hero and I apologize. Most of my knowledge of Guitar Hero’s from South Park and we won’t talk about that. So it allows you to do all those different things and then record them. As far as soloists are concerned, that’s where I use it more. I use it more to play along with a collaborative pianist that’s not there. And it follows you. So for instance, I’m going to pick the clarinet Copland concerto that I’ve prepared for, and it’s got a piano reduction to it. It’s a very difficult piece to put together when you’re a clarinetists like me, who’s a saxophone player that also plays clarinet well. So I use it a lot of time to prepare before I meet with my collaborative pianist. And I’m going to pick an arbitrary section in the middle so you can hear the piano part. And the piano part will play and then once I play along with it—and I’m not actually going to play—it will record it, and I can play it back so I hear all my mistakes which is really nice, and also my pitch errors and stuff. And then if I am unsure of myself, I can actually set it so that it will play with the clarinet part along with me so that I can hear that part to make sure I’m rhythmically accurate with it. That comes to one of the problems with being able to play along with something that has 100% rhythmic accuracy, but you don’t. You know, our ear can pick up on things so quickly that… let me give you the example. When I was in high school, there’s this lady named Amanda and she was the first-chair alto saxophone. And Amanda would have the rhythm perfectly the first time and then me, who would after listening to her having it perfect the fifth or sixth time, so I didn’t really learn how to count properly. And by playing along with the Smart Music, where the piano is actually playing your saxophone part or clarinet part, and you’ve been listening to that. Then when it comes to actual real performances, if you play along with it all the time, then you’re relying on it so much and your rhythm goes out the window. So that’s where some technology can get in your way, specifically with rhythmic attributes, but also with other things too. But yet again, it’s a tool and if you recognize that, you know, I prepare it rhythmically with the students first, I don’t let them use Smart Music until they’re at almost the performance level. So we’re going to play a little bit of the Copland Clarinet Concerto somewhere, but 140 measures into it. I’m going to play it initially with the accompaniment but with nobody playing, and then I’m going to set it and you can hear the difference now with the clarinet part being played by a different sounding piano.

[COPLAND’S CLARINET CONCERTO PLAYS ON PIANO]

So that was just the piano alone. Now I will do it with the clarinet part being played by a secondary piano, and you’ll hear how it fits in and that’d be what I’d be playing along with if I needed the rhythmic help or I wanted to check my pitch.

[COPLAND’S CLARINET CONCERTO PLAYS ON PIANO WITH SECONDARY PIANO]

That wasn’t as obvious an example but if you knew the part you’d be able to hear there’s extra additional things added to that which should be the clarinetist. So this tool is just remarkable for that and as I said, for band directors in public school and at the university level too, it seems like an appealing thing. I just haven’t figured out a way to automate it or include it in my, you know, as part of the practicing routine for my students. And if they don’t come from it from the schools, it’s a little bit more difficult to implement. I’m not sure at the university level it’s ideal for me to implement. I’m still wrestling with that as I am with implementing any technology.

John: So how have your colleagues responded?

Trevor: One of my colleagues, Rob Auler, he uses it as much or more than I do, we always check out, “Hey man, check this app out. Do you think it works really well?” So we check out different apps and sometimes those apps apply—or in the case of Rob, a tuning app wouldn’t apply for him because, you know, somebody else tunes his piano for him—but he really loves Amazing Slower Downer. He uses the iReal Pro probably less than I do, because he could collaborate with himself in his left hand while he’s playing solo lines in his right hand. And he doesn’t use it as a tool like Smart Music because, you know, pianists are not usually playing along with other pianists, they’re the collaborators. And we’ve talked about that, too. He knows it’s not a tool that can replace a pianist. But when we work together, Rob is a consummate musician and so he learns things way faster than I do. His mother would feel the TV to see if he was practicing five hours a day when he was younger, where I was probably you know, playing out in the backyard with dirt or something. He learns things so much faster and he doesn’t want to spend time rehearsing. So for me I’ll prepare on Smart Music and I’ll prepare on these other apps and get to that point where now I don’t have to spend so much time rehearsing with him. We’ll just do it once or twice like professional musicians do and he’s always been that way. For me, it’s upped my ante and I’ve really enjoyed that. So we’ll talk about all kinds of technologies, he’s huge into it. Other colleagues, for vocal people, they prefer having a pianist collaborate with them—which makes total sense—so it’s not as prevalent and I understand that. But most of them are not against, they just don’t understand how to tie things into doing it because what they’re doing already is fine. And that’s one of my philosophies too: If something is working, and it’s not going to be any better by adding the technology, there’s no need to add it. But if it can be an efficiency, the fact that usually in history—and as I got older, I realized this—that convenience wins out. And if I look at audio recording, you know, tapes. Small tapes were horrible audio quality compared to vinyl and CD had finally came up to a very good audio quality, probably not as good as vinyl, but most people can’t hear that spectrum anyways for those people that believe it.

John: And usually it’s better the first couple times you play the vinyl. After, the needles have worn down the grooves, unless you’re reading it with a laser, the quality of vinyl degrades really quickly.

Trevor: It does. And then MP3’s everybody thought, “Oh that won’t stick,” you know, everybody kept to analog, and really, the quality doesn’t sound nearly as good. But most people are listening through cheap headsets and on phones and using it for different background thing. Most people are not audiophiles and most musicians are and they think that technology is not going to last. So unless there’s a technology that other than convenience as a musician that I think benefits my students, I don’t really buy into it too much. A lot of people have embraced it and for different reasons. I think everybody owns a tuner on their phone. Everybody has a metronome on their phone. My colleague Juan La Manna uses the digital recordings and visual recordings of Zooms and stuff like that for his conducting students to record them and give them feedback on it and he uses it to record himself on concerts and a lot of technology being used, but not the Smart Music so much outside of wind players, I think. And it’s also ingrained in the band world. Jazz musicians have always used technology as far as like play along recordings with Jamey Aebersold and later on so it’s only a natural progression to use something like iReal Pro, I think. But you know, most of my faculty enjoy technology when it’s beneficial if it doesn’t help them out, there’s no need to learn it.

Rebecca: So I always wrap up our podcast by asking, what’s next?

Trevor: One of the things I’ve held off on and my colleague Rob Auler has not is reading off of tablets for music instead of bringing paper along with them. So for him it made a lot more sense, because he can use the AirTurn Duo Bluetooth pedal, and he doesn’t have to rely on anyone to turn his page. It’ll take them a little time to practice it. You can actually write and notate things in there if you’re in the middle of rehearsal, you know, we want more piano here, or we listen to the cello or whatever. But for me as a single line instrument, it didn’t make that much more sense other than the convenience of it, which we’ve talked about, which is to not have a lot of music in your backpack. But then a few things have changed over the course of the last few weeks that made me think I need to do it for my students and myself. They should know this technology. So I went to a rock gig—I hadn’t played a rock gig for 20 years at a wedding one of my friends asked me to do—The bread, the money was pretty good so I decided to do it for the first time in a while. And most of the people were playing on that, I had this like four inch binder that the guy gave me. He wouldn’t give me his iPad, and I played there and then I did a soloist with the Syracuse Symphoria and I was there just watching the orchestra play in one of the pieces and the harpist… and harpists are one of the people that are not known for…you know, they play an antiquated instrument…. It’s a gorgeous instrument… But she was using an iPad. Ironically she was reading a hard copy book while she was waiting for her turn to play in different pieces [LAUGHTER] so I thought that was ironic but she was using an iPad and I thought, “Oh, maybe I should do this.” And also the advent of the iPad Pro, where it’s large enough. You know I tried it when Vornhagen came here when I was doing some pieces, and I wrote it and sketched it and put it on the iPad, it just wasn’t conducive to reading with low light. And then Rob mentioned something to me that I thought would be the ultimate reason why to get one. He goes, “Wouldn’t it be nice if everybody played off the score?” By that I mean piano score that has a cello and the clarinet part there and we could all take a look to see vertically what’s going on as opposed to what we normally play off, the cellist has a single line as do I. Obviously the problem behind that would…we have to like learn how to read music differently. Instead of going from one line to the other, I’d have to skip them things, but I’m a conductor so that’s not a problem. But it would actually save a lot of time in rehearsal if we got off, or more importantly, if we want to phrase something together or we’re talking about something like, “What happens there?” I’d have to go over and look over Rob’s shoulder. So that to me, outside of the convenience of it, and that I probably should get back into the 21st century and actually get playing off them so our students know how to do it. I think it’s something that my next step is going to be starting to do that. And yet again, you can edit it. You can draw on it. You can put in repeats. It just seems so much more logical to do. I put in for grant for this and at the end, they’re evaluating it and they asked me specifically, “What would you do if you didn’t get this?” and I said, “Probably use paper” other than than what Rob just mentioned to me right after that session where it’d be great to read off the score ourselves. The advantage of it is more mobility and portability. These fake books are two and a half to three inches thick and there’s three or four of them and he has them all on his iPad and I have to slug them around with me, so I have nothing against paper.

John: But it’s a whole lot easier carrying around an iPad where you can have tens of thousands of scores with you.

Trevor: It is. And a battery pack too. That’s the other thing, if you don’t have a way to charge it and stuff like that, but that nowadays that doesn’t seem like it’s a concern at all.

John: And iPads generally got 10 or 12 hours at least of battery life.

Trevor: Right.

Rebecca: Well, thank you so much for joining us, Trevor. It’s been really interesting to hear you talk about your practice with mobile technology. And I know for me as a visual artist, it was making me think about the kinds of tools that we use as a visual artist and kind of translating the kinds of activities that you’re thinking about with students to different kinds of activities that I could do with my students and also thinking about how to use those to kind of push students just a little bit further.

Trevor: Nice. That’s great.

John: Thank you.

Trevor: Thank you, appreciate it.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandla, and Jacob Alverson.

65. Retrieval Practice

Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, Dr. Michelle Miller joins us to discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Her academic background is in cognitive psychology and her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications.

Show Notes

  • Miller, M. (2014). Minds Online: Teaching Effectively With Technology. Cambridge, MA: Harvard University Press.
  • Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.
  • Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968.
  • Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 1199327.
  • Kahoot
  • Bray, Niki (2018). 43 to 0: How One University Instructor Eliminated Failure Using Gamified Learning. Blog post
  • Retrievalpractice.org
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Pennebaker, J. W., Gosling, S. D., & Ferrell, J. D. (2013). Daily online testing in large classes: Boosting college performance while reducing achievement gaps. PloS one, 8(11), e79774.
  • Pauk, W. (1984). The new SQ4R.
  • Thomas, E. L., & Robinson, H. A. (1972). Improving reading in every class. (a discussion of PQ4R)

Transcript

John: Retrieval practice has consistently been shown to be important in developing long-term recall. Many students, however, resist the use of this practice. In this episode, we discuss methods of overcoming this resistance and examine how retrieval practice may be productively used to increase student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today we’re welcoming back Dr. Michelle Miller. Michelle is the director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general-interest publications. Welcome back. Michelle.

Michelle: Thank you so much. It’s so great to be here today.

Rebecca: We’re so happy to have you again. Today’s teas are:

Michelle: Well, I’m drinking Coco Loco, which is a blend from a local tea shop here in Flagstaff, Arizona. Steep Leaf Tea. And Coco Loco is a lot like what it sounds like. It’s chocolate and banana. So tea snobs may scoff at my choice, but it’s wonderful.

John: IIt sounds good.

Rebecca: And I think I saw a nice silver teapot that was poured into a green and blue tea mug.

Michelle: Yup.

Rebecca: A nice tall one.

Michelle: That’s what I need.

John: And I’m drinking ginger, peach green tea.

Rebecca: I went with a Christmas tea today. So Michelle, we invited you here today to talk a little bit about retrieval practice. Can you first start with defining that for us and letting us know what it is?

Michelle: Right. So retrieval practice is essentially the act of pulling something out of memory. So that is, in memory research, what we would term retrieval. So something is stored in memory and we want to pull it out so we can actively use that information, have it in our conscious minds and so forth. And so we go through this usually very fast process called retrieval. So retrieval practice is specifically the act of doing it, and we contextualize that with learning. So, when I’m trying to learn something or I’m the process of learning something, to say, “Oh, what was that fact I remembered, or what can I say about this?” When we do that, it produces something that we can call the testing effect. So this is kind of the clearest example… not the only… but the clearest example of retrieval practice in action during learning is when we sit down to take a quiz, take a test or something like that. So all the excitement that’s happened around retrieval practice in higher education, and really in the rest of education today, is around this finding, which has been replicated many, many times: that tests are, as one person put it, not neutral events in learning. When we take a test on something, that has a very powerful effect on our ability to remember it in the future. So, really simplified down to its core, tests help us remember in the future; when we take a test it strengthens our memory. So that’s what retrieval practice is and it can, as we’ll maybe talk about today, take many, many forms in learning settings. And I did want to clarify too, this is something that I definitely don’t want to take any kind of credit for discovering this. This has been around and has been known about for a long, long time. Some of the big names who are associated with this: Jeffrey Karpicke, Robert Bjork. Roddy Roediger, there are quite a few really heavy hitting cognitive scientists and cognitive psychologists who have established this. But there are many, many of us who are out there trying to disseminate this to other teachers around the world so that we can all tap into the power of this. And I have done a little bit of work in this area with my colleague here at Northern Arizona University, Laurie Dixon, who’s another psychologist… and we teamed up some time ago to look at a very practical implementation of retrieval practice in an Introduction to Psychology course that we conducted some years ago and this is a course, you can imagine, where just trying to get students to perform even a little bit better is a big project. So we examined how even something kind of basic… it was very high tech at the time… but just basic web quizzes that came packaged with the textbook. We said, “Well, if we actually assigned students to do these as part of the course, and if they went through and treated these as opportunities to learn, not just assessments in and of themselves, would that have any systematic impact on course performance?” And we found that in fact, there was a significant improvement associated with that. So that’s kind of the landscape of what retrieval practice is, and why we’ve been so interested in discussing this in the psychology of teaching and learning.

John: In fact, I saw you present on that about 11 or 12 years ago in Orlando at one of the NCAT conferences and it convinced me to completely revise how I was giving my classes and it’s made a big difference and resulted in some significant improvements in student learning. Given that we know so much about retrieval practice. Why are faculty resistant to doing this?

Michelle: Wow, that’s a great question, and that is one that I have been really facing a lot these days in my own practice, talking to other faculty members, and disseminating this through some different activities I do in this area. It is easy for those of us who work in this area, cognitive psychologists in particular, but a lot of us who like you heard about this a long time ago. We’ve seen the power of it. We forget that to other faculty, this can be a very off-putting concept. And so it’s really great for us to think about why that is. And I always think that’s really good for me to kind of go back to that and say “Yeah, not everybody is sold on this idea. And there are good reasons for that.” So like with a lot of things that we talk about in teaching and learning, I think that these really break down into two neat categories: there’s the philosophical issues that people have with it, and then there’s more practical and logistical issues with it. So kind of tackling those one at a time. Philosophically, when people say, “Yeah, I understand about the research but this just goes against something that I believe as a teacher or how I want my classes to be. Here are some ways that that can play out. First off is this idea, that I’ve heard in one form or another quite a few times, and that is this notion of superficial learning. So “Okay, sure, there’s a study that showed that maybe people retain something a little bit better. But surely that’s not this deep learning, whatever that is.” That’s a concept that we all want. So do tests and exams… just testing… create just a superficial form of learning? And while, of course, I understand that, and I absolutely applaud faculty for really thinking deeply about that issue, and caring about it. I Here’s the thing… we got to define that. We social scientists, that’s what we do… we have to kind of break things down and say, “Okay, what does deep learning mean?” and I don’t know that anybody has kind of definitively done that. But when I look at that, I say, “Well, this is not just one research study that showed a little improvement in a lab test. There’s quite a few studies that do use realistic types of materials. It’s not all just contrived laboratory studies. Furthermore, there’s also studies that show that when students engage in more quizzing and testing on material that they actually are able to transfer that learning better. And that is a very, very big deal in teaching and learning as a lot of us know is not just getting students to be able to solve a problem in one context or work a concept in one context, but can they do it in the next circumstance? And that very difficult process is aided by quizzing… and to me, what could be deeper learning than learning that transfers? So that’s part of it. Some of it is perceptions around multiple choice quizzes and tests. There’s an assumption too that if we’re talking about quizzing we must be talking about multiple choice questions… and first off, sometimes in larger classes, those are the final assessments… in that Introduction to Psychology course that we studied years back, that’s what the assessments were… so, having students practice in that format I don’t think that we should necessarily dismiss that. And as we can talk about in a little bit, there’s lots of ways to induce retrieval practice that actually don’t involve multiple choice questions. So, there’s a bit of that as well. And something that I’ve talked about with some faculty recently, too, is this baggage around K through 12. And maybe that’s something that’s resonant with you all.

John: Yeah, that’s given the testing effect a somewhat bad name, because high-stakes testing is being used in a lot of what’s going on with K to 12. But that I don’t think is what retrieval practice as you’re suggesting is all about.

Michelle: Right, and I have to be cautious here. I really like how you laid that issue out in K through 12, that there is a reputation problem… and that has happened because of the high-stakes standardized testing policy in the United States. And I got to be careful because I don’t want to represent myself as an expert in K through 12 or in K through 12 education policy. But I don’t think you have to be an expert in that to know that there’s been a lot of same pretty well justified public pushback against over-testing in K through 12. And yeah, I think that we absolutely do have to be aware of that. Students come to us in higher education… that’s a system that many of them have been through and our faculty are very aware and very cognizant of that too. So, nobody’s a blank slate here, not our students, not our fellow faculty. We have assumptions and ideas and experiences about testing that happen. I think those can be addressed. But, yeah, that is definitely another very big barrier. We got to differentiate between high-stakes standardized testing for the reasons it’s done in K through 12, and low-stakes testing and quizzing for learning as proponents of retrieval practice would have it.

Rebecca: Some of the pushback I’ve heard from faculty fall into two categories related to this as well. One is that they assume that retrieval practice is best implemented in 100, 200 level introductory classes instead of upper level 300, 400, graduate-level classes. And then the other area is that paper and pencil tests don’t make sense and all disciplines. And so they assume that a test has to be in a paper/pencil format, which could be online testing, or it could be multiple choice or it could be essay questions. But I think that, from being someone in the arts, like there’s other ways to test beyond that, but we don’t think of those as tests.

Michelle: Right. That’s another great lens through which to look at this issue; that we do need to broaden the definition to draw more attention to this and to make it a more appealing concept. But yes, how can we make it broadly appeal across lots of disparate disciplines? Not only does it not have to be a multiple choice type of exam, maybe it’s not a pencil and paper exam at all. And we as faculty have to think about what makes sense there. You make a good point about the levels concept. I think, these days most of us have heard of, “Well, there’s one particular Bloom’s taxonomy…” which is a wonderful framework for getting us thinking about being systematic about what we’re asking students to do with the information that we’re bringing to a course and trying to do things like align the teaching we do with the assessments that we have. That’s wonderful. However, I think it does ingrain in us that idea that “Well, just knowing things is sort of at the bottom… You sort of get that out of the way, and then we go on to the good stuff.” And from a cognitive perspective, these relationships are much more fluid and much more interdependent, so that yes, absolutely, the higher thinking that is what we want. That is what we should want. Or if we’re in highly applied disciplines (if we’re in the arts, for example), we need students to be able to do things with that information. But they have to have that. So I think it challenges us to think of new ways with that concept as well.

John: One of the barriers I think some people have is they don’t like to grade tests and so forth. But one of the things you mentioned in your book is that the testing effects been known for a long time, but it was really difficult to implement in terms of low-stakes testing, particularly when you’re teaching at a larger scale. But, as you’ve done yourself, and as you suggest in your book, computer technology makes it easy to automate some of this… certainly more easily for multiple choice and free response and similar things. But it makes it a whole lot easier for both students to have multiple attempts at learning something using some type of mastery quizzing and it makes it a whole lot easier for faculty who don’t have to spend all their time grading.

Michelle: Right and that’s absolutely where the practical stuff comes in. So, we’ve worked through some the philosophical objections, that: “No, this does not turn your classroom into some terrible assembly-line concept of learning. It’s not going to create a bad relationship with your students. It’s not going to simply carry on a legacy of bad policy as people perceive it. It’s not going to do those things.” And then, we do get to: “Alright, but what is this going to do to my life as a faculty member?” …and this is important stuff. Those who know me know I’m a big fan of James Lang’s work in his books. One of his more recent books is called Small Teaching and there’s a lot of different takes on it in that book, but it really hits home with respect to “We do have to think about not everybody’s in a position to, nor should we even try sometimes, to just take everything down to the foundation and rebuild.” That we do really need to think about “Do I want to do this, if it’s going to create 800 more questions for me to grade?” Is this a sort of a situation where, “Well, I don’t want multiple choice, so I’m going to have to give these open-ended questions. I’m gonna have to give feedback and I have 200 students, what will happen? If I am going to go with multiple choice questions… well, how am I going to do this? to have to write all of this?” And yes, it’s one of the most powerful outcomes of the educational technology revolution that makes it workable, and scalable in a sense, even with large classes to do these types of things and to bring them in. So, that is definitely a message that I hope faculty think about if they’re on the borderline of wanting to bring in more retrieval practice into their classes.

Rebecca: I’m in a discipline where the multiple-choice questions are using things digitally doesn’t always work for testing and practicing some of these basic things, and there’s not a good way to automatically grade it. But one of the strategies that I’ve used is actually some self grading, which has actually worked pretty well. I just check and I have them write notes about anything that they got wrong. So, it demonstrates that they’ve tried to understand when we go over it, and I give credit based on how thorough those notes are, rather than whether or not they got the question right or wrong. And that’s made a difference in my classes. And had I not come up with that solution, I think I would have abandoned it because it would have been too much work. But it it actually is working pretty well.

John: When we had a reading group on Small Teaching last year, one of the things that was widely adopted by faculty was a very simple form of retrieval practice where they had students at the start of each class reflect back on what they had done in the previous class. And most of them have continued to do that in subsequent classes as well.

Rebecca: One of the other barriers that faculty might raise is the idea that students don’t take low-stakes things seriously, or that they don’t put the same kind of time into it that they might for something that’s high stakes. Can you talk a little bit about how we might help students find value in retrieval practice and subsequently also with the faculty then?

Michelle: Right, so that, ”Well, we can put it out there, but will they do it?” I’ve kind of crossed this philosophical and practical barrier for myself of giving some credit for pretty much anything that I am hoping that students will do. I put the work in to set it up and I do believe as a teacher that there’s reason to believe that will help their performance, that I need to work it into the syllabus somewhere. I don’t think it detracts from learning, necessarily, to say, “Yeah, there’s some points associated with this.” And especially with our students who, for many of us, are going in a million directions at once. They’re juggling jobs, multiple classes, sometimes their own families. So having an incentive in the form of points—having some kind of a payoff—I think, helps them make that decision that this is at least worth the time to do. I think the other thing that we probably can all do more of, and that I’ve done more over the years, is framing and honestly marketing this to students… communicating with them about why. And when students disengage from an activity like this, when they say, “Ah, why do I have to sit down and do this thing? This is just another test. Oh, no.” …really conveying the excitement and the goodwill that we have in setting those things up can go a very, very long way. Of course, a student, if they just look at it inside and they have no context for why this was put into place, they’re going to have them say, “Well, maybe I won’t do that.” But when we can market to students, we can say “There is a lot of research that shows that this is a very, very good use of your time. And hey, you’ve probably taken a lot of tests in your life that were really about measuring or sorting you and figuring out what you know and what you could do. This is a very different kind of test.” So that can go along way and get the C students nodding: “Alright, I get it. I get why she put this assessment right here.” I think a lot of us have hit on the practical strategy too, that the little Easter eggs or goodies that we plant in the form of questions that get re-used on the higher stakes assessment. So most of us will have tests for measurement at some point in our courses. And yes, students really do pick up on it when you use one, two or more of those items that were in, say, the gamified quiz that you ran in class or the reading quiz that they did beforehand. They can see those and say, “Oh, wow, I got feedback on that. I got an opportunity to practice…” and if it draws in a few more students who see it as almost a legitimate form of cheating, honestly…. like a fun and sanctioned form of getting an advance sneak peek at the exam, then great! Then that maybe is an opportunity for them to come in and see that. So there is that. Actually taking it seriously ourselves, not just in the form of saying, “Well, here’s the points I’m going to give you for this,” but spending class time on it. That’s a big bridge for a lot of us to cross, right? Because we as teachers tend to be very focused on “Oh my gosh, class is for covering material. We use all these sort of distance metaphors to talk about what we want to do with our class time. But if I say, “You know what, guys, I believe in this, and I believe in it enough to where we’re going to spend the entire class period before the final exam….” I did that twice last week myself, when running exams. Or we’re going to spend five or 10 minutes of the beginning of every class period doing this, as one project recently published about doing. If you show yourself doing that, and offer them that, I think that also goes a long way towards it. And I guess to just say, well, taking it seriously… here again, what does that look like? What does that mean to different people? And we can kind of a little tongue-in-cheek say, “Well, why do we have to take it so seriously?” Sometimes games and learning can happen when it is presented in a more fun context. So not everything has to be deadly serious or spending hours and hours and hours of stressful time on. There are occasions when a light-hearted approach can be perfectly good and can still get us involved in that really critical activity of retrieval

Rebecca: I can share an example of doing that in my classes. We’ve done design challenges and sometimes we challenge other classes that are happening at the same time. That reinforces some of the basic principles that we think that students should be doing and reminds them of it… and they might work in a team…and then we have a competition. And it’s fun and what have you. And students like those, it breaks up the day, it makes it more fun. And then I’ve also done things where I give class time to do little design challenges in class that might be individual and then they can level up to working with a partner to finish solving a problem or something… and students value that. They recognize that next time they’re trying to do a project on their own,that it’s easier because they’ve had that practice or that opportunity for the retrieval practice. And my students have actually ended up asking for more of those opportunities.

Michelle: Great.

John: Could you go back just a little bit and tell us what you did in those couple of classes last week before your final. We’re recording this, by the way, in early December during finals periods in both of our campuses, but we’ll be releasing it a few weeks later… to put that in context.

Michelle: Oh, okay. Well, I’d love to. …and I’ve been talking about retrieval practice as you pointed out for years, and I still discover new ways to infuse this into courses. And the context for this is my cognitive psychology undergraduate course. It’s a 200 level. So it’s a lower-division course. And it’s about 60 or 70 students. And as you can imagine, it is a bit of a tough sell. For many of the students it’s their first encounter with this side of psychology. It’s not as intuitive as some other areas of psychology. So there’s a lot to learn and a lot of motivation to be done. One of the ways we bring this and in this course is using a technology called Kahoot, that’s spelled K-A-H-O-O-T,, and it’s really very intuitive and functions very smoothly… relatively free of bugs. That’s good stuff. It’s a program for doing gamified quizzes of various kinds. What I did, and at different points in the semester, and then really amped up in the last week of the semester is running these gamified quizzes. And this is something, by the way, that I hit on and got the idea to try based on a colleague named Niki Bray, who’s from Tennessee, and has actually done some really systematic work in reformulating some of her courses around in class quizzing in just really ingenious way. So I saw some of her presentation and I said I’ve got to try this for myself. I went with multiple choice questions. Kahoot does have some parameters… questions do have to be short…very, very short. And to some people that may be off-putting, but you can put together quite a few of these. And so we would put this up on the projector and students have the option of dialing with either their laptop or their smartphone and weighing in on each question. The neat thing about it is it has an algorithm for giving points based on your speed as well as your accuracy and it’s got a little leaderboard so you can actually have a little in-class competition. Now some people who use this do require all students to do it and they actually issue points for performance. Now, I presented it very much as a practice activity… and made it very, very clear because of my philosophy, I’m not going to assume that all students have devices or have smartphones or laptops or that they want to do that. But I said, “Look, remember we’re talking about retrieval practice, guys. So the real meat and potatoes of this is not buzzing it on your phone. That’s fun. But the real benefit of this activity is what you’re doing sitting there in your seat. And you could be doing this with a piece of paper if you want.” And that is what a few students opt to do. They try to answer the questions, they know what they need to go back and review and so on. And it’s nice because it spits out at the end, a whole report that tells me right away… Okay, which questions do we need to revisit? Which ones did students have the hardest time with and so on. That’s one of the things that I just brought in. And yeah, it was a big deal. I sacrifice a chapter of “coverage” so that we would have more time at the end of the semester. But to me, I would rather have students going into the final knowing that they’ve had this retrieval practice and they have a better chance of performing really well. Earning a good grade on this material I care about than honestly cramming in a little bit more mileage in terms of the quantity.

Rebecca: Sounds like fun.

Michelle: You know what, it does really bring a fun factor. There’s been a lot of different variations on in-class polling, and I will admit to this, I actually purchased and am the proud owner of a physical buzz-in quiz device, complete with a whole spaghetti nest of wires and an incredibly abrasive, buzzer sound, and everything. So previous to this, my educational technology did include… I think I could have up to eight intrepid volunteers who would play a quiz game and then I would have to appoint a points keeper and all this… and props are always a lot of fun. So when I say I believe in bringing in retrieval practice, I really do walk that walk. But I will say doing it by a smartphone does allow for more participation, and I don’t have to worry as much about the minutiae of scorekeeping and stuff like that.

John: I played with Kahoot a little bit at my classes at Duke and students have loved it.

Rebecca: We’ve talked a little bit about framing things so that students take the practice seriously. What do we do about the students who just push back, it’s like “This is too much work. This is a lot of extra time…” or that sort of argument.

Michelle: I think that that’s another piece of that barrier to more faculty adopting this, not just the work involved, but realistically, student opinions. Student evaluations matter a lot to faculty life. And of course, we all want to have that wonderfully rewarding semester, not the semester where we feel like we’re at odds with our students. So I do think a little piece of this is we do anticipate sometimes worse and more pushback than what actually happens in the end. So I think we have a fair amount of sort of a dread factor when we go into something new like this. But that said, when students have an issue with more quizzing or more testing, here’s how those come out. I think, first of all, we do have to sometimes separate out the technology aspects of it from the quizzing or testing, per se. So, as John mentioned, this is one of the amazing things that educational technology does. But the flip side of that is, if you’ve ever used technology for education, in any shape or form, you know that it breaks down. And when it breaks down at a big class, that’s a headache for everybody, it’s a misery. So if you’re trying it, and things are not working out, you just got to figure out “Okay, how much of this is the assignment, the activity per se, and how much of it is that wonky quizzing thing that I got from the publisher and it fell apart, or students hanging up when they’re trying to dial in with the poll and fine tuning those. I mean, when I first used kahoot, I decided it would be a lovely idea to put four chapters worth of material into a 40-question quiz. And when you get used to as you know, that they took a long time, I mean, 12 questions can keep you going for a very long time, especially if you’re discussing… and some students got kicked out part way through and then they weren’t on the leaderboard and I could have set the whole thing aside. But really, that was more I needed to get the technology working. So that’s a big piece of it. I think that sometimes it is a perception issue with the timing. Now, those of us who are just all over this as a teaching technique, we like to do reading quizzes before we talk about that material in class. So chapter three is up on the syllabus and your chapter three reading quizzes due on Sunday night before we do that…, and that can provoke a fair amount of confusion and honestly griping with students. They say, “Why did I get tested on this when we didn’t do it… we didn’t cover it in class yet?” And what do you know, that’s another framing and communications issue. Once we know that that is why we’re doing that you get so much less on that side of things. But here too we do also ourselves have to follow along with what we say. If we say you’re doing this reading quiz so that we’re establishing a foundation. we don’t have to teach everything in the class itself. We could spend the time applying. Well, guess what, then you do have to do that. So if I assign you to do the reading quiz on chapter three, and then on that Monday morning I go, “Okay, we’re going to go over chapter three starting at the top. And I’m going to show you all the PowerPoints for all these things that you already read.” Well, yes, then student morale and student support will fall apart at that point.

Rebecca: Those are very good reminders.

John: Yeah, I’ve pretty much adopted this approach all through my class beginning when I first saw you present it. So I have students do the reading in advance, I have them take reading quizzes with repeated attempts allowed on those. And then in class, I have them working on clicker questions, and I and the TAs go around and help them when they get stuck on problems. But there is an adjustment and students, especially when I first started doing it, would generally say, “But you’re not teaching me” …and you do have to sell students on this a bit. One source of resistance is that when students take quizzes, they often get negative feedback. When they read something and they read it over again, it looks more and more familiar. They’ve got that whole fluency illusion thing going and they become comfortable with it, and they feel that they’re learning it… or similarly, if they hear someone give a really clear presentation on a topic, it feels comfortable. They feel like they’re learning it until they get some type of summative assessment where they get negative feedback, and then they feel the test was somehow tricky. But it’s a bit harder to convince them that actually working through retrieval practice, watching videos at their own time and pace, reading material as needed, and then spending class time working through those problems is as effective. I’m getting better at it. But it’s been a long time trying to convince students and I did take a bit of a hit in my evaluations, especially the first few times.

Rebecca: Do you mean learning’s not easy?

John: Students would like learning to be easy.

Michelle: You know, it’s funny, I think almost in a way… see what you think about this idea. But it’s almost a mirror image of our illusions as teachers, right? That I gave a wonderful clear lecture and I assigned wonderful readings and I saw students highlighting them. Therefore, students must have assimilated this knowledge and it must be in there. So I think it challenges our students but it also challenges us as well. It can be quite an eye-opening experience to running something like a Kahoot. And that brilliant point that I gave this great example for…. what do you know…. 7% of the students actually nailed it. And so we can all use a reality check… teachers and the students. And you mentioned re-reading, that’s another one where I’ve had some pretty intensive conversations with other faculty, I’ll kind of say, “Oh, well, and there’s this great research that shows that students tend to re-read when they’re studying and we know that from a memory standpoint, that is really, really ineffective.” And faculty will say, “Whoa, whoa, whoa.I want my students to be re-reading.” Of course, now when they say re-reading, they may be picturing deeply interrogating a text… annotating it… looking at it from a different perspective. And absolutely, that’s a wonderful part of scholarship. But that isn’t what we’re talking about. We’re talking about students re-reading as a study form and mistaking highlighting for deep interrogation. So, just like with the rest of knowledge bases versus higher-order skills. This is another words “both and” it cannot be “either or.” Yes, we want students re-reading in the right ways, but students or teachers cannot mistake that for learning sometimes.

Rebecca: So, let’s say you’ve just convinced all of our listeners that we need to be doing this, how do we bake it into our course designs?

Michelle: Well, I think that, really getting creative with this, and as I talked to faculty when I visit other schools or talk to faculty of my own institution, I just see all of these new ideas all the time about how to do this. Once you do get that critical epiphany of alright, a test doesn’t have to look like a test on the surface… it does not have to be the ritual of “Okay, you’ve got a number 2 pencil that’s breaking and I’m standing over you while you have a panic attack for an hour.” That’s not what it is—that it’s about the retrieval. Once you get that, all kinds of creativity opens up. So I’ve also started sometimes on the very first day with the syllabus quiz. So, especially if you have a smaller class, we all struggle with that “I spent all week writing the syllabus and it’s incredibly important, and then I’m getting questions about stuff that’s on it all the way through the semester.” So, really on the first day, what I do is I take a very light-hearted approach and I divide the students up into teams, just physical teams. Everybody’s got a copy of the syllabus, sometimes I fake them out and I say, “Okay, we’re going to go over the syllabus point by point…” and I say, “No, you can read the syllabus. We’re actually going to do this other thing which research shows will actually help you remember it.” So the task here is that each team formulates a set of questions… you can make just a few. like three… so a little bit of teamwork. So formulate three good questions off the syllabus. And you’d be amazed, they come up with questions that even I can’t answer sometimes without having to cheat. So they talk about it. And then of course, you go around the room in some arrangement and each team gets to ask the other team their questions, and if you really want you could keep score and everybody loves bragging rights for being the team that stumped the other teams and won the most points. So, it can literally start right then. With that idea that I don’t read stuff to you in this class, this is about you. And I’m also not piling a huge amount of work on myself either apart from being the moderator and the MC and having written the thing in the first place. I don’t have to write questions. They’re writing the questions and they’re answering them too. So those are some of the ways that we can do that. I think especially in our larger classes we do want to think about things like peer grading or peer review of open-ended question responses. We do want to take advantage of things like publishers’ test banks to set those up as reading quizzes. And you’d mentioned earlier about “Well, what about this not being suitable for upper-division classes?” I had an upper-division class that just wrap this semester where we had reading quizzes as well. It may have been an upper-division course but it also serve that purpose of “Hey, you take a basic quiz over the chapter on Sunday and that really sets the stage for us to have a more substantive discussion.” All of those things. are ways that we can do this. I think open-ended reflection as well… so tests that look nothing like a test but are still retrieval practice. You’d mentioned about reflecting on what you learned last class period. So this sometimes goes by the name “brain dump.” And I did a bit of this last semester as well. So, this by the way, I do want to credit the great website retrievalpractice.org. So retrieval practice is actually that high profile it has its own website. It’s an absolute treasure trove of ideas. So, with the brain dump the way that I did it, or similar to what it sounds like you did, every now and again we start off class with you writing down everything you remember from last time on a piece of paper. I had students then turn to their neighbor and compare notes and see what they came up with. And I didn’t grade those. This was not a heap of grading for me. But in the end what they did turn into me for accountability and a few points was a very short reflection, just on an index card. So, “What surprised you?” and I review those and they’re very eye-opening but again I’m not there to police or micromanage what they put on their cards. So, that’s retrieval practice too. There’s lots of different flavors that we can bring in.

John: I do something similar with asking students to reflect on the reading in each module we work through. But because I teach a fairly large class, I didn’t want to have to deal with all the index cards. So I just have them fill up a simple Google form. And then I can skim through it and assign grades much more easily than shuffling paper. But it’s the same basic idea and it’s worked quite well.

Michelle: Absolutely.

Rebecca: Do you have any other examples of the kinds of little tests that faculty can run that don’t look like tests?

Michelle: Right? So tests that don’t look like tests. Well, here’s another that was probably a little bit more practical for a small class, and this is how I ran this. It, like the brain-dump exercise the way I ran it, was also very cyclical and very student generated. So we started out each class at the beginning of each week with students generating a set of quiz questions based out of the assigned reading. So we didn’t actually, in this particular example, have pre-quizzes or something like that. But students came in knowing that they can bring their book if they wanted. But they would need to sit down and write for me three questions, whether short-answer or multiple choice. Then I can flip through those and I would really quickly put those together for a quiz that went out the subsequent day. So, we’re alternating between generating questions and then returning to those questions. And then I would have pass out. I told them treat this like a realistic test. Actually try to retrieve everything, but, you know, when times up, you’re going to get to go back to it. And then they would grade it themselves. So I didn’t actually do the grading either. And that is really great for spurring discussion. And “Oh, my gosh, I thought I knew the difference between reliability and validity, but now that I tried to answer it, I realized that I didn’t.” And then you can throw it back out to the student who now has bragging rights for having had their question selected and say, “Hey, what did you mean? What was the right answer? And why did you put that down there?” And at the end of the day, they kept that quiz too. So it was really very much in their own hands to do. So there’s that. Other creative ideas that I’ve run across over the last couple of semesters… There’s a great project out there, run by Bruce Kirchhoff at University of North Carolina at Greensboro. So I got the wonderful pleasure of talking to that group last year. And he and some colleagues working in the area of botany actually put together a freestanding, custom built mobile app that students could take with them that presented different kinds of quizzes over the sorts of things that botany students really need to know like the back of their hands like how to identify different plants and how to discriminate among different examples and they found some empirical evidence that this actually raised performance up quite a bit. I’ve heard too of another Professor put together some surveys in Qualtrics, the surveying software took advantage of its ability to actually text message people and send them the questions. So this was an opt in sort of activity and it’s one that they didn’t just have it run 24-7 because it was a little intrusive, but what it did is it sent students questions that they could answer at different intervals throughout the day, which also takes advantage of another principle from applied memory research which is spacing. So students are getting these unpredictable questions, they have the option to answer them, and they could be happening even when they’re not in class. So those are some other ways. Some of them look like tests, some of them don’t. But those are creative ways to get students engaged in that practice.

Rebecca: As a faculty member, we often advise students and mentor students who might be struggling in other classes, ones that we don’t have control over. Are there ways that we can help those students use this methodology to do well in those classes where it might not be embedded?

Michelle: Right. And that’s so great that you bring that in as well because ultimately that is what we need as teachers and that maybe circles back to yet one other piece of objection that I’ve seen… actually this time in a published article from a few years back that said, well are we doing too much for students? Are we scaffolding them too much so they’re going to grow really dependent on these kinds of aids like reading quizzes, and reading questions. But if we also have in our mind very intentionally, that what we want students to have at the end of the day is also something they can walk away with, I think that we do have to be very mindful of like, “Okay, let’s not create the impression that just because retrieval practice is so important that you have to sit there and wait for me to put together this specific kind of reading quiz for you.” So I think here, the really powerful message is once again that one students take this to heart… once they’ve not just been told this, that “Oh well, you should quiz yourself as a study strategy,” but they’ve seen that I believe in it to the point where I’m going to put time, energy, and work into it as part of my class. And maybe they’ve even seen the results… they’ve now had their own little before and after experience of what happens when I do this… that they can be more likely to take this forward. So having more faculty across the curriculum endorse this is a powerful idea that “Hey, this is how your mind works. This is how your brain works. Your brain doesn’t just soak up stuff that’s in front of it. You soak up stuff that you have to answer questions about.” That I think is going to be a powerful message. And there’s actually another article out there that I just absolutely love it was done by a psychologist named Pennebaker and some colleagues at University of Texas at Austin some time ago. They replaced high-stakes assessments in their Introduction to Psychology course with these mobile in-class quizzes that were done every day. And one of the things that they report in this article is not just that students did better in that class, but that certain subgroups of students actually showed improvement, specifically closing of achievement gaps, really, in classes that they took after the psychology course. And these were classes that we’re not even in psychology. So you think about that for a minute. How does that happen? Now, nobody knows exactly for sure, because this was kind of an unexpected finding in the study was my impression, but a real possibility is that when students have sat in this class and every single day they have seen the power of taking a quiz… of spacing out their learning… and attacking it in a very active learning approach… Once they’ve seen that happen, they’re more likely to go home and say, “You know what? I can do this in my biology course too. I don’t have to sit there with the teacher’s quiz. But, if I attack it in the same way I might get the same results.” So I think with those things, we can have students walking away with that enduring practice that we want them to have. And it is funny too, because it brings up one of the, I’ll just say it was a really heartwarming teacher moment that I had this last semester when I did bring in these Kahoot quizzes. In the run up to the first exam, I had done my fancy little in-class quiz and was kind of patting myself on the back of what a great leader I was. So I came in to, I think it was the class right before this exam, and I come plowing into the classroom and it’s dead quiet. And there is a student at the front of the classroom. He has commandeered the podium and the computer and what has he done? He’s accessed the Kahoots, which I gave them the links. You know, they’re out there for all the students to see. And he is running them and the rest of the students in the class are taking them just as seriously as when I was administering them. So they’re running through the questions again, giving it another shot. I didn’t tell them to do this at all. It was one of these moments where I just backed out and closed the door after me. And I came back in when they were finished. And so I think those are the kind of moments that we can set ourselves up for when we really do bring this into our classes and we get behind it and a really authentic way.

Rebecca: I think one thing Michelle, in the way that I asked the question, I had also asked it from the perspective of a student hadn’t had the experience of doing a retrieval practice in a class. So, if you’re working with a student, maybe outside of class or as an advisor, are there things that we could do to help those students adopt those practices, even if they haven’t seen it modeled for them?

Michelle: So how do you help students when they haven’t had some of these experiences on their own? And I think this is a part of a bigger package of goals that I think a lot of us should have in supporting our students to really put students in the driver’s seat. To say, “Yeah, you don’t have to wait for a certain style of teaching or certain subject material in order to succeed with it.” I see that isn’t really part of a larger package of growth mindset honestly. So what can students do to make themselves the masters of this? Now, I think that there are some old standard, very traditional, approaches that are worthwhile. And when you look at those approaches, sometimes retrieval practice is at the core. So it may not be a matter of trying to get a very unfamiliar set of terminology or anything for students before, but really getting them to look at some of these approaches in a new way. So things like you’ve ever heard the term SSQ4R or PQ4R, there’s a couple of those that have acronyms and the things they have in common are that they tell students when you sit down to study or read a text or prepare for a test, here’s what you do. You don’t just start reading from the top with no goal in mind other than “Oh, I want to get a good grade later.” What you do is first of all, you set yourself up with questions. Which is, after all, a lot like retrieval practice. Start with a question, say, “What do I want to answer? Can I answer that now?” And if not, why not? To read your text or go through your material very intentionally around this questions, and then there’s always this piece of recite or review. That’s what one of those Rs stand for in some of those traditional systems. And that means closing the book. That means closing the book and saying, “Okay, I’ve sat here with this material for this amount of time. What can I actually say about it at this point?” So, sometimes directing students back into some of those and saying, you need to adopt these strategies, which are completely teacher-independent, they’re fairly discipline-independent as well. That can be good. If you’re doing them for the right reasons and with the right approach in mind, that’s very good. Encouraging students to take advantage of the publishers’ companion sites… Now this is a little bit more of an uphill climb. That is where we run into “Well, if there’s no points for it, Why should I do it?” But encouraging students to say “Look, you already paid for this textbook. To be crass about it, you paid all this money, did you know there’s a website over here and if you interact with those materials in this particular way that’s really, really likely to pay off for you?” So, besides just ensuring that our students have what I consider to be these basic foundational pieces of knowledge about the mind and brain: that we remember through testing, we remember through retrieval and an active engagement. All students should have that, but those are some specific things that we can counsel students to use across all their studies that really should pay off.

John: Do you have any other suggestions for those faculty that are thinking about expanding their use of retrieval practice?

Michelle: You know, just to really encourage and support faculty who are starting this journey, as it sounds like that you all have, to really re-examine how we can bring in this incredibly powerful principle, and to really reassure each other that “Yeah, this is not about really just piling so much more work on yourself.” We can sometimes even just re-examine the assessments and assignments that are already in the course and so that’s kind of one last piece of practice that I think that a lot of us can really stand to bring in. And that was a big thing for me as well to say, “Well, if I’m going to administer these tests… we administer tests, we administer midterms anyway… what else can we do to increase their value as learning experiences… as learning events. So here’s another where I’ve brought in various forms of test discussion activities. Instead of standing in front of the class with that deadly the day-after-the-test class period where I say, “Let’s go over it” to realize, you know what? That probably will not work that well. But one of the amazing things that we have learned from the research on retrieval practice is that it’s not just in the moment in the taking of the test, where this advantage to memory happens. It actually creates a sort of a receptive window for learning and for review when we’ve just taken a test on something. So if you’ve got a midterm in your class already, well, hey, why not carve out the time after that test is up to say give it back to the students… like I photocopy the exams before I’ve graded them. They have no feedback on them whatsoever. And I hand them back to students as a group discussion exercise. I say “Alright, here’s a blank copy of the exam, your group can fill out this blank copy together, just knowing what you know, revisiting all those questions, having those good discussions about what you understood and what you didn’t. And I offer a little bit of extra credit for really good performance on that. And there’s other ways to work that out so it actually takes off as a small group exercise in class. But regardless of the specifics of how you make something like that work, the spirit of it is the same. That when you sat down to do this, to try to drag all this information out of memory, that is not an end unto itself. It should be part of a bigger picture of learning in the class. And it’s sort of an untapped vein of potential we have as teachers and that our students have as well and that we can access it regardless of what the discipline is or how the class is setup.

John: …and students see it’s very relevant. I started doing something very similar in my econometrics class last year following a suggestion from Doug McKee who had been doing that in his class. And it worked remarkably well… and turned what was normally a pretty unproductive class period where we’d go over the test… and the people who did well with just be really happy and pretty much ignore any discussion, and the people who did badly were just sitting there unhappy and not really being very receptive. But when they sit there, and they’re explaining it to each other, it seemed like a really ripe time for them to learn the material much more deeply.

Rebecca: I remember the first time that you implemented that. You sent me a text message with a photograph of his students taking a test and it looked very active. [LAUGHTER}

John: …and they were having fun.

Rebecca: Yeah.

John: One of the nice things that came out as I was wandering around listening to their conversations, and I was hearing people say, “Oh, yeah, now I see where that came from.” Or someone would say, “Well, when did we learn this?” And then someone else would say, “Well, remember when we worked on these problems?” …and it just helped them make connections… and the power of peer instruction is so remarkable. I’m going to do it in as many of my classes I can.

Michelle: That’s perfect. That’s what we want as teachers, and that’s what our students benefit from.

Rebecca: So as you know, we always wrap up our podcast by asking what’s next?

Michelle: Oh, wow. So I am looking forward to kind of rebooting a course that I have not taught in some time. My senior capstone course in technology, mind, and brain. And that is a fun one. But it’s one that is going to take a lot of revision, since it is technology and things change so rapidly, as we all know. So that is going to be a big part of next semester. I also have a crop of research projects in various angles on teaching, learning and educational technology that I’m really excited to be moving forward in the next calendar year. One of those… really foremost among them is a project on virtual reality for learning. We have an incredibly creative and dynamic team looking at virtual reality here at Northern Arizona University. They put together an amazing series of interactive exercises that are part of the organic chemistry course here and teach some of the challenging concepts in that extraordinarily challenging gateway course. And so we now have a whole set of data from students who went through and did this at varying points during the semester. We got their feedback, we’ve got all different kinds of psychometric measures that we gathered from them at the time as well. So, I cannot wait to be tackling that and looking at all kinds of angles on how this part of technology is impacting student learning in this course.

Rebecca: Sounds like some exciting adventures.

John: That sounds like a wonderful research project. …looking forward to seeing what you find.

Rebecca: It was a real pleasure to talk to you again, Michelle. Thanks for spending some time with us.

John: It’s wonderful talking to you. Thank you.

Michelle: Oh, likewise, always a pleasure to talk to your listeners.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

61. A Motivational Syllabus

Do you wish your students knew what was on the syllabus? In this episode, Dr. Christine Harrington joins us to explore how we can design a syllabus that helps us improve our course design, motivates students, and  provides a cognitive map of the course that students will find useful. Christine is a Professor of History and Social Science at Middlesex College, and is the author of Designing a Motivational Syllabus (and several other books related to teaching, learning, and student success). Christine has been the Executive Director of the Student Success Center at the NJ County of Community Colleges.

Show Notes

  • Harrington, C., & Thomas, M. (2018).  Designing a motivational syllabus:  Creating a learning path for student engagement.  Sterling, VA:  Stylus Publishing.
  • Bain, K. (n.d.). The promising syllabus.  The Center for Teaching Excellence at New York University. Retrieved from: http://www.bestteachersinstitute.org/promisingsyllabus.pdf
  • Listeners to this podcast can purchase Designing a Motivational Syllabus at a 20% discount by visiting the Stylus Publishing order page and using the offer code: DAMS20. This offer applies to the paperback, hardcover, and ebook versions and is valid through 6/30/2019.
  • www.scholarlyteaching.org  – Christine’s website.

Transcript

John: Do you wish your students knew what was on the syllabus? In this episode we’ll explore how we can design a syllabus that helps us improve our course design, motivates students, and provides a cognitive map of the course that students will find useful.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Dr. Christine Harrington, a Professor of History and Social Science at Middlesex County College, and the author of Designing a Motivational Syllabus—and several other books related to teaching, learning, and student success. Christine has been the Executive Director of the Student Success Center at the New Jersey Council of Community Colleges.

John: Welcome, Christine.

Christine: Thank you, it’s a pleasure to be here.

Rebecca: Today our teas are…

Christine: I am not drinking tea; I’m not a tea drinker, I just do water and I will do iced tea once in a while, but not at the moment.

[LAUGHTER]

John: I’m drinking Tea Forte black currant tea.

Rebecca: And I have Prince of Wales today—mixing it up, you know?

John: We invited you here to talk a bit about your book, Designing a Motivational Syllabus, released just this past May, and the syllabus of many faculty tends to read sort of like a legal document and it often tends to be a bit off-putting and some people just provide a list of topics. You have a much different approach and it seems really productive. Could you tell us a little bit about that approach?

Christine: Sure, I’d love to. Thanks so much, John. I really believe that the syllabus is an underutilized resource. As we’re beginning our semester as faculty we always are required to put together a syllabus that explains to students what the expectations of the course are. But as you mentioned, many faculty treat it as a list of do’s and don’ts, making sure that we’re communicating what the classroom rules and expectations are and maybe the course topics, but it often kind of starts and stops there. So, I really think that it’s an opportunity for us to invite students to our course and Ken Bain is actually someone who’s written a lot about this as well, you know, inviting them to the feast. I think that’s what we want to do: we want to give them the excitement and passion that we feel as faculty and get them really excited about the course too. So one of the areas I saw as a gap or missing in the literature was the motivational angle of the syllabus. In addition to providing some really good resources and providing a course map to students, I think we can motivate our students through communicating our passion, telling them a little bit more about what to expect in the course in a more conversational style… by the words that we use… by the images that we include on the syllabus… and then also providing them really helpful information so that they view this as a course that they’re excited about and they will feel supported in.

John: We actually had Ken Bain here about 12 years ago, I believe it was, and he gave an all-day workshop on building a syllabus and I attended that and it was wonderful; much of your book reminds me of that, but you also go quite a bit further and provide a lot more suggestions in detail, so I like your approach. Could you tell us a little bit more about how the syllabus serves as an entry point to course design or redesign?

Christine: Sure. I think that for many of us the idea of redesigning or designing a course for the first time even is quite daunting and overwhelming to really think about how to engage our students and achieve all of our course learning outcomes. So, I’ve used the syllabus as a vehicle for that, as an entry point that I find that faculty find it a little easier if you’re working with a more concrete practical document to help them understand course design. Now, I have been accused by some of my colleagues of doing the old switch and bait, you know, I did a syllabus workshop that really was a course design workshop and they’re like, “Wait a minute, I think that you tricked me here,” and I said, “No, I didn’t, I thought you were talking about the syllabus, and if you’re going to talk about and think about what kinds of assignments and assessments you’re going to use—this is course design.” So you need to have a larger conversation. So, I said it’s an easier way for faculty to begin the dialogue and really take a good deep look at, well, what is it that I’m asking students to do and how does this fit into the overall course as well as the overall program that the course fits within… so, seeing the larger picture in terms of the course and program learning outcomes and revisiting the assignments and assessments and perhaps moving away from some of the “always have done this” kinds of assignments… you know, traditional exa…, paper… presentation… that almost all of us have in our course in one way or another. It doesn’t mean you have to abandon ship, but it is a great opportunity to step back and say, “Wait a minute, are these the assessment tools that are really going to help support student learning and help ensure that they’re going to achieve the learning outcomes that we set forth in the syllabus?” and then the syllabus really becomes the map for students. So it is the document that communicates the design of your course. As you are crafting your syllabus, you’re really thinking about: “What is it that I’m asking students to do? Why am I asking them to do that? and what kinds of supports am I going to put in to place so that they can accomplish those tasks successfully?” So, I really believe that it is a course design tool and that if you do it well, a well-designed syllabus really will show students exactly how they get from point A to B and the kinds of supports that are available to, you know, really enhance their learning journey along the way.

Rebecca: [If] faculty were to use the syllabus as a way to redesign, where in this syllabus should they start?

Christine: Well, that is not an easy question. You know, I think it depends on—I tell faculty they have two choices when they’re thinking about redesigning their syllabus: you can take the big approach, which is the course design approach, which is going to be looking at your course learning outcomes and then looking at what kinds of assessments or assignments are aligned to those learning outcomes… and then what I usually do for this big approach is ask them to think about what are the key summative assessments that you’re looking for and then work backwards from there using the backwards design process and determine what kind of formative assessments they need to use… and then as you start to craft the way your course is going to be developed, I say take your course outline your schedule of what you’re going to do—week 1, week 2, week 3 and start to plug in those summative assessments and then start to plug in the formative assessments that you’ve identified… and then that will help you determine what needs to happen in week 1, 2 or 3 to help them be successful on those tasks. So, it’s really kind of using that backwards design. I like to say it starts with the learning outcomes, it shifts over to the assessments that you’re going to use and then it starts to move into the course outline, you know, or sequence of topics that would be really important. But I said to you there’s kind of two ways that faculty can begin to, you know, redesign their syllabus. This is the big way and the way that I would really love faculty to do it, but if someone’s saying to me “The semester’s starting next week, that’s just too monumental of a task…” You cannot engage in this process in a day or two. It takes a significant chunk of time for you to really re-evaluate what it is that you’re trying to accomplish. So there’s also a lot of takeaways in this book about how you could do things literally in five minutes or less that would enhance the motivation and engage students in learning. For example, I did a workshop on my campus where we had a syllabus redesign summer camp and at the beginning of the semester we had faculty submit their current syllabi and at the end we had them submit their final syllabi… and the transformation just from a visual perspective alone was really incredible. So, if you didn’t even look deeply at the design piece… for instance, having a nice photo to draw students in (that’s related to your course content)… there was a biology faculty member who put this amazing, great engaging skeleton on the front page… and that really was much more effective than having her first syllabi, which had all rules and regulations…you know: “do this, don’t do that…” and a welcome statement and a picture of yourself… really just some of those kinds of elements really can make a difference. There’s some research studies out there that support adding a few additional words like “please come and talk to me” makes it much more likely that a student will come and talk to you, and it communicates that you want them to come and talk to you. There are some very easy fixes; changing it from formal language such as “the professor will” and “the student will” to “I am going to” and “you will do this…” Just using that more personal language can really help. So those fixes are literally… you could do it in five minutes or less if you want to make a couple of minor changes to increase motivation, but the overall course design is obviously a much bigger process, I’m not going to pretend it’s not.

Rebecca: I think it’s always a good reminder that you can always do small things before you can jump into a big thing and that the big thing is, you know, valuable. We were laughing at the beginning of what you were talking about a minute ago because I did the same exact thing here where I did a syllabus workshop that was a complete course redesigned workshop.

John: …and I suggested we rename it in the future as a course redesign or course design or redesign, but maybe leaving it as a syllabus workshop might work.

Christine: Yeah, I think you’ll get more people to participate. It’s less scary. [LAUGHTER]

Rebecca: It’s sneaky.

John: It is. It’s a sneaky way of getting in.

Christine: …and it also really allows faculty to walk away with something very practical… tangible… that they actually have done as evidence of participating in that workshop, and that’s great for administrators to see as well.

Rebecca: That’s a good point. So you talked a little bit about the syllabus as a tool for faculty to help think about organizing their class and redesigning it, making sure that students are going to learn what we’re hoping that they’re gonna learn. Can you talk a little bit about how the syllabus is a tool for students?

Christine: Sure. I think it’s really critical for students to not just take this document and put it aside but to recognize the value that it has… and I will tell you that students who see a more in-depth, comprehensive syllabi have a much more positive perception of the faculty member and also of the student experience of being in that class. From the student perspective, it’s motivational for them to know that they have a faculty member that cares enough to put together this really comprehensive package. Having a long syllabus that does not have any visual tools in it and is overwhelming… whether it’s legalese… that is something that students are not going to use much. But when you create a syllabus that’s motivational and engaging and visually effective, students will use that document and they really will appreciate it. Now, they do need reminders about how to use that. I think that it is a document that all of us quite frankly emphasize in the first day or first week of the semester and then often don’t revisit except for to say “in the syllabus…” which may or may not help a student if they’re having trouble navigating it. So, I’m a very big fan of making sure that we make it a document that is student friendly. If it is a longer document, including a table of contents, so that they know they don’t need to read all of this. Maybe the last half of the syllabus is the rubric section with specifics on the assignment… that they just need to know it’s there when the time comes for them to look at it. So I will often encourage students to view the syllabus very much like their textbook. You don’t need to read the entire textbook during the first week of class, but you certainly need to know what’s in the textbook so you’re not just focused on chapter one. You need to acclimate yourself to all of the information that’s in the text and what kinds of topics and resources are included. Well, the same goes for the syllabus… so really helping them use it as a resource and not feeling like they need to read it and memorize it, but instead use it as a tool to help them be successful.

John: One of the things you suggest is doing a screencast with the syllabus perhaps to make sure that students do look at it and to make it a bit more welcoming. Could you talk to us a little bit about that?

Christine: The screencast, I think, is a very valuable strategy, especially in the online class, but it can also be helpful in an in-person class. We all know that sometimes students are adding and dropping in the beginning of the semester and might miss an important conversation, and this really allows you to communicate about the syllabus to students. In addition, I will tell you that I have had students… I tend to have a fairly lengthy syllabi, as you can imagine, based on my textbook, I like to include a lot of resources… and I have had students say to me, when I got the syllabus via email from you, I really was overwhelmed and I was ready to run away from this class; I thought it was going to be a lot of work because it was a long syllabus, and once you explained it and we started to see the resources in it, I discovered that’s not the case at all. So I think that having that personal touch and the the nonverbals that you can really communicate through a screencast with a web video as well as the audio really does help students understand the value of the syllabus and we have so many great online tools now, like screencast-o-matic, that are free… things like that… that you can really easily do that in a short period of time to do an introduction, and students can refer back to that as they need to throughout the semester.

Rebecca: Can you talk a little bit about the kinds of things you would recommend faculty highlight or introduce about the syllabus on the first day? You mentioned identifying some of the resources and things in it. But, as you know, there’s a lot of faculty that call the first couple class days syllabus days and some of them actually read the syllabus to the students. What would you recommend?

Christine: Well, I certainly would not recommend reading the syllabus to the students. [LAUGHTER] That is not engaging. I think that the part of the syllabus that doesn’t get as much attention as it should is the “Why are we together?” The syllabus communicates: the purpose of the class, the goals of the class, and “What are the takeaways that they’re going to get as a result of being in this class?” Students, what they’re going to immediately want to know is about the grade, right? They go directly to the page that has information on the grade and the assignments that they need to do. But if they go there first, they’re missing the big picture. So, I think that we as faculty have a wonderful opportunity, whether it’s through a screencast or live in a regular classroom setting, to emphasize the learning outcomes of the course in a user-friendly way… not necessarily reading the learning outcomes, but to passionately explain why this course matters so much and the value of the course and the skills, and not just the knowledge that they’re going to get, but really the experience and the confidence that they’re going to get as a result of being in this course. So I really find that to be the most important piece to emphasize, and then helping them see the direct correlation and connection between what it is that they’re going to achieve and those learning experiences. So whether they’re assignments or assessments… the why behind all of those… so they don’t just view it as a big long checklist of “this is what I have to do because it’s a college course,” but they understand that that’s the roadmap that’s going to help them accomplish all those tasks. So, for instance, if I can give you one example, quizzing is something that not every faculty member does, sometimes it’s more of a more high-stakes midterm/final kind of situation… but faculty who really want to provide that opportunity for students to have formative assessments along the way would also include quizzing… and when you do that what happens is is that you’re helping students learn those skills along the way and help them self-regulate whether or not they’re on task to achieve the learning outcomes. But students may view them as busy work or that you don’t believe they’re going to read without being held accountable. By explaining the why in the rationale and bringing some of the research in on the testing effect and explaining to them that the reason for me doing this is because the research shows that if you test yourself you are much more likely to learn that content and it will stick with you throughout much longer periods of time. So providing the why, I think, is probably the most important part that I would bring their attention to and I think that we don’t do that enough as faculty.

John: Just as a plug for a future podcast, Michelle Miller will be a guest in a few weeks where she’ll be talking about the testing effect and retrieval practice.

Christine: Terrific.

John: But that is an issue. Students see testing as something negative; it’s not something they find quite as enjoyable… so providing that rationale is really useful and students don’t always buy it but the more you can convince them and the more evidence you can provide, I think the more likely it is that they’ll see the benefits.

Rebecca: Yeah, John and I have talked about this before that when I started doing that in my design classes, which is a place where testing is not as common, I had students actually asking for more, which I found to be very bizarre initially. You don’t generally have students asking for more tests or quizzes, but when they started realizing how it was helping keep them on track they actually found them really valuable.

John: In helping them assess their learning and to help improve their own metacognition of what they know and what they don’t know, it can be really useful.

Rebecca: One of the things that you have in your syllabus is a teaching statement. Can you talk a little bit about why you include that and why you recommend including that? Because that’s not something you commonly see in a syllabus.

Christine: Absolutely, in fact, there are a couple elements that I think are essential if you want to use the syllabus as a motivational tool, and I see the teaching statement as being one of the key elements; it’s an opportunity for you to start to build a relationship with your students, and it gives you a chance to share some background about who you are and why you’re passionate about the subject and what they can expect to happen during class. As we all know, the professor-student rapport is probably one of the most important predictors of success. Students who have professors who they believe care about them and are interested and engaged are much more likely to be successful than students who have faculty who are much not engaged and maybe not as connected to them. So, I believe that we could use the syllabus to begin developing that relationship, because we often send this out prior to even meeting a student for the very first time, and it also might be something that is shared on some kind of management system within the university or college setting for students to decide which classes to take. So it can invite them to why they should be taking your class –it’s really a wonderful way for you to share a little bit about yourself and your professional background expertise and passion.

John: You also suggest in your book that the syllabus can serve as a communication tool and it also makes it easier to be transparent in terms of how you grade and letting students know this, and that can increase equity, or at least a perception of equity. Could you talk a little bit about that?

Christine: Sure, I think it’s really critical that we are being as explicit and as transparent as we can be. There are going to be some students who can more easily connect those dots than others and when we make the dots connected for them we’re equaling the playing field to ensure that all of our students know what it is that they need to do in order to get to the finish line and how the different tasks relate to one another. So the more you can communicate and ensure that some of the students who may not naturally see those connections can see those connections, I think that really does improve learning and the academic experience for all students.

Rebecca: You mentioned earlier about referencing the syllabus and having students use the syllabus as a tool throughout the semester; you also mentioned early on that faculty have a tendency to say “it’s on the syllabus” without really providing much more guidance than that. Can you talk a little bit about ways that you recommend using the syllabus at farther points in the semester to help support students and continue to motivate students beyond just the beginning of the class?

Christine: Sure. I think that is critical. You know, many of us do activities on the very first day of class. I’m hoping that many of us are not reading the syllabus anymore and we’re starting to get more engaging strategies at the start of the semester. I know folks do a syllabus quiz and things of that nature. I actually think that having a group quiz format,, something that’s more interactive, is great. I do jigsaw classroom exercises at the beginning of the semester on the syllabus. They’re diving into that resource and understanding it and reporting back and teaching their classmates about the different section that they were assigned to. I think setting the stage at the beginning of the semester is really important, but we can’t stop there; we need to then follow through and revisit the syllabus throughout the semester. So what I typically do is I will often ask students to, at the beginning of class, (I always ask them to have their syllabus with them)… and I might give them a few minutes and I do this activity called dusting off the cobwebs… where they have to recall what we talked about last class and maybe from the readings and then we can look forward. So after we clean up our house in terms of where we were then what’s coming next, so what are we talking about today? How does this link up with the concept that we talked about last class, and then what what’s coming up in terms of what’s due? So instead of me putting on the board or on a PowerPoint slide, “Next week, don’t forget you have to submit the first part of your project” or whatever it might be. I’m having students give those daily reminders. So you can literally spend five minutes or less in a class, and maybe once a week; it doesn’t have to be necessarily every class… but maybe Monday’s will be your dusting day and looking forward opportunity. So I think that’s really helpful. The other time where I spend a little bit more time on it is when there is a big project that’s coming up. So at this point of the semester I will often have students work in either a partner group or a small group and in that situation I’m asking them to look at pages 12 to 14 that outline the details related to assignment 1 and the rubric of how you’ll get assessed on assignment 1. I want you to review that. I want you to put that in your own words… tell your classmate about what it is… and then you have an opportunity to ask me any questions about it. So, I’m basically training them to engage in that process. Again, this doesn’t need to eat up a tremendous amount of class time; it can be a few minutes. But by doing that you end up often getting better products to grade which makes your life much happier when it’s time for all the papers to be handed in because even though you put it in the syllabus, it doesn’t mean that they’ve looked in the syllabus… or they knew where to look… or maybe something didn’t make sense to them and they were not comfortable asking without the opportunity given to them in that very explicit way. So, I find that that really as a very helpful process. I also like to do an activity kind of mid-semester looking at the learning outcomes… so, going back to saying “Okay, so here’s what we said we were going to be able to learn and be able to do at the end of the semester. We’re about halfway done. I want you to look at the learning outcomes and do a self assessment. Where are you at on a scale of one to five? What do you need to do in order to get to the level five at the end of the semester? …and some of that’s going to happen obviously in classes or through the assignments. But, what else can you do to ensure that you’ll achieve all of those learning outcomes?” So, I like to use it in a self-regulatory way as well.

John: One of the things related to that is you suggest the use of an assignment grade tracking form. I’ve always kept my gradebook in Blackboard so students can see where they are but students don’t always seem to pay much attention to that. Having them create their own assignment grade tracking might be useful. Could you talk a little bit about what the form is and why you recommend that?

Christine: Sure. I do think that with our current technologies {Blackboard, Canvas, thinks of that nature), the LMS systems really do have a pretty robust gradebook feature where students can easily track their progress. Because in order for them to self-regulate they need to know whether or not they need some external data to see if they’re on track or not. To me, as long as they’re engaged in that checking and self-regulatory behavior, I don’t think it matters whether it’s definitely in the syllabus or in Canvas or Blackboard. But unfortunately, not every faculty member is using the gradebook to its full capacity, so sometimes students are left wondering about their grade and I want them to feel in charge of knowing how to do it. I also think that they have a hard time sometimes seeing the weighting of assignments so that they might view a smaller assignment as being equal to that of a larger one and not recognizing the significance that can have on your grade. In the absence of some of the technology tools… and there are great apps for this too so if your student has a faculty member that is not using an LMS gradebook, they can go ahead and download an app… and I think that’s a great way to track it as well. But just including something like that on the syllabus helps them see the breakdown on the weighting of the different grades so they can see how that final grade is determined.. Because I think you’re right. In the LMS’s I see that students are often looking only at the current calculated grade rather than looking at all of the pieces of how that grade came to be. So anything we can do to help them better understand the grading process and how those elements go into the final grade, I think, is useful.

Rebecca: In your book and also in the example syllabi that you’ve provided (both on your website and also in your book) you talk a little bit about your assignment sheets with rubrics and things completely spelled out… so not something that’s more generalized but something incredibly specific. Can you talk a little bit about the choice to do that and the advantages of doing something like that?

Christine: Sure. So I think that we all provide students with details about our assignments; it’s about where does that happen. For some of us, we think that that should happen outside of the syllabus in the LMS in a different place… under assignments or some other tab rather than being in the syllabus itself. I think it’s really helpful for students to have a complete package in the syllabus. Now, just because I think that doesn’t mean that it’s true, right? Actually I did a really neat study with a colleague of mine Crystal Quillen at Middlesex County College where we examined the student perception of syllabi length and we shared different syllabi. There was a 6-page a 9-page and a 15-page syllabus and they were randomly assigned to different groups. What we discovered was that students who were reviewing a medium or long (which was 9 or 15 page) syllabus actually found that syllabus to be more positive. So they had a more positive perception of the faculty member in terms of being motivated and things of that nature. In addition, we asked the specific question of the students “Would you rather have all of the details about your assignment in one place in the syllabus or is that not what you want? Would you rather just know ‘I have to write a paper’ and then have those details about the paper be provided at the point that you need them and in a different place within your learning management system?” …and 66% of the students said we want it all in one place. I think one of the challenges that our students have is that every faculty member sets up their LMS page a little differently… and I know colleges really work hard to try to have some consistency across the different course shells that exist… but students really do struggle with trying to find that information. If we can guarantee to students that all of the essential information you need about your assignments and your learning path are in the syllabus, then I think that makes a lot of sense. I really think it’s important for faculty and students to understand that it’s almost like an addendum to the syllabus, but it is in that document. So that they don’t need to get overwhelmed by it on day one but they know that it’s a resource… very much, like I had said before, like the textbook is.

Rebecca: I think that’s an interesting point right, a lot of our students are in five classes and if there’s five different ways of doing things and it’s organized five different ways with five different evaluation systems it can get a little overwhelming after a while. It’s a lot to keep track of. We often complain that students don’t keep track of things, but we certainly don’t help it.

John: We’d like to reduce the cognitive load.

Christine: Yeah, that’s for sure. Unfortunately, it’s the case and sometimes we need to get ourselves back in from the student lens to see what does life look like from their perspective. …and if I could just say one other thing about the “It’s in the syllabus” comment… I mean, believe me, I pour my heart and soul into creating my syllabus. My husband often laughs at me because he’s like “Haven’t you taught this course before? You look like you haven’t taught this before…” because I’m spending hours and hours and hours and I just had it last semester but I know it could be so much better. I’m trying to find ways to communicate it better. So, I know that the information is in the syllabus because I put it there. I spent many hours doing that. But if I give if a student is active enough and engaged enough to ask me a question about an assignment or the course and I just say “It’s in the syllabus” my syllabus is a long document. I do need to navigate them to which part of the syllabus it’s in, because my syllabus is probably a little different than other syllabi that they have looked at. So, I feel like it’s so easy and frustrating for us when the students may not have looked carefully before asking us. So that’s a skill we need to help them learn. But maybe they did look and they didn’t find it as quickly as they would like to. Let’s be honest. You and I also are not going to spend an enormous amount of time looking at a website if we can’t find what we’re looking for right away. We’re going to ask someone. So, we want to make it as easy to navigate as possible, and having consistency I think across different courses does help… not that you need to have a rigid standardized syllabus that looks exactly the same in every course. I think you need to have a little bit of room for the flavor and the personality of the course to show.

Rebecca: Those are things that we just forget about. We forget that it can be really overwhelming to look at documents. Yet we all complain about the same thing when someone else makes a document that we have to look at and we can’t find something. So, it’s good to double-check ourselves. So, I appreciate the reminder.

Christine: Absolutely.

Rebecca: One of the things that needs to be in a syllabus to some extent is the policies… there’s college policies that might have a particularly language that you have to include, but then also maybe what some of your own policies are as an instructor. How do you suggest including those in a motivational, inspiring, and supportive way? Because sometimes they don’t feel very inspiring or supportive.

Christine: That’s a great question. In fact, I have noticed that probably one of the biggest demotivators of a syllabus is the policy section… and many times faculty add more and more policies based on negative experiences that they’ve had. So something happens in a classroom setting and they’re like “I need a policy on that, so I’m going to add another policy about that…” and it starts to become this really big long laundry list. And clearly we have to have policies… I’m not saying we shouldn’t but I think the way in which we communicate our policies really do matter. When we have a list of “don’t do this: don’t cheat, don’t plagiarize…” all these kinds of rules and regulations… we’re kind of communicating to students that we think you’re going to do this, so I’m going to set you straight right now… rather than using more positive language. Instead we could communicate the same policy… I like to use the academic integrity one. So instead of saying “don’t plagiarize” instead… if you have a policy about academic integrity and the importance of why that matters so much and how everyone is expected to uphold academic integrity and engage in honest actions… That I think sets a very different beginning to that policy. The other piece is that we sometimes create policies that promote, I think, more achievement gaps… and actually gets back to that question you asked me earlier about equity, because many of our policies do not promote equity. I’ll take a late work policy, for instance… and I recognize the fact that we all need to be timely with our tasks. I mean in the world of work people are going to expect you to complete tasks on time and I recognize that that’s a very important skill. However, I also know that we are all human beings with a life on the side, you know, so that life happens sometimes that may prevent us from being on time with a task. I know I personally have not always been on time. I’m a very timely person, but there have been times when I have missed a deadline and haven’t been exactly where I needed to be at the time I should have been there. It’s not a pattern, but it does happen. So I think we need to have policies that are building in some of the flexibility that communicates to students that we respect them… we recognize that they have a life outside of school or at least outside of our class… sometimes our policies don’t even seem to recognize that they have other classes… like our class is the only one. So, students complain about that quite a bit… thinking that you’re looking at this only from your angle and not recognizing that this is one of many classes that I’m taking. When you think about policies such as that, it’s important to communicate it in a way that isn’t taxing on your time so that you’re taking late work every minute of every day… but is respectful. So, a very simple way to do that is “Here’s the policy. I expect you to be on time with tasks, especially if you’re doing a group task and your classmates are dependent on you.” I tend to be a little bit more rigid with my policies when it’s a group related task versus an individual task. But I also know that life happens and if you are in a situation where you’re not able to meet a deadline, please come and talk to me.” Because, if you put in a policy that says no late work is accepted… everything must be handed in on time. Well, certainly you won’t have to deal with any late work… that helps you with your time management, but it really is inequitable because the student who comes from a culture where it’s fine to challenge authority might come to you and say “My grandmother passed away last week. I have this really horrific thing happened in my life…” and, many of us… I know I myself… I had at some point a no makeup policy. It wasn’t a real policy… if you came to me and it was a good reason then I gave you an extension. But I only did that if you came to me. I did have no makeup policy on the syllabus. So, the problem with that is that there are certain groups of individuals that are not going to challenge authority and take your word at face value. So now you’ve put them at a distinct disadvantage in the class. So I think it’s important for our policy to do a couple of things: one is first of all they should be accurate… so I did not really have a no makeup policy… I had a “makeup if you have a good reason.” So, it wasn’t accurate. So now my policy is much more reflective of my current practices. I expect you to be on time but if something happens, communicate with me and we will see what we can do. It’s not promising them the world but it’s certainly promising them at least the conversation… and second in addition to having it be clearly communicated, it really needs to be equitable so that everyone gets an opportunity and it’s not a case-by-case situation where if they’re more willing to challenge authority they’re going to be more likely to get a positive outcome.

Rebecca: One of the things that you mentioned earlier is shifting the language in the syllabus to something that’s more personal from something that feels more like legalese or something. In those circumstances where an institution might impose a particular policy that’s written in a particular way that doesn’t match the voice of the rest of the document, how do you suggest dealing with that in a syllabus, when it might be required of you?

Christine: I think this is one of the big challenges that faculty face, is when there’s required elements that are not very motivational in nature. So first of all I would say try to start a conversation on your campus about revising that language, not necessarily the policy… that’s what the policy is going to be… but can we introduce it in a different way? So, I would say if you can do that that would be ideal because then that would be benefiting all of the students in all of the classes across your entire campus if they’re required. So I think that’s probably the point of intervention that I would encourage you to take and you could go back and refer to some of the ideas in the text or talking with colleagues about other ways to better word some of that policy language. If you’re not able to switch the language, or you want a quicker fix while that conversation is happening, I think it’s appropriate (and again you need to find out on your campus if it is) to maybe have an introductory statement: “The next section of the syllabus is going to be the institutional policies that every faculty member needs to include.” So, not saying they’re badly worded, but you’re saying that they’re different… like you can definitely see that I need to include these and I certainly wouldn’t have them (unless you’re required to) on the first page or two. Let the more positive motivational pieces be front and center and then have the policies be later on in the syllabus. So almost like you have a cover page or some kind of introduction before getting into the more typical policy language, I guess, if you need to include it I think can be helpful.

Rebecca: I’ve done things where, for example around intellectual integrity, there’s a campus statement (and I label it as such) and then my policy which kind of interprets that and puts it into context and is in my language… so the same idea that you have about introductory or it’s you’re kind of separating the two and making it clear like whose is whose.

Christine: mm-hmm

Rebecca: …and I think that sometimes has helped students too… but I’ve always found that to be jarring.

Christine: Yes. I would agree I think that definitely is, and I like your approach really of summarizing it because sometimes those policies that we get handed down are very lengthy and students probably aren’t going to read them. So, even if you gave the one- or two-sentence summary of what that meant…translation is… you know this is what you need to do… Be honest. you know, engage in honest action. It really matters. We all want a good reputation and we all want to learn. So in order to do that, these are the kinds of strategies that you need to engage in… and going to the integrity topic again, I think so many times students are unintentionally plagiarizing… not always necessarily doing it on purpose. So maybe helping them understand how they can better learn how to cite sources appropriately or how to paraphrase more effectively… So, pointing them to resources that are going to help them with all the tasks.

John: In your book, in addition to providing a lot of great resources about the syllabus and a lot of great recommendations and the evidence behind them you also provide quite a few some very nice sample syllabi at the end of the book and it’s a great resource and your publisher has very gracefully provided us with a discount code to any of our listeners which is DAMS20 and you can do that by going to the Stylus Publishing website. We’ll include a link to that in the show notes.

Christine: John, if you don’t mind I’d like to also share my website which is just www.scholarlyteaching.org. If you go to that website you will see several other teaching and learning resources,including several sample syllabi… and the syllabi that we used in the research study that I mentioned earlier on the length of the syllabus are provided there as well.

Rebecca: There’s also really good videos… a syllabi checklist… There’s some really great resources on the website. So definitely I recommend checking that out…

John: …and also information about your other books.

Christine: Yes, thanks for that John… appreciate it.

John: We always end our podcast with the question what are you doing next?

Christine: Well it’s interesting that you asked that. I am looking at options right now but I am very much interested in staying connected to the teaching and learning space and how we can improve what we do in the classroom. I’m spending some time thinking about moving it up to a higher level and engaging administration in some of the conversations that we’re having about teaching and learning and putting the teaching and learning centers kind of front and center really in conversations about student learning and engagement on campuses. So, for instance I work in the community college system as you know. There’s a national movement called Guided Pathways and this national movement is all about improving the success outcomes of students and it happens to a variety of ways. They talk about making sure our programs are clear, so that they’re defined and students know how to get from point A to point B. They talk about helping students choose a pathway and stay on a pathway, and they also talk about ensuring learning… but, having been a part of this national conversation, the “insuring learning” really is very much an afterthought, I think, unfortunately. I feel like we’re dancing around the classroom. So I’d like to take some of this work that I’ve been doing and work that’s been very directly helpful to the faculty and try to shift it to being helpful to the community college leadership as well as leadership at the four-year universities as well… to emphasize the importance of good and effective teaching practices. So we’ll see where that takes me. I’m not really sure how that’s going to transpire, but I did just present on that topic at the POD conference. I’m putting teaching and learning centers right in front and center in the Guided Pathways movement, getting them at the table of these conversations. So, I’m very interested in further pursuing that at this point.

Rebecca: That sounds like that could be really valuable to a lot of the faculty because translating that information to the administration is always really useful in finding support and all working together to have these really stronger outcomes for students.

Christine: Absolutely.

John: Well, thank you. This has been a fascinating conversation and this is a book I’m going to recommend to all of our faculty.

Rebecca: Yeah, so glad you were able to join us.

Christine: Well, thank you. I really appreciate the invitation and I hope that everyone listening is able to design motivational syllabi and if that happens our students are the ones who will benefit at the end of the day. So thank all of you for listening and for supporting students in their learning journey.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

57. Scalar

Imagine an online environment that makes the thought processes of a writer visible, including the loops they get stuck in, the relevant tangents they pursue, and the non-linear way in which their ideas evolve.  Now imagine that all of these features are easy to use and implement in the classroom. In this episode, Fiona Coll, an Assistant Professor of Technology and Literature at SUNY-Oswego, joins us to discuss how Scalar, a free open-source publishing platform, can help achieve these goals.

Show Notes

Transcript

John: Imagine an online environment that makes the thought processes of a writer visible, including the loops they get stuck in, the relevant tangents they pursue, and the non-linear way in which their ideas evolve. Now imagine that all of these features are easy to use and implement in the classroom. In this episode, we examine how Scalar, a free open-source publishing platform, can help achieve these goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Today our guest is Fiona Coll, an assistant professor of Literature and Technology at SUNY Oswego. Welcome, Fiona.

Fiona: I am very happy to be here, Rebecca.

John: We’re very happy to have you here. Our teas today are…

Fiona: Today I am drinking Cranberry Blood Orange Endless Sunshine tea, which is a very, very ambitious kind of tea by the Republic of Tea, and I just have to note that on the side, it proclaims that drinking this tea will “create social balance one sip at a time.”

Rebecca: So maybe that’s what we should all be drinking right now. [LAUGHTER] I’m drinking English Breakfast.

John: I’m drinking Bing Cherry Green tea.
We invited you here to talk about your work with Scalar. What is Scalar?

Fiona: Scalar is an online publishing platform designed for long-form, media-rich writing. In the words of Scalar’s creators, this means media-rich digital scholarship. It’s an open source platform created by a group called the Alliance for Networking Visual Culture, and the whole idea behind this platform is that it was built to serve scholars who were working on non-traditional, long-form academic writing, specifically projects that might involve visual culture or media culture. There are particular features of Scalar that have been geared towards this use case, but I would like to argue that Scalar is actually a fantastic tool for teaching because of some of its unusual features. Can I tell you about them?

John: Sure.

Rebecca: Yeah, please. [LAUGHTER]

John: We’d be asking anyway. [LAUGHTER]

Fiona: The best way to approach these unusual features is, I think, to describe how you use Scalar, and so I will. The basic unit of content in Scalar is called a page, and it seems fairly unremarkable when I begin talking about it in this way. When you’re creating a Scalar page there’s a text box where you enter a title, there’s a text box where you enter a description, and then a large text entry field where you can put in text and format it. You can choose from a few layout options. You can integrate media into that page. You can enter metadata. You can annotate. You can add comments. So far, so WordPress… fairly straightforward. However, things now get interesting once you create a Scalar page. Once you’ve created a series of Scalar pages, you can start building routes through that content. There are two ways to organize the pages that you create in Scalar: the first is to use tags, which create nonlinear clusters of organized content, or you can use the path feature in Scalar, which is, as it sounds like, a path—a linear, step-by-step progression through a sequence of Scalar pages that you determine. You can get very creative with this path structure; you can create branching paths or very complex forking paths; you can create recursive or looping paths that come back to steps you’ve already been through; you can create rabbit-hole paths that lead people away from the main branch of your content into an unretrievable nether place, but the point is Scalar does not impose any sort of order on the content that you created and indeed that’s why the platform is called Scalar—it comes from this reference to two ways we think of quantifying movement in the world, I suppose: Scalar versus vectors. Vectors are quantities that have both magnitude and direction to them and a scalar quantity is one that has only magnitude and so Scalar, the publishing platform Scalar, does not force you to do any sort of particular relationship between the things that you create. Again… doesn’t sound especially revolutionary, but remember how I mentioned you could add tags and comments and annotations to a piece of Scalar content? When you do that, when you create things like tags and annotations and comments, those all become Scalar pages themselves, and they can participate in this larger set of relationships. So a tag, which is also a page, can be tagged with something else, it can be a path of its own, a comment can be a tag, an annotation can also be a comment, can also be a tag, a path can be a tag on something. So, any piece of content in Scalar can be given any sort of relationship to any other piece of content, and what this means is that there’s a sort of radical, non-hierarchical organization to the way Scalar allows you to approach the products of your own creative work. So this becomes really, really interesting if you imagine what this means for creating something like an essay. We have a long tradition of thinking of an essay as an extraordinarily linear thing that begins at the beginning, that moves through a sequence, and that ends. But Scalar allows us to reimagine what an essay might be, not just what it might contain, so not just moving beyond text but moving beyond that linear structure, and when I first understood just how radically Scalar allowed the breaking down of this old-school essay model, I became very excited to imagine its possibilities in the classroom. So, I learned about Scalar and immediately thought that this would be a fantastic way to defamiliarize the writing process for students, and by “defamiliarize the writing process for students,” what I mean is I thought that this would be a fantastic tool to get students to reimagine the way that their thoughts unfold in writing. I wanted them to reimagine writing as actual making, as actual construction, and not just as a sort of tragic endpoint for a thinking process.

Rebecca: It’s active, basically.

Fiona: It’s active. It is a process; writing is a process and I always say writing is thinking, and students I don’t think quite understand what I mean by that, but what Scalar might allow me to do, I imagined upon first encountering the platform, would be to get students to think about how sections of their thoughts work; how ideas might connect to other ideas… not in linear ways… but in roundabout ways that might meander through other references or images or clips they came across on the internet or things from other classes… that thought is not linear no matter how much we try to get them to package it into straightforward, well-behaved writing.

Rebecca: So this is really exciting, I can imagine writing something like “I have this thought and now I’m in a loop and I can’t get out; I’m cycling through ideas and trying to get myself out and I just can’t, but sometimes that happens when you’re writing and it’s like, oh, this isn’t gonna work; I don’t have a conclusion.

Fiona: This is how the process of writing works; you do get in loops. It is a reiterative experience where you try something out and you might end up back where you started; you try it again, you come back where you started, but perhaps the loop needs to be there for a particular reason but there’s a little exit ramp you might find to some other form of thought and Scalar doesn’t force you to try and pretend that that is not happening, that that complexity is not happening. It allows you to in fact mark the way that your thought is moving and branching in non-linear ways and allows you to capitalize on those threads and those directions. One of the things that Scalar does that very useful in getting students to think about their writing process is that it timestamps every iteration of a particular page and it saves every iteration of a page. So there’s a sense in which students are free to revise or rethink. There’s a sense in which Scalar holds safe and secure all of the versions of their thought, so it works well in terms of allowing them the space to experiment and the space to make mistakes while also giving them a time-stamped chronology of the work that they’ve done. So, there are multiple ways in which Scalar allows for the thinking process to be represented.

Rebecca: It also seems like it’s a good model for students to know how long they’ve spent writing because their idea or conception of how much time they may have spent doing something might be really inaccurate.

Fiona: That’s a fantastic point because I think students do have a strange dislocation from the actual effort it takes, the actual labor that goes into producing something like a polished text. So, on the one hand there’s just an awareness of the sheer time that goes into that, but there’s also a sense in which Scalar allows students to really, really dig into the revision and the editing process, which often is hard. So, students sort of do the standard essay writing and I often find it difficult to convince them to let go of certain aspects of what they’ve written or to radically or drastically revise…

Rebecca: But it’s still there.

Fiona: But it’s still there; they don’t have to worry about losing it—they can try something completely different and perhaps see what happens when they release their hold on that idea that writing is just something you open up a word processor and do… start at the beginning and go until you’ve hit the word limit.

Rebecca: You’ve got pathway one like normal way, then it’s like here’s my cycle weird way, here’s my figure eight way.

Fiona: Yes!

John: When you have students work with this are they working individually or in groups?

Fiona: This is the next thing I wanted to mention, which is that Scalar allows for both. It is extraordinarily flexible in terms of this exact question. I usually begin by having students create content, create Scalar pages on an individual basis, but all of the students are creating within what I call one great big bag of Scalar content. Scalar uses the term “book” to describe one project in this way. So, the students can create their own content, tag it as their own content, organize it according to their own methods, but then I get students to interact with each other’s content, so they read each other’s content, they start to make tendrils of connections between their content and other students’ content, and then eventually I build up to students generating content collaboratively. So it works really well in allowing a wide range of writing collaboration, and the point I make as these networks of connection get more and more elaborate is that this is how knowledge works, this is how knowledge is created; it’s a collective, collaborative enterprise and nobody does best working in isolation.

John: When they do this is it something that’s shared just within a class or is it shared publicly?

Fiona: Again, Scalar has both options available. I discuss this issue of public versus private writing with the students and we usually make a decision together as to whether or not the students want to make their material, their writing, public or private. I’ve also had a class in which one student was very happy to make her Scalar project public, but all the rest of the students wanted to keep theirs private, so she was able to easily take her content, make a whole new Scalar book and proudly display it for everyone to see, so it is remarkably flexible in terms of what it allows you to do with what you create.

Rebecca: What about the converse, though, when it’s a one or two students that have a reason that it needs to be private?

Fiona: And there’s absolutely no problem in keeping a Scalar book entirely private. I also give students the opportunity to erase what they’ve done… so to remove it entirely. We do talk a lot about privacy, public writing, and issues of copyright is another angle that seems important to talk about in terms of the Scalar ecosystem. The group who has built Scalar is deeply invested in promoting open access and fair use of cultural resources as part of their commitment to generating very dynamic and free intellectual exchange, so they created something called the Critical Commons—it has a relation conceptually or figuratively to Creative Commons—but their Critical Commons is a place where copyrighted media is taken and transformed critically and then posted for fair use purposes. So you’ll see people who have taken clips of movies, for example, or a television shows but transform those clips through a critical apparatus that Critical Commons enables, and this allows students to really think about what they’re doing when they reference a piece of culture, whether that’s a photograph or a song or a video, that by adding their critical commentary to it, they are transforming it, they are generating ideas that are making that piece of content new, and Scalar’s link to Critical Commons allows them to really think about issues of copyright, issues of intellectual openness, what happens when something is locked down and is unavailable for access to them to write about, so it becomes a much, much broader discussion about the nature of knowledge, the nature of information in our 21st century.

Rebecca: It sounds like the emphasis then with this Critical Commons is the idea of fair use and understanding fair use and describing fair use and putting in a structure in place that embodies and enforces fair use.

Fiona: And that embodying and enforcing of fair use that you describe then becomes part of how the students think of themselves as creators, so what does it mean to take something that another student has written and to use it in some way in your own thinking? Where do the bounds of fair use lie? It’s often something students haven’t thought about and this actually relates to the labor-related facet of Scalar that I find really useful in terms of student learning. I often feel that students see the Internet as this place where disembodied text has just appeared and exists, but by generating it themselves they have to confront the fact that a lot of work or a lot of effort went into generating the things that they don’t think very much about, and so Scalar allows students to think about the writing process in new and interesting and productive ways, but it also allows students to think about the nature of information that they engage with on a daily basis.

Rebecca: It’s really funny that we’re talking about fair use today because I was talking to my students about fair use this morning. We had a visiting artist who uses fair use in her work and then there was like a thousand questions when she was here. I said, you know, “We’ll talk about fair use, I promise, on Monday, when we all get back and she’s not here and we’re not taking up her time to dig into it.” But, it’s funny because they have this commercial point of view and then also the cultural maker point of view and they conflate it as if it’s all the same and that is really different. Context matters… and that you need to be thinking about these things, so we tried to untangle that today, but you’re right, students don’t think about that at all; in fact, scholars don’t think about it very often either.

Fiona: It’s true, and I first used Scalar in a class that was comparing and contrasting 19th-century book technologies with 21st-century digital writing and publishing technologies and part of the reason that worked the way it did is that 19th century literature is, of course, out of copyright—it’s public domain—and so we were able to play very freely with the literature from that period, and then students had to stop and think and realize that the 21st century, again, literature in various interesting forms, was different, was fundamentally different because of this legal category that we use to distinguish between what is public domain and what isn’t, and students are fascinated by it, while also not understanding it or understanding its logic, necessarily. So, Scalar’s making visible of something that students just hadn’t thought about before is one of its many, many strengths or one of the many valuable ways in which it operates in a classroom.

Rebecca: Can you take us on an adventure through one of your classes to get us a better sense of how you’re actually putting it into play in a specific class with a specific group of students?

John: In terms of maybe the type of assignment that they might be working on?

Fiona: For sure. I first used Scalar in a class that contrasted 19th-century material book production with 21st-century digital publication technologies and I asked the class to really consider the ways in which genre, in particular, is affected by the shape of publishing possibilities. So students are used to thinking about genre as something that is an intellectual idea or an abstract idea informed by author influence or cultural anxieties, but they rarely think about genre as something that is shaped by the actual material affordances of publication, so we read 19th-century texts, we read 21st-century texts and then I asked the students to produce their own creative or critical response to the material in our classes, and what that meant was that some students wrote relatively traditional research essays that incorporated media, sound, video. It meant that some students created choose-your-own-adventure type creative stories that played with the notion of genre as Scalar allowed them to unpack conventions that were and were not possible in that electronic form. Students also used other sorts of technologies to play with the way that technology shaped the kinds of stories they could generate, but that’s a broad overview of how Scalar worked in one particular class. I am using Scalar currently in a class about digital literary studies and the students are making digital editions of 19th-century texts. So, students are in groups, they’ve each been assigned a story by an obscure local Oswego author and they are in groups deciding how they want to present these stories to the world… new and refreshed by their 21st century perspectives on the stories, so some of them are emphasizing maps and timelines. Some of them are emphasizing illustrating the stories. Some of them want to actually remix some of the stories and generate alternate routes through the stories. So, they’re able through Scalar to invent and create these approaches to literary interpretation—they’re making arguments about the text through their use of Scalar, and I should mention that one of Scalar’s appeals is that it’s possible to do a lot with minimal technical knowledge. It’s also possible to do a lot if you have maximal technical knowledge. There’s a lot of room for customization if you are fluent with CSS and that sort of business. It accommodates a really wide range of technical skill.

John: Could you tell us a little bit more about how the choose-your-own-adventure type things work? Is that difficult for the students to program the branches?

Fiona: The great news is the students don’t really have to program the branches. The student I’m thinking of in particular wanted to write a choose your own adventure story that turned into a different genre of story depending on which path you took through her story…

Rebecca: I love that idea.

Fiona: It was a fantastic idea and it really showed just how well she grasped the possibilities that Scalar offered. So she began—there was an introductory page that set up a scenario—it was a mystery, perhaps a murder mystery story at the beginning, and then she had a couple of options: you could choose to follow one character or one event; and as each choice branched a little bit further and a little bit further, so there were many, many iterations of the story, and again each arm of the story took on a slightly different generic set of conventions. It was relatively straightforward; literally in Scalar you simply mark, using a little sort of dialog box that you check or uncheck. you mark what pieces of content you want to attach to a page. So, there’s no encoding, there’s no high-level function that students need to worry about; they can simply imagine what they want to connect and they can make those connections relatively easily. I will say that one of the other things I love about Scalar is that it generates productive difficulty for the students, it generates a lot of intellectual uncertainty which is something that I find… [LAUGHTER] I enjoy producing in students in a constructive way, obviously. Because Scalar is this enormous bag into which students just throw pieces of content, it can get overwhelming really quickly—there can just be this amorphous, chaotic mass that they struggle to make sense of—but that’s part of the advantage, I would argue: it really, really makes them think about high order levels of structure and organization. So even though they can do multiple kinds of organization… even though they can be very creative about how they organize, they do have to really think about how they want their content to relate to one another. So Scalar has this ability to get students thinking at that high level of structure while also allowing students to pay very, very close attention on the level of annotation and close reading—it combines those two levels and sort of everything in between in a way that I find very, very useful for students to be doing. I haven’t even talked about the kind of media annotation that’s possible. But, you can annotate, on a pixel level, images. You can annotate in various time stamps on a video or a piece of audio. There’s an extraordinary level of very, very specific detail that you can attend to as well as dealing with these large high order or large-structure levels of organization.

Rebecca: How did you learn how to use Scalar and then also how do you help students learn how to use the platform?

Fiona: This is a fantastic question. I learned to use Scalar in a very short, informal lunchtime demonstration given by Cathy Kroll—who I believe is at Sonoma State University—at the 2015 Digital Humanities Summer Institute in beautiful Victoria, B.C. and Cathy Kroll simply went through the process of making a Scalar page and she simply explained—and there are all sorts of interesting, cool things you can do with this organizational system—and that was enough; that was enough to allow me to at least discover its possibilities. So, the barrier to entry is low, but then you can ramp up things an awful lot, and I do find that I’m learning more and more as I go. I first imagined using Scalar in my own scholarly work—I am working on this, again, obscure local Oswego author—and I was trying to imagine ways to experiment with bringing these stories back to digital life, but I found that I was almost more excited by the possibilities I was seeing in students and so I thought I would take the exact same approach. I tend to give students a very, very basic introduction to what Scalar can do and then just let them loose, so allow them [LAUGHTER]—again, productive frustration—they make mistakes, they lose pages, they can’t figure out if they’re tagging a page or if they’re making a page a tag. I allow this brief sort of beginning phase of crazy-making exploration and then I ramp up the features, so I introduce more and more features. I begin by, I suppose it’s the carrot versus the stick analogy, so I begin by showing some of the very cool things Scalar can do, so with a basic knowledge of how metadata works students can produce these very gorgeous timelines or maps; I show them how they can use iframes to pull in content from various places on the web and enliven their writing. But, I also then ask them to think very hard about how they’re engaging with other students’ work, and so it feels as though I start with one page and then just allow them to explore on their own while giving them pushes in certain directions to make sure that they are exploring as fully as possible.

John: Maintaining those desirable difficulties as they develop more skills.

Fiona: Maintaining the desirable difficulties, exactly. I’m still trying to figure out how much I should push them, so how much I should demand of the students. I know that other people have used Scalar simply as a writing tool, so just dealing with text and organization. I know that others have encouraged students to make use of the multimedia affordances of Scalar and I’m still figuring out what the balance is for my students who are mostly students of literature.

Rebecca: The first thing that comes to mind to me is how the heck do you grade that? [LAUGHTER] There’s a lot to keep track of and map and pay attention to, so how are you evaluating students in like what criteria and and how do you actually just sort through all of that content?

Fiona: This is a fantastic question and one I am still figuring out… [LAUGHTER] the answer to. As I’m introducing students to Scalar and as I’m letting them make a mess and generate multiple versions of a single page and get confused themselves, I do encourage them to keep in mind that ultimately they want to be imagining not just their own thought process and writing process but what it might feel like for a reader to come across their material, specifically a reader that is me; [LAUGHTER] specifically a reader that will be assigning them a grade. [LAUGHTER] At the very same time, I do try and emphasize process over product, and because students come with such a range of technical capabilities, I build into my rubrics how hard a student has worked to correct a deficiency or to overcome a limitation in their ability to understand Scalar. Ultimately I am interested in the argument that they’re making, but I do reward and encourage what I call bravery—willingness to try new things; willingness to fail; willingness to get things wrong but then to turn that failure into something useful or to meditate or reflect on it in a conscious way. So, there’s a metacognitive aspect to all of this, and essentially in every assignment I’m still trying to figure out what the balance between rigorous analysis and explorative risk-taking might be. I tend to err on the side of appreciating the risk-taking, I will say that.

Rebecca: So do they submit like a URL to you?

Fiona: No, what happens is they tell me where they want me to enter their work. I usually create an index page and I ask them to put their starting point on that index page, so they’re all contributing to one page that serves as my starting point and that’s the easiest way to wander through things. I can go hunting if I need to. I encourage the students to tag what they’re doing with their own names. If there’s a good search function, for example, if I’m looking for something that’s been lost. It definitely feels like hunting in a barn full of hay sometimes. That’s not quite the same as hunting in a haystack but it’s not quite not the same either. [LAUGHTER]

John: Have any other faculty in your department or on campus adopted Scalar yet?

Fiona: I don’t think anyone else in my department has adopted Scalar. I do think as my classes perhaps turn more towards public facing projects that might change, because I do think there are a number of faculty and approaches that could do very cool things with Scalar, but so far I have had to pull my examples from elsewhere… from other campuses. But hopefully soon there will be some robust Oswego examples.

John: Have you ever had students build upon the work of earlier classes?

Fiona: I have not, and that is something that I’m trying to figure out how to do successfully, for a couple of reasons. So it would be easy to do if I just kept one giant Scalar project and had students continually reiterate upon the work that had come before; I haven’t actually repeated a course yet that I’ve used Scalar in, so that in fact might be a next step for my work with Scalar—it would involve, of course, getting permission from the students to do this or to allow them to anonymize their work, but those are things we could work out—but I have not yet done so. I could imagine the Digital Archiving Project as being one that would lend itself towards that sort of semester after semester continuation.

John: How have students responded to this compared to more traditional writing classes?

Fiona: The great news is that students seem very, very excited by what feels like them to be freedom. They respond really well to the autonomy that Scalar offers them. They tend to respond in a slightly opposite direction when they realize that freedom comes at a price and that price is an awful lot of work and figuring out technical details, and some students truly do flounder—some students just find it absolutely maddening to try and understand what’s happening. But some students absolutely thrive and really run with the creative remixing possibilities and really embrace the radically democratic approach that Scalar allows them to take to their own writing and writing in groups. So, I would say that there’s a now predictable sort of curve: initial excitement as students think about the possibilities, then there’s an inevitable drop in enthusiasm as the students realize just how much work this involves and how much new thinking they have to do to wrap their minds around the defamiliarization that Scalar offers, and then perhaps two tails after that: one very enthusiastic skyrocketing of competence and then one more medium flavored… just sort of making peace with what I’ve asked them to do, and I do always offer students options, and if someone just feels absolutely unable to grapple with Scalar there’s always the possibility of doing a different sort of project, but I haven’t yet had a student who has completely resisted.

John: This is a nice follow-up to our earlier podcast with Robin DeRosa where she talked about open pedagogy and it seems like this would be a nice tool for students to create materials that can be widely shared, if they choose to.

Fiona: If they choose to, and I do think I’m gonna bring the concept of open pedagogy or open ed more and more explicitly into classes in which I use Scalar to make that a part of my justification, or something else to get students thinking about. It’s a growing and very exciting movement—the open pedagogy and open education movement—and I’m excited to see how Scalar can continue to be a part of that.

Rebecca: Does Scalar offer, by default, a way to license individual pieces of content using Creative Commons or is it more how you would traditionally license a website by copying and pasting the code from Creative Commons, for example, on an individual page?

John: Or is it just a Critical Commons option?

Fiona: That is a fantastic question. I think you would need to attach your own Creative Commons licensing; I don’t know that there’s a built-in feature. However, I should say that that doesn’t mean it doesn’t exist; it just means I don’t know about it at the moment, but I do think you, again, get to choose your own approach to that very issue. but I’m gonna look into it and see if I can figure out if there’s a built-in tool or aspect of Scalar.

Rebecca: We can make a note of that in our show notes, too, afterwards.

Fiona: I will follow up. [LAUGHTER]

John: And if we find any links we’ll include them.

Rebecca: ..[If] people wanted to get started, do you have a couple of examples that you might recommend for people to look at?

Fiona: I definitely have a few examples that I can recommend. I can add those to the show links, perhaps, and there are examples that range from student projects through elaborate library-based projects to very beautiful, customized versions of Scalar projects. I’d be very, very happy to share them and encourage people to try out the platform.

John: We always end our podcast with a question, what are you going to do next?

Fiona: To this point I have used Scalar in upper division literature courses where students come to the course already equipped with a certain set of writing and thinking skills that I can leverage and encouraging the curiosity and bravery I mentioned. So, next semester I’m gonna try using Scalar with a first-year composition course, and so I’m in the planning stages right now to see how that particular experiment might unfold.

Rebecca: That sounds really exciting.

Fiona: I’m super excited about it. As you might be able to tell, I really, really, am fascinated by the ways in which Scalar seems to activate student curiosity and student agency in their own intellectual work.

John: And if you reach freshmen with this they might perhaps suggest it to some other faculty as something they may wish to try.

Fiona: I like it, I like it as a plan.

Rebecca: Sounds like we’ll have to do a follow-up.

Fiona: I am here for it. [LAUGHTER]

John: Well thank you. This has been fascinating.

Rebecca: You’ve piqued my curiosity; I’m gonna go explore, so I can’t wait for those extra links so I can find a way in.

Fiona: If I’ve piqued your curiosity, I believe I have done my job.

John: And I did create an account a couple years ago when you gave a workshop and I kept meaning to go back, but now I’m more likely to. [LAUGHTER]

Fiona: Well let me know how you find it; let me know what you discover.

John: Thank you.

Rebecca: Yeah, thank you very much.

Fiona: Thank you so much; it was wonderful to talk with you.
[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen and Dante Perez.

52. Metaliteracy

Do your students create digital media in your courses or just consume it? Does the concept of information literacy seem too limited in this context? In this episode, Tom Mackey (Professor in the Department of Arts and Media at Empire State College) and Trudi Jacobson (Head of the Information Literacy Department and Distinguished Librarian at the State University of New York at Albany) join us to discuss metaliteracy as a framework for improving critical thinking and metacognition while students become active participants in the construction of knowledge in online communities.

Show Notes

Transcript

Rebecca: Do your students create digital media in your courses or just consume it? Does the concept of information literacy seem too limited in this context? In this episode we discuss metaliteracy as a framework for improving critical thinking and metacognition while students become active participants in the construction of knowledge in online communities.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guests today are Tom Mackey, Professor in the Department of Arts and Media at Empire State College and Trudi Jacobson, the Head of the Information Literacy Department and Distinguished Librarian at the State University of New York at Albany. In fact, she is currently the only Distinguished Librarian in the SUNY system. Welcome, Tom and Trudi.

Trudi: Thank you.

Tom: Thank you. Very happy to be here. Thanks for the invitation.

Rebecca: Today our teas are…

Trudi: I am drinking highland blend.

Tom: I’m drinking sweet cinnamon spice.

John: And I am drinking chocolate mint tea, a Harney and Sons blend.

Rebecca: I’m back to my good ole English afternoon.

John: Such a surprise.

Rebecca: Sometimes you just need to have the dependable option.

John: Both of you have written very extensively and done a lot of research and workshops on metaliteracy, with three books, three MOOCs (with a fourth one under development), several articles, a badging system and the metaliteracy.org blog. Could you tell our listeners a bit about what metaliteracy is?

Tom: Sure. Thanks, John, I’ll start. Metaliteracy is a pedagogical framework that empowers learners to be active producers of information in collaborative environments. So that’s our elevator speech right there in terms of what it is. Basically, it is an approach to teaching and learning that prepares individuals to be reflective learners in addition to being critical thinkers, and we’ll talk a bit about how that reflection piece is especially critical for a metaliteracy, which, of course, applies metacognition. By doing so, learners are informed consumers of information, which means they ask good questions about the information they encounter in a variety of environments, and as you know, that’s important today with all the different environments and social media environments and access to different kinds of new sources that we have; it includes those especially mediated by technology. And we’ll talk as the idea was first introduced and developed why that was so important to the concept. When we first introduced it we really argued that because of the emergence of social media, online communities especially, think about web 2.0 and the change from the original web—what a critical moment that was—that what we really needed was a metaliteracy that promoted effective participation in these environments. As we know, these collaborative social environments have an engagement piece that is important; we build that into this metaliteracy framework; we thought there was a real need for that—how we were developing it. We also acknowledge that in addition to acquiring information and looking critically at information that individuals really needed to creatively create and share information in this connected network world. This idea of consuming information versus producing information, it’s an idea that’s been around for some time, but we really thought it was critical to develop it into a metaliteracy that also focused on reflection as a core concept. The idea of a metaliteracy is that we look at some of the common characteristics that unite different forms of literacy—that was the other piece of this. We introduced it as this comprehensive, unifying framework. The idea for that was that in this social media environment what we really needed was to try to better understand different competencies, different characteristics of literacies instead of just coming up with a new literacy every time there’s a new technology. We were trying to look at things in a more comprehensive way. As the idea developed in the first book, especially the meta in metaliteracy, intentionally invokes this idea of metacognition. Or thinking about your own thinking: this is really key to metaliteracy because metaliterate learners are reflective about their own learning experiences and they really take charge of their literacy and learning which is really where the empowerment piece comes in.

Trudi: Meta derived from the Greek… also means “after.” Metaliteracy is what happens after literacy. Basic reading and writing, what comes after that. Also what comes after information literacy, which is g enerally thought of as finding and locating information. The definition of information literacy has expanded since we started work on metaliteracy. In addition to reflecting on their own thinking, the metacognitive aspect of metaliteracy also means that individuals have the capacity to self-regulate their own learning, which means that they identify their own strengths and weaknesses and play a role in preparing themselves to adapt to new learning situations. Metaliteracy prepares learners to adapt new technology and to do so in a critical way, that is asking questions about how technologies are designed and the ways that technologies or platforms may impact how we access and create information as well as how we communicate with information. Originally we developed metaliteracy to emphasize how individuals participate in social media environments. And Tom, would you like to talk a little bit about that?

Tom: That piece is really essential to what we’re doing. We see this framework is relevant to a range of collaborative teaching and learning situations, but it is interesting that we saw a real need for emphasizing the social media aspect, online communities, this post web 2.0 environment that we are in, but we also don’t want it to be limited to that. We really see metaliteracy in all environments, all collaborative environments… communities of practice. This is something we should be thinking about beyond just the technology, but really how we engage with each other, how we participate and perhaps also how we blend the technology, how we mediate technology with those spaces as well.

Trudi: One of my favorite parts of metaliteracy is that it advances the idea that learners are teachers. We see this in collaborative environments where learners support and teach other learners, but what’s really important is that often students, for example, don’t think they have any particular expertise in something, and encouraging them to empower to teach others often leads to really interesting situations.

 Tom: That part is so key and that’s something that we saw in our own teaching experiences that when we had students in collaborative situations… group work… building technology tools together… building collaborative websites, for example, that the students themselves were as much a teacher as I was, and trying to foreground that so that they can see it, is critically important.

Rebecca: This is a really interesting framework and you’ve given us a lot to think about. Can you help us make it a little more concrete by providing an example of how you might emphasize metaliteracy in a class or what you mean by a student who might be metaliterate?

Trudi: One of the things that I would do in my classes is encourage students to be information creators and to explore the technology in doing so. So they don’t have a final paper that they have to write, but they may need to create a video or a tutorial or we’ll be talking about our badging system later, maybe creating content for that and doing it in small groups. If they’re doing something where they have to use a technology; I don’t teach them that technology; they sort of learn together and that “learner as teacher” often comes out in those situations because often there’ll be a student in a group who will have more expertise in that area or be more willing to just jump in and see what happens, and then the rest of the group will learn from that. One of the more interesting teams that I had when I’ve taught is one where none of the students felt they could do anything, but they actually accomplished it and their sense of pride and empowerment in doing that was wonderful.

Tom: I have an example: I’m currently teaching a course at Empire State College called “Digital Storytelling,” and the whole point of the course is that students learn about these resources, they locate them (with some prompts from me in the course), but it’s a fully online course and in many ways they have to figure this out on their own, they have to adapt to these new technologies, and I think that they’re looking at their own use of technology in a different way. So, for example, the very first assignment they have to create a selfie video with their cell phone. So they all have cell phones, they probably all done videos before, they probably all done selfies before, but this assignment is really designed for them to introduce themselves to everyone else in the class in a fully online course. From the very beginning they have to challenge themselves to present themselves a certain way to the class… to be themselves but to also think through that presentation, to really be the active producer of information in a collaborative setting where they’re doing something on their own but they’re sharing something about themselves to the other class. In an online course it allows us to get beyond just the text-based introduction and online discussion and to really seeing the students, to hearing from them. I posted a video of myself and it was great to see their response, so it was very much like a classroom situation but it happened asynchronously and online and it was a great way to get the class started, so from the very beginning they saw themselves as digital storytellers and they know that they now are starting their story and that we’re all going to participate and learn from them.

John: So it’s encouraging students not just to critically analyze information as consumers but to be active participants in social dialogues as producers as well. Is that a reasonable short summary?

Trudi: Yeah.

Tom: Absolutely. And what does that mean? …especially in today’s environment, which is very participatory but were divided and partisan in so many different ways. How do we get across those divides? What does it mean to be a responsible participant of information now? What does it mean to be an ethical contributor to these spaces? The whole idea is to really to get them to reflect on this, and not just to produce and share something, but now especially to think about the implications of that so that the informed consumer part is still important so that they’re thinking about these different sources that they’re encountering but also thinking about what they’re creating themselves and sharing.

Trudi: I think when they’re asked to be information producers in this way they think about themselves differently. They create information and share it on social media, but they don’t really think of themselves as information producers, and so I think it expands their horizon.

Tom: They may not have necessarily been asked to do so in an academic environment. This blurring of boundaries between informal learning and formal learning, I think it helps to push that a little bit. Not to say that they’re not beyond our classes, because they might be, but clearly they’re doing it in their everyday practice with their cell phones and the way they consume information now, but this really foregrounds, I think, in some of what the responsibilities are and what the empowerment of that is as well when they’re asked to construct something, so instead of a research paper maybe that is a collaborative media project with their peers—what kind of learning do you gain from that experience?

Trudi: Just one other point. The projects that I was talking about, they need to create them for public consumption. It’s not something that’s just directed at me as the professor of the course. They have to think about it a bit differently.

Tom: That’s a great point, because in the digital storytelling class they’re not just creating it even for the Moodle environment that we’re in; they have to actually upload their selfie videos to YouTube so that they’re thinking a bit about that public consumption piece even beyond the instructor and even beyond the class itself because now it’s up on YouTube and hopefully that’s having an impact on what they’re thinking about in terms of how they present themselves in the information that they’re producing.

Rebecca: I’m hearing two key things bubble up in what you’re talking about and one is audience and the second is reflection. Are those two key things to move up beyond traditional information literacy to this metaliteracy level?

Trudi: I think that those are two key pieces, but I think, well, there’s the old definition of information literacy and then there’s the newer one, which somewhat influenced by metaliteracy, but I think that often information literacy is thought of primarily as consuming and evaluating information, so not the responsible, creative production of it. It’s also too often, I think, seen in the academic setting as just related to research and not sort of life-wide. I think that that’s another element here.

Tom: In many ways that’s what I think we were really originally working against that original information literacy definition, the ALA definition and also the Association of College and Research Libraries, the original standards, b       which were very prescriptive in the way that they were designed, so that we were as a framework were really just trying to open this up and also take into account the technology piece—not make it all about technology, certainly, but in many ways the advance of web 2.0 and emerging technologies was kind of being, at the time, anyway, sort of avoided. We knew that there’s a real shift happening in our culture and I think that we’re sort of on the other side of that now, but I think that was important to bring that into the learning experience to have students really reflect on those environments and what they’re doing in those environments.

John: You both mentioned the new ACRL information literacy framework. How does metaliteracy relate to that?

Trudi: We developed metaliteracy in part because of a frustration, with this old definition as we were talking about and Tom mentioned the standards really were very prescriptive, very skills based, concentrated on behavioral and cognitive learning domains. Metacognition was not a part of it, so you identified metacognition so that reflection as something new and they didn’t explicitly address the affordances of web 2.0. So I was co-chair of the task force that was convened by the Association of College and Research Libraries and I brought the idea of metaliteracy to the group for consideration. There were a lot of forces at work in developing the structure of the framework and there were like 2000 people weighing in so it’s a very interesting process. Threshold concepts or core concepts was one of the primary features that we used with the framework. I sort of quote from the introduction to the frameworks; there are those ideas in any discipline that are passageways or portals to enlarged understanding or ways of thinking and practicing within that discipline. For example, in biology, evolution would be a threshold concept. That was one element and then the other was metaliteracy. The idea of learners as information creators as well as consumers—which we’ve talked about—definitely has a presence in the framework. There are four learning domains in metaliteracies: behavioral, cognitive, affective and metacognitive. These all have made their way into the framework, so there really is in part a close relationship between the two. For example, the affective domain maps to the whole sections on learner dispositions. I think that there really is a close relationship and I think metaliteracy has gotten additional notice from people because it is explicitly mentioned in the framework.

John: So it’s complementary that they fit well together, they link well together.

Trudi: That’s right.

Tom: I think that’s a good way to put it that they’re complementary, because that also allows each approach to still move forward because we see metaliteracy as this evolving concept and we’ve been working together—we’re working with a team of colleagues called the “Metaliteracy Learning Collaborative” on these ideas, we’re writing together and we’re developing this different MOOC and badging projects. Every time we do something new we’re learning something new and we’re trying to build that into the core ideas here. I think that this idea of complementarity is really important to these two; they’re not mutually exclusive, they work together, and as Trudi mentioned, when we go out and talk to different audiences on this they’re interested in both concepts and working with both. One interesting comment we often hear from people is that with metaliteracy they’ll say you found a way to talk about something that we were trying to do or that we were already doing but you found a name for that really made sense. We really like that: the fact that we were able to name something that really probably was in practice but maybe didn’t have as in-depth of a framework built around it and we like that dialogue with practitioners and something we try to do so this idea of theory and practice for metaliteracy is critically important and allows you to move forward.

Trudi: And the ACRL information literacy framework information literacy is not something that can be taught only by librarians so it’s really directed also towards faculty and administrators. It still seems to have a librarian focus to it, whereas metaliteracy, I think, extends beyond that. Librarians are interested in it but we’re also seeing all sorts of things that are being written or talked about by academics in a really broad range of disciplines.

Tom: And we’ve found that in the books we’ll talk about the two unedited books we’ve done in addition to the first metaliteracy book and we saw evidence of that when we do a call for proposal; it’s really from a wide area of academics. We definitely have librarians, but we also have faculty from many different disciplines, and also instructional designers. That piece of it has been really fascinating as well because we’ve been trying to really open it up to as many people as possible and not seeing it just within one particular discipline.

John: How have faculty and librarians responded to your work?

Trudi: There’s been a lot of interest in it to explore one of the collaborations. Somebody that I’m working with at the University at Albany is a political science professor. This will give you an indication how at least one person has responded to our work. She teaches a 200 level political science course that includes some of the general education competencies, one of which is information literacy, and she was developing this course from Pollock. She came to me to talk about information literacy. We ended up talking about metaliteracy and she was so excited by some of the things we’ve talked about that it would do for her students, so this idea of information creators, the empowerment that she has made metaliteracy sort of a key part of her course. She has the students do about 8 activities connected to metaliteracy. These activities come from a digital badging system that we can talk about a little bit later. She actually has students create an activity that would fit into this digital badging system, which is pretty exciting. This year she asked us to extend what we’re doing and we have been creating questions for the students about what it means to be an information creator, information producer, a teacher, a translator of information and we found this very exciting. It’s not just a collaboration in that she is using some of this material for her students, but her students are creating things for us and she’s giving us ideas. It’s just one example but it’s one where it has become a core part of this course, not only when she teaches it but when others teach it as well.

Tom: Collaboration has been key to what we’ve been doing from the very beginning. The first SUNY IITG we received was really to initiate to launch a metaliteracy learning collaborative and that first project led to the development of our first connectivist MOOC… b eginnings of the digital badging system, although it wasn’t part of the initial grant, but that’s something that we started working on, and also what was most important at the time was the development of the first metaliteracy goals and learning objectives which we’ve recently revised but it was important when we developed that that instead of just Trudi and I working on this together, we really opened it to a SUNY-wide audience that included faculty and librarians. Those goals and learning objectives are available via metaliteracy.org and we recently revised them as well. I think that collaboration with the metaliteracy learning collaborative also led to thinking about metaliteracy in a different way and thinking about those four domains of learning that Trudi mentioned previously; we would look at the metacognitive, which we’ve mentioned is key but also the behavioral, the cognitive and the affective domain so that what we’re really looking at is really the whole person. We’ve also through the metaliteracy learning collaborative we’ve been working on papers together, we’ve been working on these MOOCs; we were lucky enough to have the experience of working on a connectivist MOOC really early on and then I took Coursera MOOC and then a Canvas MOOC and now we’re working on open edX and all those projects involve faculty librarians from Empire State College, the University of Albany and other parts of SUNY, that’s really key. We’re very lucky that we’ve been invited to speak on this which also shows the level of interest and how people are responding to it and many different venues and last year we were lucky enough to present at a conference at the University of Guadalajara in Mexico for this literacy and learning conference and it was just a great experience to be there with international scholars who were talking about literacy in various ways and then we added something by talking about metaliteracy and there’s a lot of interest in what we were talking about. We appreciate those opportunities to have conversations that are both theoretical and practical; the response has just been really positive.

John: We should just clarify the IITG program you mentioned is a SUNY-wide competitive grant program for all of the colleges and universities within SUNY. You were one of the early recipients of that and have received some further funding from that, just to explain that to our listeners who are not as familiar with the SUNY system.

Trudi: John, since you mentioned the innovative instruction technology grant, just to show sort of interest from others, we did get one with School of Education faculty member, actually one from Albany and one from Empire State College because they were really interested in the digital badging, but also the idea of a digital citizen. The plan was and happened that graduate students in education who were going to be teachers would have an opportunity to learn about digital citizenship that’s important for them when they’re teaching, also what digital badging is, so there were a couple of different takeaways. We were able to move metaliteracy or an aspect of metaliteracy into graduate education for educators.

Rebecca: There’s been a lot of mention of metaliteracy badges so maybe we can talk about those?

[LAUGHTER]

Trudi: Yeah, certainly. This was something that developed out of one of those innovative instruction technology grants. We’ve been working on them ever since. What we did was we took the learning goals and objectives for metaliteracy and created open content, very ambitious scheme. There’s four digital badges in the system. Each one of which has anywhere from 12 to 20 activities, starting with lower level quests, moving up to challenges and ultimately you get to these four digital badges. They were written by members of the meta literacy learning collaborative. Tom has written some, I’ve written some. Students have written some, so undergraduate and graduate students they’re being used currently at Albany about 2,500 students have gone through parts of this badging system. The only ones so far who’ve actually earned badges are ones who have taken my courses. It’s content that can be used in classes across a range of disciplines. Also, adaptable to the disciplines. I mentioned earlier the political science professor and sometimes she sort of tweaks the assignments in there so it really relates to what she’s teaching in her political science course. The badge system itself at this point is restricted to University of Albany because there’s a single sign-on process, but we do have a website that has all of the content openly available. People are welcome to use this.

Tom: And from the perspective of someone who has developed some content for this it’s really a fascinating experience because you know that you’re somehow reaching learners that are not in your course but that it’s something that you’re opening and you’re sharing, so this idea of thinking about them as open educational resources that can be then adapted for different contexts. It’s really interesting and exciting to know that something I might create as a learning object could be used by a faculty member here at the University at Albany who’s having their students go through it. Some of them that I developed are based on learning activities I had created in some of my information science courses when I taught here at the university, but I’ve adapted them or updated them. That piece of it from a faculty perspective, as long as you’re open to it, is really engaging and interesting and a way to reach other learners who may not be students in your class but you’re sharing those ideas with them.

Trudi: And I don’t know if it’s ok if I plug a book that I just co-edited with Kelsey O’Brien… Just published this month, September 2018, Teaching with Digital Badges, which was published by Rowman & Littlefield. In that book there is a chapter written by Kelsey O’Brien on the metaliteracy badging system.

Rebecca: Great, you’re both working on a new book together, right?

Trudi: Yes.

Tom: Yes.

Rebecca: Can you talk a little bit about that new book and how it connects to your earlier work?

Tom: Sure, the new book is called “Metaliterate Learning for the Post Truth World.” We’ve shifted somewhat from I think what was a really optimistic view of the connected world and how great it is to be producers of information and be participatory to really trying to further emphasize some pieces that were there but I think needed to be fleshed out a bit more for the new environment we’re in post truth, which is based on confirmation bias and misinformation, false information and questions about new sources and all kinds of misleading facts that are being sent out. We really wanted to take that head-on because we saw metaliteracy in many ways even though it’s an idea that had developed previously as something that is a strong education response to some of the concerns and issues that we’re seeing today. Soon after the 2016 election we wrote a piece about fake news and that term is certainly changed even from the time that we originally wrote it. Wrote a piece for the conversation called “How to Reject Fake News in a Digital World,” so again taking a metaliteracy approach to looking at fake news in a critical way. Since that time even the term fake news, of course, has been weaponized, so we have conflicting thoughts about even using that term based on the research some educators think that it’s important to still keep using it and others want to reject it completely but I think we all generally know the narrative of that. The new book we decided to foreground metaliteracy in this environment and to make it an edited book so that we could engage other educators about this idea. Wasn’t just us but that it was other educators who were dealing with it. About half of the book is very theoretical and the other half of the book is more practical. When we did a call for proposals we tried to intentionally keep that open because we wanted different perspectives on this. I wrote the framing chapter to really talk about post truth, to reframe metaliteracy within this context and to also talk about a new figure that Trudi and I developed together based on the metaliteracy learning characteristics. The new book is going to present a new image, a new figure that further develops the metaliteracy idea from a theoretical perspective and talk about the importance of those characteristics in the post truth world. We’re joined by incredibly prestigious authors who from a theoretical standpoint look at things such as the importance of documentation in metaliteracy, and again, what they’re doing is they’re flushing out pieces of metaliteracy that we have not engaged with yet, so it was really exciting to see that. Another author talks about inoculation theory preparing learners to in many ways be resistant to some of the post truth issues that we’re currently engaged in. Scientific literacy, so there’s a whole chapter on the importance of scientific literacy and looking at it through the lens of metaliteracy. Also, looking at the synergy of word and image and photojournalism, Tom Palmer who teaches here in the journalism program at the University of Albany and it was also a journalist who works for the Times Union wrote that chapter. A few of the chapters do deal with the ACRL framework for information literacy for higher education, so we had that perspective. We were talking previously about both concepts are complementary and we have a few authors who really prove that. We also have a few authors who look at such topics as teaching students to be wrong, genre writing in the first year, writing instruction and the application of poetic ethnography in digital storytelling to create narratives in Philadelphia neighborhoods. I’m very interested in digital storytelling. I mentioned that previously and one of our authors also talks about digital storytelling to empower voices and to encourage students to really raise their voice in the current times that we’re in.

Trudi: And earlier you sort of asked how faculty, other educators, librarians have responded to metaliteracy. I think it’s really interesting. Tom and I did a workshop on metaliteracy at Temple University and a couple of these chapters actually came from people who were in that workshop. It was really sort of exciting to see the immediate impact that that had had.

Rebecca: That’s cool. So this sounds like a really great book; when can I get it?

Trudi: Next spring. [LAUGHTER]

John: Ok. Your current MOOC is a Coursera MOOC but you’re developing a new open edX MOOC. Could you tell us a little bit about that and how that new MOOC will differ from the prior MOOCs, because you’ve had more than one in the past?

Tom: This is part of a continuum of those three MOOCs. We actually wrote a paper in Open Praxis that talked about metaliteracy as a pedagogical framework that was applied in these different MOOCs, so we did a kind of compare and contrast of the different MOOC environments but also talked about our experiences and those different platforms and what it was like, and at the end of that paper one of our conclusions was that what we really needed to do next was create a kind of hybrid MOOC environment because what we had previously was the connectivist MOOC which was our first one and that Stephen Downes type approach. We actually used his grasshopper programming to run that MOOC, then we had the more structured xMOOCS, the Coursera and the Canvas. In many ways the paper was about that but what we decided at the end of the paper was we analyzed it was that we needed a hybrid version and it would it be possible to do that, is there a platform out there that has the learner-centered freeform approach of the connectivist MOOC with some of the structures that were valuable in the video that was really key to the xMOOCS. One of the ideas that propelled this idea forward… We also then, of course, had this shift to this thinking of a transition of kind of a connected world to a post truth world and what does that mean, and because we were working on this book “Metaliterate Learning for the Post Truth World,” we thought that’s a theme for a MOOC. We won’t go out there and call it the metaliteracy MOOC, but it’s a post truth MOOC that’s powered by metaliteracy that really applies the metaliteracy framework to each of the modules within the MOOC, so we’re really excited about that. We did apply for another SUNY IITG and we did receive funding for that, which allows us now to build a team—again it’s another Empire State College, University at Albany team—and we’re really excited about it, we’re developing it now, we’re exploring the open edX environment and as part of that too we’re working with the University of Buffalo because they’ve just launched an instance of open edX for their continuing education program and so they’ve already done a lot of the analysis and a lot of the footwork in terms of creating this instance of open edX on their campus, so they’re letting us experiment with what they’ve done and the idea is that our experience as one of the first two SUNY institutions beyond UV that are using open edX that we will hopefully pave the way for other SUNY faculty librarians that want to develop an open edX MOOC.

Trudi: One of the things that we’d like to do with this—Tom mentioned earlier—we’ve recently revised the metaliteracy learning goals and objectives. We are using those as the framework for this new MOOC. We would like to address issues such as confirmation bias, the role of expertise and authority in today’s environment, issues related to safety, security and personal privacy online, representations of reality in a virtual world and all the while sort of empowering participants to raise and share their voices while rebuilding communities of trust.

Rebecca: Who do you see is the audience for this particular MOOC?

Trudi: I think that we’re really hoping that it’s a very broad audience. We’ve had that, for example, with the Coursera MOOC where there were a lot of international participants everywhere from high school students to non-traditional types of students. We learned about their professions which just ran the gamut and I think that although we do hope to introduce this MOOC as part of courses both at Empire State College and at the University at Albany we’re really hoping that the participants are traditional learners and non-traditional learners. I think that what we’re going to be including in the way of content is something that needs to be broadly disseminated.

Tom: I think because that’s one of the advantages of MOOCs is that they do open up a potentially global audience, so we’re hoping for that international perspective as well, and as Trudi mentioned, we are developing courses so that we could on each of our campuses—I’m calling them wraparound courses—so that the courses that introduce students to the MOOC and they can then earn credit for doing so, because that’s been one of the big questions about MOOCs; can you learn credit, so what we’re doing is creating separate courses and in my version of the course I’m doing a full semester course so that the first half of the course will be introducing students to, well, what is a MOOC? What is post truth? What is metaliteracy? And I have a whole section on how to prepare for success in taking a MOOC, and then that will hopefully prepare them to be a successful learner in a MOOC environment so then they’ll take the six-week course and then there will be reflection piece at the end, which is very metaliteracy, and I actually think that a course about a course is very meta, so we’ve got that piece of it, and that idea to emerge from our very first connectivist experience where we tried to do it for credit and sure, you can talk about this experience at the University at Albany. In particular, in many ways the students were not prepared for the connectivist environment, so what we’re trying to do is in mind, since mine will be a full semester course, is invite students to take it but to really prepare them for being successful in MOOC because we know too that completion rates and MOOCs are not always great, but what if you offer it and prepare students for that environment. I think it is unique enough of an environment where that’s worth exploring.

Trudi: And Tom referred to our connectivist MOOC, which I did use as part of a course, essentially a blended course, and I was amazed when the students actually asked for more in-person class meetings because they couldn’t really grasp the idea of the MOOC and the fact that they were making decisions about their own learning. They were making decisions about which readings would be important. They needed to participate through a personal blog that was sort of elected and shared, and what they essentially did was doubt. I had about a 60% dropout rate in the course and the ones who were left were the ones who just wanted their hands held essentially through the rest of the course and that’s where we really learned that what Tom is going to be doing with his course, which is a full semester course, mine will be a quarter course again, is preparing them for this. This MOOC will be a more directed connectivist MOOC, but it was a very important takeaway.

Tom: And I’m hoping that by doing that it prepares them not only for our MOOC but it opens up the possibility of picking other MOOCs for lifelong learning. So that I think there are potential benefits, even beyond this experience. We’re hoping to launch the MOOC,—we’re developing it now—but we’re hoping to launch it for March 2019. It will be called “Empowering Yourself in a Post Truth World,” which is really important because we really want it to be a positive learning experience and one that provides resources for learners to be successful. You can imagine that talking about the post truth world could be a real downer, but what we really want it to be is a real positive focus of how to address the issues, look at these issues critically, but then to leave with some concrete ways of dealing with it. It also builds on some of the other MOOCs we had. The Coursera MOOC, for example, involved empowering yourself in a connected world and we’re running that now as an on-demand version. So when we first ran it in Coursera we were in the course and it was moving along and we were there in the discussions and following it but then Coursera changed its format a little bit and open up this possibility of on-demand and we actually like that because it allows us to have that content out there and to have learners engage with it in a self-paced way. Up to this point we’ve had, based on the stats we continue to receive from Coursera;—it’s running all the time—we’ve had 1,900 registrants and 900 active learners. We were really happy about that because it really gets some of these concepts out there, and I think it’s probably it’s been out there for a couple years now; it’s probably due for a revision, but that’s one of our projects that we’d like to do eventually, but I think that the post truth MOOC will in many ways build on that as well, so if someone wanted to go back they could look at that on-demand version, but as Trudi mentioned, the post truth MOOC is a six module, six-week learning experience on a very specific topic. I think it will be even more of a clearly-defined focused than even the other one.

John: Would be really nice to have all voters taking in the next couple of years. [LAUGHTER]

Trudi: We would like that.

Tom: Yes, yes.

Rebecca: So you’ve talked a lot about the learner side and some of the tools and materials and MOOCs and things that can help learners become more metaliterate. How do you help faculty coach students through this kind of process? What are the takeaways for faculty? They’ve listened to this episode, they’re really interested in the idea; where do they get started?

Trudi: I think not to just promote our books, but I think that perhaps if they took a look at the two edited volumes they might get a sense of how others are doing it and the range of disciplines is pretty broad, so they might find someone in their own or a related one. I think that that might be a good place to start. I think also taking a look at the learning goals and objectives might provide some ideas of things they’re already doing, but perhaps finding ways to highlight them or frame them slightly differently.

Tom: And not to promote our blog, but metaliteracy.org; everything is in there, including the goals and learning objectives. Summaries of all the books, because we’ve had the blog now for a few years, so it’s interesting even to kind of go back and look at some of the original postings, but it links to the books, it links to all the presentations. The presentations are available, and a few of the keynotes that were recorded are in there. I do think the metaliteracy goals and learning objectives are definitely key because those can be easily applied. Should we mention what we were just invited to write because that would actually address this audience as well?

Trudi: Yeah, we’re going to be writing a piece for higher education jobs. They have a couple of newsletters and going to be talking about the importance of teaching or emphasizing metaliteracy on campus for administrators and also what instructors can do. We think that those are going to be appearing in November.

Tom: Because we’ve had a commitment to making everything open—I know it’s a lot to look for, but we do have the metaliteracy YouTube channel, the blog, of course, the presentations and a lot of these resources were intentionally constructed that way so that other educators could use them, so go to “Empowering Yourself in a Connected World” on Coursera and access the videos, use the learning activities in any way you want. Go to the first module; there’s a PDF in there that has the metaliteracy learner roles and we’ve used them as learning activities in our own classes and it has some reflective questions, so you have this diagram that really explains the different roles a learner could take and then it has questions for learners to really think about those roles. So I think a lot of those resources can be adapted in any way that people want, and it’s really an open concept, so we want people to get involved and apply their own approaches to this.

Rebecca: We wrap up by always asking: what next? You’ve given us so much, but what else? [LAUGHTER]

Tom: That’s a really good question. The next book that we mentioned is coming out in the spring. We’re currently working on the open edX MOOC, “Empowering Yourself in a Post Truth World.” We also, of course, will be launching that in the spring.

Trudi: With the digital badging system we would like to if we can find some more funding have a learning pathway portion to it where instructors can really tailor the information or add components for their own disciplines. We’re also working on a metaliteracy module for another innovative instruction technology grant funded project called “I succeed,” which is being developed in western New York, and they’ve asked us to provide a module on metaliteracy and this is going to be directed to high school students who aspire to college or first year college students and can be used by instructors, so we are putting that together with four units.

Tom: We have a few upcoming panel presentations that OLC accelerate in Florida in November.

John: I may see you there.

Tom: Oh, great! I haven’t been there in a couple years so I’m looking forward to getting back and that’s such a great conference.

John: It is.

Tom: And of course there will be continued research and writing. I’m certain that the open edX experience that we’re currently immersed in will lead to a paper, and we’d like to do a research project that assesses the application of the metaliteracy goals and learning objectives. So much of what we’ve been doing is really theorizing and talking about practice and developing these environments, but we would like to delve into that a bit more. We might have an opportunity to work with an international scholar that we met last year at the University of Guadalajara, but we’re not sure about that if that’s going to happen, but that would allow us to really expand the metaliteracy concept: working with international scholars. So there’s a lot of possibilities. Perhaps a coil courses in our future, and that’s another SUNY resource; it’s a collaborative online international learning environment. I think that’s something that we would love to do with an international scholar, so we’ll see if that happens some day. A lot of ideas, got a lot going on, but we’ll see.

John: You got a nice track record of being really productive with us.

Rebecca: Thanks so much for joining us and spending time and giving us lots of things to think about.

John: Yeah, you’re doing some wonderful work.

Trudi:Thank you.

Tom: Thank you so much, we really enjoyed this.

Trudi: Yeah.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

37. Evidence is Trending

Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode, Michelle Miller joins us to discuss recent research that will likely shape the future of higher education.

Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.

Show Notes

Rebecca: Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode we talk to a cognitive psychologist about recent research that will likely shape the future of higher education.
[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Michelle Miller. Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.
Welcome, Michelle!

Michelle: Hi, I’m so glad to be here.

Rebecca: Thanks for joining us.
Today’s teas are:

Michelle: I’m drinking a fresh peppermint infused tea, and it’s my favorite afternoon pick-me-up.

Rebecca: …and it looks like it’s in a really wonderfully designed teapot.

Michelle: Well, thank you… and this is a thrift store find… one of my favorite things to do. Yeah, so I’m enjoying it.

John: I have Twinings Blackcurrant Breeze.

Rebecca: …and I’m drinking chai today.

Michelle: Pretty rough.

John: We invited you here to talk a little bit about things that you’ve been observing in terms of what’s catching on in higher education in terms of new and interesting innovations in teaching.

Michelle: Right, that’s one of things that I really had the luxury of being able to step back and look at over this last semester and over this last spring when I was on sabbatical… One of the really neat things about my book Minds Online, especially now that it’s been out for a few years, is that it does open up all these opportunities to speak with really engaged faculty and others, such as: instructional designers, librarians, academic leadership, educational technology coordinators… all these individuals around the country who are really, really involved in these issues. It’s a great opportunity to see how these trends, how these ideas, how these innovations are rolling out, and these can be some things that have been around for quite some time and just continue to rock along and even pickup steam, and some newer things that are on the horizon.

John: You’ve been doing quite a bit of traveling. You just got back from China recently, I believe.

Michelle: I sure did. It was a short visit and I do hope to go back, both to keep getting involved in educational innovations there and, hopefully, as a tourist as well. So, I was not there for very long but I had the opportunity to speak at Tsinghua University in Beijing, which is a really dynamic institution that’s been around for about a hundred years. For a while in its history it specialized in things like engineering education polytechnic, but now it’s really a selective comprehensive university with very vibrant graduate and undergraduate programs that are really very relatable for those of us in the United States working in similar contexts. My invitation was to be one of the featured speakers at the Future Education, Future Learning Conference, which was a very interdisciplinary gathering of doctoral students, faculty, even others from the community, who were all interested in the intersection of things like technology, online learning, MOOCs even, and educational research (including research into the brain and cognitive psychology), and bringing all of those together… and it was a multilingual conference. I do not speak Chinese but much of the conference was in both English and Chinese and so I was also able to really absorb a lot of these new ideas. So yes, that was a real highlight of my sabbatical semester and one that I’m going to be thinking about for quite some time.

I should say that part of what tied in there as well is that Minds Online, I’ve just learned, is going to be translated into Chinese and that’s going to come out in May 2019. So, I also got to meet with some of the people who were involved in the translation… start to put together some promotional materials such as videos and things like that.

Rebecca: Cool.

John: Excellent.

Rebecca: So, you’ve had a good opportunity, as you’ve been traveling, to almost do a scavenger hunt of what faculty are doing with evidence-based practices related to your book. Can you share some of what you’ve found or heard?

Michelle: This theme of evidence-based practice, and really tying into the findings that have been coming out of cognitive psychology for quite some time, that really is one of the exciting trends and things that I was really excited to see and hear for so many different quarters I visited in different institutions… and so I would say definitely, this is a trend that is continuing and is increasing. There really does continue to be a lot of wonderful interest and wonderful activity around these real cognitively informed approaches to teaching, and what I think we could call scientifically based and evidence-based strategies. One form this has taken is Josh Eyler’s new book, called How Humans Learn: The Science and Stories behind Effective College Teaching. This is a brand new book by a faculty development professional, and a person coming out of the humanities, actually, who’s weaving together even from his humanities background everything from evolutionary biology to classical research in early childhood education to the latest brain-based research. He’s weaving this together into this new book for faculty. So, that’s one of the things that I’ve noticed and then there’s the issue which i think is another great illustration of best-known practice which is the testing effect and retrieval practice.

John: One of the nice things is how so many branches of research are converging… testing in the classroom, brain-based research, and so forth, are all finding those same basic effects. It’s nice to see such robust results, which we don’t always see in all research in all disciplines.

Rebecca: …and just breaking down the silos in general. The things are all related and finding out what those relationships are… exploring those relationships… is really important and it’s nice to see that it’s starting to open up.

John: We should also note that when you visited here, we had a reading group and we had faculty working on trying to apply some of these concepts, and they’re still doing that… and they still keep making references back to your visit. So, it’s had quite a big impact on our campus.

Michelle: This wasn’t true, I don’t think, when I first entered the teaching profession… and even to the extent when I first started getting interested in applied work in course redesign and in faculty professional development. you would get kind of this pushback or just strange looks when you said “Oh, how about we bring in something from cognitive psychology” and now that is just highly normalized and something that people are really speaking across the curriculum… and taking it and running with it in a lasting ongoing way, not just as a “Oh, well that was an interesting idea. I’m going to keep doing what I’m doing” but really people making some deep changes as you mentioned. This theme of breaking down silos… I mean I think if there’s kind of one umbrella trend that all of these things fits under it’s that breakdown of boundaries. So, that’s one that I keep coming back to, I know, in my work.

So, the idea of retrieval practice, drilling down on that one key finding which goes back a very long ways in cognitive psychology. I think of that as such a good example of what we’re talking about here… about how this very detailed effect in cognition and yet it does have these applications across disciplinary silos. Now when I go to conferences and I say “Okay, raise your hand. How many people have ever heard of retrieval practice? How many people have ever heard of the testing effect? How many people have heard of the book Make it Stick (which really places this phenomena at its center)?” and I’m seeing more hands raising.

With retrieval practice, by the way, we’re talking about that principle that taking a test on something, that retrieving something from memory actively, has this huge impact on future memorability of that information. As its proponents like to say, tests are not neutral from a memory or from a learning standpoint… and while some of the research has focused on very kind of stripped-down laboratory style tasks like memorizing words pairs, there are also some other research projects showing that it does flow out to more realistic learning situations.

So, more people simply know about this, and that’s really the first hurdle, oftentimes, with getting this involved disciplinary sometimes jargon riddled research out there to practitioners and getting it into their hands. So, people heard of it and they’re starting to build this into their teaching. As I’ve traveled around I love to hear some of the specific examples and to see it as well crop up in scholarship of teaching and learning.

Just recently, for example, I ran across and really got into the work of Bruce Kirchhoff who is at University of North Carolina – Greensboro and his area is botany and plant identification. He has actually put together some different really technology-based apps and tools that students and teachers can use in something like a botany course to rehearse and review plant identification. He says in one of his articles, for example, that there just isn’t time in class to really adequately master plant identification. It’s just too complex of a perceptual and cognitive and memory test to do that. So, he really built in from the get-go very specific principles drawn from cognitive psychology… so, the testing effect is in there… there’s different varieties of quizzing and it all is about just getting students to retrieve and identify example after example. It brings in also principles such as interleaving, which we could return to in a little bit, but has to do with the sequencing of different examples… their spacing… So, that’s even planned out exactly how and when students encounter different things that they’re studying. It’s really wonderful. So, for example he and his colleagues put out a scholarship of teaching and learning article talking about how this approach was used effectively in veterinary medicine students who have to learn to identify poisonous plants that they’ll see around their practice. This is something that can be time-consuming and very tough, but they have some good data showing that this technology enhanced cognitively based approach really does work. That’s one example. Coincidentally, I’ve seen some other work in the literature, also on plant identification, where the instructors tagged plants in an arboretum… they went around and tagged them with QR codes… that students can walk up to a plant in the real environment with an iPad… hold the iPad over it… and it would immediately start producing quiz questions that were are specific to exactly the plants they were looking at.
So, those are some of the exciting things that people are taking and running with now that this principle is out there.

Rebecca: What I really love about the two stories that you just shared was the faculty are really designing their curriculum and designing the learning experiences with the students in mind… and what students need and when they need it. So, not only is it employing these cognitive science principles, but it’s actually applying design principles as well. It’s really designing for a user experience and thinking about the idea that if I need to identify a plant, being able to identify it in this situation in which I would need to identify it in makes it much more dynamic I think for a student… but also really meets them where they’re at and where they need it.

John: …and there’s so many apps out there now that will do the plant identification just from imagery without the QR code, that I can see it taking it one step further where they can do it in the wild without having that… so they can build it in for plants that are in the region without needing to encode that specifically for the application.

Michelle: I think you’re absolutely right once we put the technology in the hands of faculties who, as I said, they’re the one to know: “Where are my students at? Where are the weak points? Where are the gaps that they really need to bridge?” and that’s where their creativity is giving rise to all these new applications… and sometimes these can be low-tech as well… or also things that we can put in a face-to-face environment… and I’d like to to share just some experiences that I’ve had with this over the last few semesters.

In addition to trying to teach online with a lot of technology, I also have in my teaching rotation a small required course in research methods in psychology which can be a real stumbling block… the big challenge course… it’s kind of a gateway course to continued progress in our major. So, in this research methods course, some of the things that I’ve done around assessment and testing to really try again to stretch that retrieval practice idea… to make assessments really a more dynamic part of the course and more central part of the course… to move away from that idea that tests are just this kind of every now and again this panic mode opportunity for me to kind of measure in sorts of students and judge them… to make good on that idea that tests are part of learning. So, here’s some of the things that I try to do. For one thing, I took time out of the class almost every single class meeting as part of the routine to have students first of all generate quiz questions out of their textbook. So, we do have a certain amount of foundational material in that course as well as a project and a whole lot of other stuff is going on. So they need to get that foundational stuff.

Every Tuesday they would come in and they knew their routine: you get index cards and you crack your textbook and you generate for me three quiz questions. Everybody does it. I’m not policing whether you read the chapter or not. It’s active… they’re generating it… and also that makes it something like frequent quizzing. That’s a great practical advantage for me since I’m not writing everything. They would turn those in and I would select some of my favorites I would turn those into a traditional looking paper quiz and hand that out on Thursday. I said “Hey, take this like a realistic quiz.” I had explained to them that quizzes can really boost their learning, so that was the justification for spending time on it and then I said: “You know what? I’m not going to grade it either. You take it home because this is a learning experience for you. It’s a learning activity.” so we did that every single week as those students got into that routine.

The second thing that I did to really re-envision how assessment testing and quizzing worked in this particular course, was something inspired by different kinds of group testing and exam wrapper activities I’ve seen, particularly coming out of the STEM field, where there’s been a lot of innovation in this area. What I would do is… we had these high stakes exams at a few points during the semester. But, the class day after the exam, we didn’t do the traditional “Let’s go over the exam.” [LAUGHTER] That’s kind of deadly dull, and it just tends to generate a lot of pushback from students… and as we know from the research, simply reviewing… passing your eyes over the information… is not going to do much to advance your learning. So, what I would do is… I would photocopy all those exams, so it has a secure copy. They were not graded. I would not look at this before we did this… and I would pass everybody’s exams back to them along with a blank copy of that same exam. I assigned them to small groups and I said “Okay, here’s your job. Go back over this exam, fill it out as perfectly as you can as a group, and to make it interesting I said I will grade that exam as well, the one you do with your group, and anything you get over 90% gets added to everybody’s grade. This time it was open book, it was open Google, it was everything except you can’t ask me questions. So, you have each other and that’s where these great conversations started to happen. The things that we always want students to say. So, I would eavesdrop and hear students say “Oh, well you know what, I think on this question she was really talking about validity because reliability is this other thing…” and they’d have a deep conversation about it. I’m still kind of going back through the numbers to see what are the impacts of learning? Are there any trends that I can identify? But, I will say this: in the semesters that I did this, I didn’t have a single question ever come back to me along the lines of “Well, this question was unclear. I didn’t understand it. I think I was graded unfairly.” it really did shut all that down and again extended the learning that I feel students got out of that. Now it meant a big sacrifice of class time, but I feel strongly enough about these principles that I’m always going to do this in one form or another anytime I can can in face-to-face classes.

Rebecca: This sounds really familiar, John.

John: I’ve just done the same, or something remarkably similar, this semester, in my econometrics class which is very similar to the psych research methods class. I actually picked it up following a discussion with Doug McKee. He actually was doing it this semester too. He had a podcast episode on it. It sounded so exciting, I did something… a little bit different. I actually graded it but I didn’t give it back to them because I wanted to see what they had the most trouble with, and then I was going to have them only answer the ones in a group that they struggled with… and it turned out that that was pretty much all them anyway. So, it’s very similar to what you did except I gave them a weighted average of their original grade and the group grade and all except one person improved and the one person’s score went down by two points because the group grade was just slightly lower… but he did extremely well and he wasn’t that confident. The benefits to them of that peer explanation and explaining was just tremendous and it was so much more fun for them and for me and, as you said, it just completely wiped out all those things like “Well, that was tricky” because when they hear their peers explaining it to them the students were much more likely to respond by saying “Oh yeah, I remember that now” and it was a wonderful experience and I’m gonna do that everywhere I can.

In fact. I was talking about it with my TA just this morning here at Duke and we’re planning to do something like that in our classes here at TIP this summer, which i think is somewhat familiar to you from earlier in your academic career.

Michelle: That is right we do have this connection. I was among, not the very first year, but I believe the second cohort of Talent Identification Program students who came in, I guess you would call it now, middle school (back then, it was called junior high) and what a life-transforming experience. We’ve had even more opportunities to talk about the development of all these educational ideas through that experience.

John: That two-stage exam is wonderful and it’s so much more positive… because it didn’t really take, in my class, much more time, because I would have spent most of that class period going over the exam and problems they had. But the students who did well would have been bored and not paying much attention to it; the students who did poorly would just be depressed and upset that they did so poorly… and here, they were actively processing the information and it was so positive.

Michelle: That’s a big shift. We really have to step back and acknowledge that, I think. that is a huge shift in how we look at assessment, and how we think about the use of class time… and it’s not just “Oh my gosh, I have to use every minute to put such content in front of the students.” Just the fact that more of us are making that leap, I think, really is evidence this progress is happening… and we see also a lot of raised consciousness around issues such as learning styles. That’s another one that, when I go out and speak to faculty audiences, 10 years ago you would get these shocked looks or even very indignant commentary when you say “Ok, this idea of learning styles, in the sense that say there are visual learners, auditory learners, what I call sensory learning styles (VAK is another name it sometimes goes by). The idea that that just holds no water from a cognitive point of view…” People were not good with that, and now when I mentioned that at a conference, I get the knowing nods and even a few groans… people like “Oh, yeah. we get that. Now, K-12, which I want to acknowledge it’s not my area, but I’m constantly reminded by people across the spectrum that it’s a very different story in K-12. So, setting that aside… but this is what I’m seeing… that faculty are realizing… they’re saying “Oh, this is what the evidence says…” and maybe they even take the time to look at some of the really great thinkers and writers who put together the facts on this. They say “You know what? I’m not going to take my limited time and resources and spend that on this matching to styles when the styles can’t even be accurately diagnosed and are of no use in a learning situation. So, that’s another area of real progress.

Rebecca: What I am hearing is not just progress here in terms of cognitive science, but a real shift towards really thinking about how students learn and designing for that rather than something that would sound more like a penalty for grade like “Oh, did you achieve? Yes or no…” but, rather here’s an opportunity if you didn’t achieve to now actually learn it… and recognize that you haven’t learned it, even though it might seem really familiar.

John: Going back to that point about learning styles. It is spreading in colleges. I wish it was true at all the departments at our institution, but it’s getting there gradually… and whenever people bring it up, we generally remind them that there’s a whole body of research on this and I’ll send them references but what’s really troubling is in my classes the last couple years now, I’ve been using this metacognitive cafe discussion forum to focus on student learning… and one of the week’s discussions is on learning styles and generally about 95 percent of the students who are freshmen or sophomores (typically) come in with a strong belief in learning styles… where they’ve been tested multiple times in elementary or middle school… they’ve been told what their learning styles are… they’ve been told they can only learn that way… It discourages them from trying to learn in other ways and it does a lot of damage… and I hope we eventually reach out further so that it just goes away throughout the educational system.

Rebecca: You’ve worked in your classes, Michelle, haven’t you to help students understand the science of learning and use that to help students understand the methods and things that you’re doing>

Michelle: Yes, I have. I’ve done this in a couple of different ways. Now, partly, I get a little bit of a free pass in some of my teaching because I’m teaching the introduction to psychology or I’m teaching research methods where I just happen to sneak in as the research example will be some work on say attention or distraction or the testing effect. So, I get to do it in those ways covertly. I’ve also had the chance, although it’s not on my current teaching rotation… I’ve had the chance to also take it on as in freestanding courses. As many institutions are doing these days… it’s another trend… and what Northern Arizona University, where I work, has different kinds of freshmen or first-year student offering for courses they can take, not in a specific disciplinary area, but that really crossed some different areas of the student success or even wellbeing. So, I taught a class for awhile called Maximizing Brain Power that was about a lot of these different topics. Not just the kind of very generic study skills tip… “get a good night’s sleep…” that kind of thing… but really some again more evidence-based things that we can tell students and you can really kind of market it… and I think that we do sometimes have to play marketers to say “Hey, I’m going to give you some inside information here. This is sort of gonna be your secret weapon. So, let me tell you what the research has found.”

So, those are some of the things that I share with students… as well as when the right moment arises, say after an exam or before their first round of small stakes assessments, where they’re taking a lot of quizzes… to really explain the difference between this and high stakes or standardized tests they may have taken in the past. So, I do it on a continuing basis. I try to weave it into the disciplinary aspect and I do it in these free-standing ways as well… and I think here’s another area where I’m seeing this take hold in some different places… which is to have these free-standing resources that also just live outside of a traditional class that people can even incorporate into their courses… if say cognitive psychology or learning science isn’t their area… that they can bring in, because faculty really do care about these things. We just don’t always have the means to bring them in in as many ways as we would like.

John: …and your Attention Matters project was an example of that wasn’t it? Could you tell us a little bit about that?

Michelle: Oh, I’d love to… and you know this connects to what it seems to be kind of an evergreen topic in the teaching and learning community these days, which is the role of distracted students… and I know this past year there just have been these one op-ed versus another. There’s been some really good blog posts by some people I really like to follow in the teaching and learning community such as Kevin Gannon talking about “Okay, do you have laptops in the classroom? and what happens when you do?” and so I don’t think that this is just a fad that’s going away. This is something that the people do continue to care about, and this is where the attention matters project comes in.

This was something that we conceptualized and put together a couple years ago at Northern Arizona University with myself, and primarily I collaborated with a wonderful instructional designer who also teaches a great deal… John Doherty. So, how this came about is I was seeing all the information on distraction… I’m really getting into this as a cognitive psychologist and going “Wow, students need to know that if they’re texting five friends and watching a video in their class. It’s not going to happen for them.” I was really concerned about “What can I actually do to change students minds?” So, my way of doing this was to go around giving guests presentations in every classes where people would let me burn an hour of their class time… and not a very scalable model… and John Doherty respectfully sat through one my presentations on this and then he approached me and said “Look, you know, we could make a module and put this online… and it could be an open access within the institution module, so that anybody at my school can just click in and they’re signed up. We could put this together. We could use some really great instructional design principles and we could just see what happens… and I bet more people would take that if it were done in that format. We did this with no resources. We just were passionate about the project and that’s what we did. We had no grant backing or anything. We got behind it. So, what this is is about a one- to two-hour module that, it’s a lot like a MOOC in that it there’s not a whole lot of interaction or feedback, but there are discussion forums and it’s very self-paced in that way… so one- to two-hour mini MOOCs that really puts at the forefront demonstrations and activities… so we don’t try to convince students about problems with distraction and multitasking… we don’t try to address that just by laying a bunch of research articles on them… I think that’s great if this were a psychology course, but it’s not. So, we come at it by linking them out to videos, for example, that we were able to choose, that we feel really demonstrate in some memorable ways what gets by us when we aren’t paying attention… and we also give students some research-based tips on how to set a behavioral plan and stick to it… because just like with so many areas of life, just knowing that something is bad for you is not enough to really change your behavior and get you not to do that thing. so we have students talking about their own plans and what they do when, say, they’re having a boring moment in class, or they’re really really tempted to go online while they’re doing homework at home. What kinds of resolutions can they set or what kind of conditions can make that that will help them accomplish that. Things like the software blockers… you set a timer on your computer and it can lock you out of problematic sites… or we learned about a great app called Pocket Points where you actually earn spendable coupon points for keeping your phone off during certain hours. This is students talking to students about things that really concern them and really concern us all because I think a lot of us struggle with that.

So, we try to do that… and the bigger frame for this as well is this is, I feel, a life skill for the 21st century… thinking about how technology is going to be an asset to you and not detract from what you accomplish in your life. What a great time to be reflecting on that, when you’re in this early college career. so that’s what we try to do with the project…and we’ve had over a thousand students come through. They oftentimes earn extra credit. Our faculty are great about offering small amounts of extra credit for completing this and we’re just starting to roll out some research showing some of the impacts… and showing it in a bigger way just how you can go about setting up something like this.

Rebecca: I like that the focus seems to be on helping students with a life skill rather than using technology is just a blame or an excuse. We’re in control of our own behaviors and taking ownership over our behaviors is important rather than just kind of object blaming.

Michelle: So, looking at future trends, I would like to see more faculty looking at it in the way that you just described, Rebecca, as this is a life skill and it’s something that we collaborate on with our students… not lay down the law… because, after all, students are in online environments where we’re not there policing that and they do need to go out into work environments and further study and things like that. So, that’s what I feel is the best value. For faculty who are looking at this, if they don’t want to do… or don’t have the means to do something really formal like our Attention Matters approach, just thinking about it ahead of time… I think nobody can afford to ignore this issue anymore and whether you go the route of “No tech in my classroom” or “We’re going to use the technology in my classroom“ or something in between… just reading over, in a very mindful way, not just the opinion pieces, but hopefully also a bit of the research, I think, can help faculty as they go in to deal with this… and really to look at it in another way, just to be honest, we also have to consider how much of this is driven by our egos as teachers and how much of it is driven by a real concern for student learning and those student life skills. I think that’s where we can really take this on effectively and make some progress when we are de-emphasizing that ego aspect and making sure that it really is about the students.

John: We should note there’s a really nice chapter in this book called Minds Online: Teaching Effectively with Technology that deals with these types of issues. It was one of the chapters that got our faculty particularly interested in these issues… on to what extent technology should be used in the classroom… and to what extent it serves as a distraction.

Michelle: I think that really speaks to another thing which I think is an enduring trend… which is the emphasis on really supporting the whole student in success and what we’ve come to call academic persistence… kind of a big umbrella term that has to do with, not just succeeding in a given class, but also being retained… coming back after the first year. As many leaders in higher education point out, this is as a financial issue. As someone pointed out, it does cost a lot less to hang on to the students you have instead of recruiting more students to replace ones who are lost. This is, of course, yet another really big shift in mindset of our own, because after all we did used to measure our success by “Hey, I flunked this many students out of this course” or” Look at how many people have to switch into different majors…our major is so challenging…”

So, we really have turned that thinking around and this does include faculty now. I think that we did used to see those silos. We had that very narrow view of “I’m here to convey content. I’m here to be an expert in this discipline, and that’s what I’m gonna do…” and sure, we want to think about things like do students have learning skills? Do they have metacognition? Are they happy and socially connected at the school? Are they likely to be retained so that we can have this robust university environment?

We had people for that, right? It used to be somebody else’s job… student services or upper administration. They were the ones who heard about that and now I think on both sides we really are changing our vision. More and more forward-thinking faculty are saying “You know what? Besides being a disciplinary expert, I want to become at least conversant with learning science. I want to become at least conversant with the science of academic persistence…” There is a robust early literature on this and that’s something that we’ve been working on at NAU over this past year as well… kind of an exciting newer project that I like very much. We’ve started to engage faculty in a new faculty development program called Persistence Scholars and this is there to really speak to people’s academic and evidence-based side, as well as get them to engage in some perspective-taking around things like the challenges that students face and what it is like to be a student at our institution. We do some really selected readings in the area we look at things like mindset… belongingness… these are really hot areas in that science of persistence… in that emerging field. But, we have to look at it in a really integrated way.

It’s easy for people to say just go to a workshop on mindset and that’s a nice concept, but we wanted to think about it in this bigger picture… really know what are some of the strengths of that and why? Where do these concepts come from? What’s the evidence? That’s something that I think is another real trend and I think as well we will see more academic leaders and people in staff and support roles all over universities needing to know more about learning science. There are still some misconceptions that persist, as we’ve talked about. We’re making progress in getting rid of some of these myths around learning, but I will say… I’m not gonna name any names… but, every now and again I will hear from somebody who says “Oh well, we need to match student learning styles” or “Digital natives think differently, don’t you know?” and I have to wonder whether that’s a great thing. I mean, these are oftentimes individuals that have the power to set the agenda for learning all over a campus. Faculty need to be in the retention arena and I think that leaders need to be in the learning science arena. The boundaries is breaking down and it’s about time.

Rebecca: One of the things that I thought was really exciting with the reading groups that we’ve been having on our campus… that we started with your book, but then we’ve read Make it Stick and Small Teaching since… is that a lot of administrators in a lot of different kinds of roles engaged with us in those reading groups, it wasn’t just faculty. There was a mix of faculty, staff, and some administrators, and I think that that was really exciting. For people who don’t have the luxury of being in your persistence scholar program, what would you recommend they read to get started to learn more about the science of persistence?

Michelle: I really, even after working with this for quite some time, I loved the core text that we have in that program, which is Completing College by Vincent Tinto. It’s just got a great combination of passionate and very direct writing style. So, there’s no ambiguity, there’s not a whole lot of “on the one hand this and on the other hand that.” It’s got an absolutely stellar research base, which faculty of course appreciate… and it has a great deal of concrete examples. So, in that book they talk about “okay, what does it mean to give really good support to first semester college students? What does that look like?” and they’ll go out and they’ll cite very specific “Here’s a school and here’s what they’re doing… here’s what their program looks like… here’s another example that looks very different but gets at the same thing.” So, that’s one of the things that really speak to our faculty… that they really appreciated and enjoyed.

I think that as well we tested good feedback about work that’s come out of the David Yeager and his research group on belongingness and lay theories, and lay theories is maybe a counterintuitive term for kind of a body of ideas about what students believe about academic success and why some people are successful and others are not and how those beliefs can be changed sometimes through relatively simple interventions and when it happens we see great effects such as the narrowing of achievement gaps among students who have more privilege or less privileged backgrounds… and that’s something that, philosophically, many faculty really really care about but they’ve never had the chance to really learn “Okay, how can I actually address something like that with what I’m doing in my classroom, and how can I really know that the things that I’m choosing do have that great evidence base…”

John: …and I think that whole issue is more important now and is very much a social justice issue because, with the rate of increase we’ve seen in college cost inflation, people who start college and don’t finish it are saddled with an awfully high burden of debt. The rate of return to a college degree is the highest that we’ve ever seen and college graduates end up not only getting paid a lot more but they end up with more comfortable jobs and so forth… and if we really want to move people out of poverty and try to reduce income inequality, getting more people into higher education and successfully completing higher education is a really important issue. I’m glad to see that your institution is doing this so heavily and I know a lot of SUNY schools have been hiring Student Success specialists. At our institution they’ve been very actively involved in the reading group, so that message is spreading and I think some of them started with your book and then moved to each of the others. So, they are working with students in trying to help the students who are struggling the most with evidence-based practices …and I think that’s becoming more and more common and it’s a wonderful thing.

Rebecca: So, I really liked Michelle that you were talking about faculty getting involved in retention and this idea of helping students develop persistence skills, and also administrators learning more about evidence-based practices. There’s these grassroots movements happening in both of these areas. Can you talk about some of the other grassroots movements that are working toward, or efforts that faculty are making to engage students and capture their attention and their excitement for education?

Michelle: Right, and here I think a neat thing to think about too is just it’s the big ambitious projects… the big textbook replacement projects or the artificial intelligence informed adaptive learning systems… those are the things that get a lot of the press and end up in The Chronicle of Higher Education that we read about… But, outside of that, there is this very vibrant community and grassroots led scene of developing different technologies and approaches. So, it really goes back for a while. I mean, the MERLOT database that I do talk about in Minds Online has been trove for years of well hidden gems that take on one thing in a discipline and come at it from a way that’s not just great from a subject-matter perspective but brings up the new creative approaches. In the MERLOT database, for example, there’s a great tutorial on statistical significance and the interrelationship between statistical significance and issues like simple sizes. You know, that’s a tough one for students, but it has a little animation involving a horse and a rider that really turns it into something that’s very visual… that’s very tangible… and it really actually tying into analogies, which is a well-known cognitive process that can support the advancement of learning something new. There is something on fluid pressures in the body that was treated for nursing students by nurses, and it’s got an analogy of a soaker hose that this is really fun and is actually interactive. So, those are the kinds of things. The PhET project, P-h-E-T which comes out of University of Colorado, that has been around for a while… again, faculty-led and a way to have these very useful interactive simulations for concepts in physics and chemistry. So, that’s one. CogLab, that’s an auxiliary product that I’ve used for some time in like hundred psychology courses that simulates very famous experimental paradigms which are notoriously difficult to describe on stage for cognitive psychology students. That started out many years ago as a project that very much has this flavor of “We have this need in our classroom. We need something interactive. There’s nothing out there. Let’s see what we can build.” It has since then picked up and turned into a commercial product, but that’s the type of thing that I’m seeing out there.

Another thing that you’ll definitely hear about if you’re circulating and hearing about the latest project is virtual reality for education. So, with this it seems like, unlike just a few years ago, almost everywhere you visit you’re going to hear that “Oh, we’ve just set up a facility. We’re trying out some new things.” This is something that I also heard about when I was talking to people when I was over in China. So, this is an international phenomenon. It’s going to pick up steam and definitely go some places.

What also strikes me about that is just how many different projects there are. Just when you’re worried that you’re going to be scooped because somebody else is going to get there first with their virtual reality project you realize you’re doing very very different things. So, I’ve seen, for example, it used in a medical application to increase empathy among medical students… and I took a six or seven minute demonstration that just was really heart-rending, simulating the patient experience with a particular set of sensory disorders… and at Northern Arizona University we have a lab that is just going full-steam in coming up with educational applications such as interactive organic chemistry tutorial that is is just fascinating. We actually completed a pilot project and are planning to gear up a much larger study next semester looking at the impacts of this. So, this is really taking off for sure.

But, I think there are some caveats here. We still really need some basic research on this… not just what should we be setting up and what the impacts are but how does this even work? In particular, what I would like to research in the future, or at least see some research on, is what kinds of students… what sort of student profile… really gets the most out of virtual reality for education. Because amidst all the very breathless press that’s going on about this now and all the excitement, we do have to remember this is a very, very labor intensive type of resource to set up. You’re not just going to go home and throw something together for the next week. It takes a team to build these things and to complete them as well. If you have, say, a 300 student chemistry course (which is not atypical at all… these large courses), you’re not going to just have all of them spend hours and hours and hours doing this even with a fairly large facility. It’s a very hands-on thing to guide them through this process, to provide the tech support, and everything else.

So, I think really knowing how we can best target our efforts in this area, so that we can build the absolute best, with the resources we have, and maybe even target and ask the students who are most likely to benefit. I think those are some of the things that we just need to know about this. So, it’s exciting for somebody like me who’s in the research area. I see this as a wonderful open opportunity… but those are some of the real crossroads we’re at with virtual reality right now.

Rebecca: I can imagine there’s a big weighing that would have to happen in terms of expense and time and resources needed to startup versus what that might be saving in the long run. I can imagine if it’s a safety thing that you want to do a virtual reality experience, like saving people’s lives and making sure that they’re not going to be in danger as they practice particular skills, could be a really good investment in these… spending the resources to make that investment… or if it’s a lot of travel that would just be way too expensive to bring a bunch of students to a particular location… but you could virtually… it seems like it would be worth the start-up costs and those are just two ideas off the top of my head where it would make sense to bend all of that resource and time.

John: …and equipment will get cheaper. Right now, it’s really expensive for computers that have sufficient speed and graphics processing capability and the headsets are expensive, but they will come down in price, but as you said, it’s still one person typically and one device… so it doesn’t scale quite as well as a lot of other tools or at least not at this stage.

Rebecca: From what I remember, Michelle, you wrote a blog post about [a] virtual reality experience that you had. Can you share that experience, and maybe what stuck with you from that experience?

Michelle: Right, so I had the opportunity, just as I was getting to collaborate with our incredible team at the immersive virtual reality lab at NAU… one of the things I was treated to was about an hour and a half in the virtual reality setup that they have to explore some of the things that they had… Giovanni Castillo, by the way, is creative director of the lab and he’s the one who was so patient with me through all this. We tried a couple of different things and of course there’s such a huge variety of different things that you can do.
There’s a few things out there like driving simulators that are kind of educational… they’re kind of an entertainment… but he was just trying to give me, first of all, just a view of those… and I had to reject a few of them… I will say, initially, because I am one of the individuals who tends to be prone to motion sickness. So, that limits what I can personally do in VR and that is yet another thing that we’re gonna have to figure out. At least informally, what we hear is that women in particular tend to experience more of this. So, I needed, first of all, to go to a very low motion VR. I wasn’t gonna be whizzing through these environments. That was not going to happen for me. So, we did something that probably sounds incredibly simplistic, but it just touched me to my core… which is getting to play with Google Earth. You can spin the globe and either just pick a place at random or what Giovanni told me is… “You know, I’ve observed that when people do this, when we have an opportunity to interact with Google Earth, they all either go to where they grew up or they’ll go to someplace that they have visited recently or they plan to visit. So, I went to a place that is very special to me and maybe it doesn’t fit into either one of those categories neatly, but it’s my daughter’s University… her school… and I should say that this is also a different thing for me because my daughter goes to school in Frankfort, Germany… an institute that is connected to a Museum. So, I had only been to part of the physical facility… the museum itself… and it was a long time ago… and part of it was closer to the holiday. So, this is my opportunity to go there and explore what it looks like all over… and so, that was an emotional experience for me. It was a sensory experience… it was a social one… because we were talking the whole time… and he’s asking me questions and what kinds of exhibits do they have here… and what’s this part of it. So, that was wonderful. it really did give me a feel for alright, what is it actually like to be in this sort of environment?

I’m not a gamer. I don’t have that same background that many of our students have. So, it got me up to speed on that… and it did show me how just exploring something that is relatively simple can really acquire a whole new dimension in this kind of immersive environment. Now the postscript that I talked about in that blog post was what happened when I actually visited there earlier in the year. So, I had this very strange experience that human beings have never had before… which is from this… I don’t know whether to call it deja vu or what… of going to the settings and walking around the same environment and seeing the same lighting and all that sort of stuff that was there in that virtual reality environment… but this time, of course, with real human beings in it and the changes… the little subtle changes that take place over time, and so forth.

So, how does it translate into learning? What’s it going to do for our students? I just think that time is going to tell. It won’t take too long, but I think that these are things we need to know. But, sometimes just getting in and being able to explore something like this can really put you back in touch with the things you love about educational technology.

Rebecca: I think one of the things that I’m hearing in your voice is the excitement of experimenting and trying something… and that’s, I think, encouragement for faculty in general… is to just put yourself out there and try something out even if you don’t have something specific in mind with what you might do with it. Experiencing it might give you some insight later on. it might take some time to have an idea of what you might do with it, but having that experience, you understand it better… it could be really useful.

John: …and that’s something that could be experienced on a fairly low budget with just your smartphone and a pair of Google cardboard or something similar. Basically, it’s a seven to twelve dollar addition to your phone and you can have that experience… because there’s a lot of 3D videos and 3D images out there on Google Earth as well as on YouTube. So, you can experience other parts of the world and cultures before visiting… and I could see that being useful in quite a few disciplines.

Rebecca: So, we always wrap up with asking what are you going to do next?

Michelle: I continue to be really excited about getting the word out about cognitive principles and how we can flow those in to teaching face-to-face with technology… everything else in between. So, that’s what I continue to be excited about… leveraging cognitive principles with technology and with just rethinking our teaching techniques. I’m going to be speaking at the Magna Teaching with Technology Conference in October, and so I’m continuing to develop some of these themes… and I’m very excited to be able to do that. I’m right now also… we’re in the early stages of another really exciting project that has to do with what we will call neuromyth… So, that may be a term that you’ve turn across in some of your reading. It’s something that we touched on a few times, I think, in our conversation today… the misconceptions that people have about teaching and learning and how those can potentially impact the choices we make in our teaching. So, I’ve had the opportunity to collaborate with this amazing international group of researchers who’s headed up by Dr. Kristen Betts of Drexel University… and I won’t say too much more about it other than we have a very robust crop of survey responses that have come in from, not just instructors, but also instructional designers and administrators from around the world. So, we’re going to be breaking those survey results down and coming up with some results to roll out probably early in the academic year and we’ll be speaking about that at the Accelerate conference, most likely in November. That’s put out by the Online Learning Consortium. So, we’re right in the midst of that project and it’s going to be so interesting to see what has the progress been? What neuromyths are still out there and how can they be addressed by different professional development experiences. We’re continuing to work on the Persistence Scholars Program on academic persistence. So, we’ll be recruiting another cohort of willing faculty to take that on in the fall at Northern Arizona University. I am going to be continuing to collaborate and really work with and hear from John and his research group with respect to the metacognitive material that they’re flowing into foundational coursework and ways to get students up to speed with a lot of critical metacognitive knowledge. So, we’re going to work on that too… and I like to keep up my blog and work on shall we say longer writing project but we’ll have to stay tuned for that.

Rebecca: Sounds like you need to plan some sleep in there too.

[LAUGHTER]

John: Well, it’s wonderful talking to you, and you’ve given us a lot of great things to reflect on and to share with people.

Rebecca: Yeah. Thank you for being so generous with your time.

John: Thank you.

Michelle: Oh, thank you. Thanks so much. It’s a pleasure, an absolute pleasure. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

36. Peer instruction

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

33. The Marmots of Finance

In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, Alex Butler, a Professor of Finance at Rice University, joins us to discuss how rich imagery can be used to help students make connections and deepen their understanding.

Alex received the Rice University Presidential Mentoring Award and the George R. Brown Award for Superior Teaching in 2018. He is also the recipient of the Jones School’s Award for Scholarship Excellence in 2011 and 2012.

Show Notes

  • Duke Talent Identification Program (TIP)
  • Medina, J. (2011). Brain rules: 12 principles for surviving and thriving at work, home, and school. ReadHowYouWant.com.
  • Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
  • Alex’s web site at Rice

Transcript

Rebecca: In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, we’ll discuss how rich imagery can be used to help students make connections and deepen their understanding.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Alex Butler, a professor of finance at Rice University. Alex received the Jones School’s Award for Scholarship Excellence in 2011 and 2012. Alex teaches financial markets and corporate finance in the undergraduate MBA, MBA, and PhD programs. Sometime in the latter part of the last century, Alex and I spent three summers teaching introductory economics to highly gifted middle school and high school students at the Talent Identification Program at Duke University.
Welcome, Alex.

Alex: Oh, thank you. Thank you for having me, John.

Rebecca: Today’s teas are:

Alex: I am not drinking tea…. just water at this point.

Rebecca: Another one… epidemic.

John: My tea is ginger peach white tea.

Rebecca: I’ve Prince of Wales today.

John: Could you tell us a little bit about the courses that you normally teach?

Alex: Sure. I have, over the years, taught almost every course that we have… either here or at other schools. Right now what I teach (and I’m glad to finally have settled into a group of classes that are my classes year in and year out) and those classes are the undergraduate business finance class and PhD courses – one in corporate finance, and one that’s a topics class on causal inference. I really enjoy teaching the undergrad business finance class in particular because I was an undergraduate student here at Rice many, many years ago, and so it’s fun to be back and be on the other side of the podium teaching the students. One of the things that I really like about teaching the undergraduates here is that they are able to appreciate my dorky sense of humor.

Rebecca: …which is indeed a very important thing.

Alex: Agreed. For years, I tried to suppress my dorky sense of humor in the classroom… ad tried to teach the course sort of straight up and dry, and it made everybody miserable… the students…. me… and so, eventually over time, as I grew more and more confident teaching, I started incorporating more and more jokes here and there… and then I allowed them to become more elaborate, and then before you know it, I’ve developed a full-blown dad sense of humor and full-on dork mode.

John: That’s actually why we invited you here. We read a little bit about that in a teaching award you just received. What do students expect the course to be about when they take an introductory finance course?

Alex: At Rice, the students have a good grapevine of information about what courses are about and so, at this point, the students come in with pretty solid expectations that line up with what the class actually is… and whereas some students are hoping that I’m going to teach them how to become millionaires in the stock market, what the course mostly is about is that’s very hard to become a millionaire investing in the stock market and how to make decisions in a corporate setting that will maximize firm value.

Rebecca: So you mentioned that some students come in with this misperception of becoming a millionaire. What are some of the strategies that you use to dispel that myth?

Alex: I should say this class is one that is a fairly standard course that’s taught in lots of different business schools all over the country, all over the world. One way that I do it differently is in the order of material that I cover… and so the very first week of class, I talk about market efficiency. That’s the notion that it’s very difficult to earn abnormal returns in the stock market. In other words, you can’t beat the stock market very easily, unless it just happens to be by luck… and so I come in the first week talking about the reasons why it’s very difficult to beat the stock market… and the reason why is because there are literally tens of thousands of people who have more money and more resources and who are faster and are doing this as a full-time job who are also trying to find the stocks that are mispriced… and so unless you are the investor who is faster, and smarter, and has more money to throw at the trading strategy that you think of, it’s very very difficult to beat the other 10,000 people… and so I spend the first week of class introducing this concept and then providing copious amounts of evidence… research that highlights how difficult it is for lay people to earn abnormal returns in the stock market. That sets the setting for the rest of the course, which is this notion of how competition affects prices and how that feeds through to other applications and the corporate domain as opposed to the financial markets domain.

Rebecca: What made you switch the order?

Alex: What I realized after teaching this class for a long time is that most of corporate finance, most of business finance, is about discounting cash flows back to the present at some appropriate discount rate, and I found I was having a hard time getting the students to understand the notion of what interest rate, what discount rate, should be used to make these cash flows that are spread through time to get a present value equivalent. I figured if I started with some aspect of that, where that rate comes from, where those prices come from. but that would make the rest of the course easier for them to understand as we go through. So, that’s why. Most people wait until after they’ve introduced things like: “oh, portfolio theory” and “capital asset pricing model” and other asset pricing concepts before they talk about market efficiency… and so I just sort of turned that around backwards… and I open the course with that. So, point number one is pedagogical, and point number two is that the lectures that I do on market efficiency are really fun, and so I really like starting the course off with something that’s really fun. So, we can talk about stories of insider trading, and we can talk about stories of surprise announcements and how that affects stock returns, and we can talk about “oh, so you think you’re going to beat the market, well let me explain to you how hard it is and the reasons why …” also were very fun. So, we can spend the first week talking about fun stuff. It’s a giant bait and switch.

[LAUGHTER]

I lure them in with fun stuff and then beat them to death for the next fourteen weeks with discounted cash flow analysis.

Rebecca: It also sounds like it’s a good way to motivate students. Not only is it fun, but it gets students motivated and interested and they buy into the class, which I wouldn’t discount that. I think that’s an important task.

Alex: Oh, absolutely.

John: …discounting in a different sense, but…

[LAUGHTER]

Rebecca: Yeah, sorry… I’m a designer

John: …but it also starts a class by dispelling that myth that they’re going to learn tools that will allow them to become really wealthy in financial investments and so forth. So, you’re setting it up by getting rid of that myth and they’re ready to start actually learning without having that at the background.

Alex: Correct.

Rebecca: I really like the idea of just meeting students where they’re at. If this is where some students are coming with, and maybe they’re super motivated in some ways but not in others, that you just tackle both of those in one week.

Alex: That’s right.

Rebecca: Cool.
You recently won a 2018 George R. Brown Award for superior teaching, and the Georgia R. Brown Awards are based on an interesting selection process. Can you tell us a little bit about that?

Alex: Yeah, so what the university does is they solicit feedback from recent alumni, people who graduated, I think, it’s two and five years ago. Now, they have graduated and they’re looking back what teachers would they want to see when these awards… and so, it’s really a neat honor, because once the students are two, three, four. five years out, they’re not responding to the short-term incentives that some professors used to gain teaching evaluations and things like that, they’re looking back and they’re actually remembering what the course was, what the professor was like, and whether it was meaningful to them. It’s really been just a phenomenal honor, and one that I honestly thought that I would never win. So, it’s been really special to have gotten that award.

John: It’s a really interesting idea to have an award given in that way, because it focuses on that long-term learning, that you’ve taught them something that’s going to benefit them later rather than, as you said, something that people do short term. I know some faculty, when we’re doing course evaluations, will give out cookies or other things just to boost their short-term course evaluations. But, that’s not going to pay off very much two to five years after graduation.

Alex: That’s correct. I went through the cookies and brownies phase myself years ago and decided that that just wasn’t who I wanted to be, so…. Now, we’re on to just just the pedagogy at this point.

Rebecca: I think there’s a couple of interesting things that I see about those awards, too, and one is that is more meaningful probably to win it because you can feel the impact but two it meshes really well with what we know about evidence-based practices in that students generally don’t like them while they’re happening, but they have longer-term effects… and that students tend to learn that material and transfer that material later on.

Alex: That’s absolutely correct, and I’m a big subscriber to that basic view that students often don’t like things that make them uncomfortable, and learning new and difficult things is uncomfortable. I think one could make students happier in the short run by giving them lots of little assignments where they feel like they’re making progress every day but they’re not actually being challenged, they’re not actually being pushed, and so instead holding them to very high standards for demonstrating their mastery of material while that is very uncomfortable for some of them, it is that that makes them better students and better scholars of that topic.

John: It reminds me of Bjork and Bjork, in their writings on “desirable difficulties.” that the most learning occurs when students are faced with feasible challenges… that if things are too easy, they get bored, and they may be happy with the course if they don’t have to struggle much… but they learn the most when they’re struggling but they see it’s possible.

We’ve heard that this award, though, based on some of the write-ups we’ve seen at your institution, may somehow be tied to marmots, wolves, and The Princess Bride. What do they all have to do with finance?

Alex: Several years ago, I was reading some books I thought would be…. some I thought it’d be helpful for my teaching, some that I just thought would be helpful for me, and one of the books was a book called Brain Rules by John Medina, and it’s basically a book that tries to take cognitive science, brain science, down to a level that lay people can understand and gives several rules of thumb of how the brain works and why the brain works the way it does… and as I’m reading this, reading it mostly for my own consumption so that I can be a better researcher and more thoughtful person, smarter, that sort of thing, I realized “Gosh, a lot of the rules here applied very directly to teaching, at least in the lecture format that I use in most of my courses…” and so one of the things that really stood out to me is how people learn better, remember better, I guess I should say, when they see images images that relate to whatever the topic at hand is. So, text maybe a PowerPoint slide with text, and you remember X percent but if you see an image, you remember much more of that material later on… and so this gave me just a license to, all of a sudden, start having fun on a completely new dimension. Reading this book and sort of embracing the notion that I could maybe help students remember the material better simply by infusing my lecture slides with some relevant images, was just eye opening for me… because now I could take my completely dry, boring slides with words and numbers and equations and now I can have fun with them…. and have this entire new dimension, a degree of freedom, to play around with what the slides are gonna look like and how the students are going to experience them. So, that’s sort of the extensive margin. The first part is: “Hey, I need images…” so the intrinsic margin is “what kind of images do I need?” “what will work best?” and Brain Rules comes to the rescue there again… and it says people respond to images that are faces, that are things that are scary, things that they can eat, things that might want to eat them, and things that they might want to mate with. Well, that last one’s kind of out for most of my lecture slides… [LAUGHTER] I can’t really incorporate that very directly. But it got me thinking “okay, what’s scary?” So, I started looking around for images of things that are scary to people… and I found this great image of this really, just terrifying, snarling wolf. I’m like “Ah, I’m gonna use that to get people’s attention…” But, as I started thinking about it, I wanted to lead into the wolf a little bit… and so the main prey of wolves in North America are marmots… yellow-bellied marmots… and so I found this great image of a little cute yellow-bellied marmot sitting on a rock somewhere in the mountain somewhere… and now when I come in to teach what could be the absolute driest lecture of the entire course which is time value of money and understanding how to discount cash flows (it’s the tool that everything else builds on, so it’s incredibly important to get it right, but it’s also potentially incredibly technical and boring)… and so I start that lecture not with an equation not with numbers but with a giant image of a marmot filling the entire computer’s projection screen…. and I just leave it up there… and the class is all looking at it… and I look back at them… and I pick someone at random. I cold call… and I say: “So, Charlotte, what do you think?”

“What do you mean what do I think?”

“What is it?”

… and we go through a series of guesses, and the guesses range from just ludicrous things: “it’s a gopher.”

“No, no, clearly it’s not a gopher. Gophers are a lowland creature. This is obviously in the mountains.”

“It’s an otter.”

“No, no, no. Otters prefer marine habitats, and this is clearly not there.”

… and go through this for a while until usually somebody recognizes it as a marmot. I say “Yes, very good. Alright..”

…and so then on to the next slide and the next slide is this picture of George Soros with no caption, no explanation, but again filling the entire screen, here’s this giant picture of George Soros… and so I go back to the first person: “Charlotte what do you think about this one? You didn’t get the marmot, how about this one?” …and so invariably somebody will eventually guess it’s a hedge fund manager.

I say: “Yes, very good. That’s George Soros, a famous hedge fund manager.” then the next slide is the wolf, the snarling wolf, and so at this point the captions on the slides read “This is a yellow-bellied marmot,” “This is a hedge fund manager” … and then it’s obvious what the wolf is. People get that right away. So, Charlotte gets to redeem herself at that point.

[LAUGHTER]

“This is a wolf, one of the main predators of the yellow-bellied marmot,” and then the next slide is another picture of a marmot but this one looking somewhat quizzical and the caption here is: “Why do wolves eat marmots, but not hedge fund managers?” …and the answer that I propose is because hedge fund managers understand the time value of money, but marmots do not. Now, that obviously doesn’t actually follow, but the correlation is there… that part is true… and so I just sort of lean into that and the ridiculousness behind that statement and from there on the marmot becomes our time value of money mascot… and so every time throughout the course I introduced a new application of time value of money then I bring in another image of a marmot to sort of tie all that together. iIt’s hard to see that the tie that binds everything together is time value of money and so the marmot, the mascot, is the visual cue, that “oh, this isn’t special, this isn’t different, this is just another application of the same technique we’ve been doing over and over again.

John: That notion of using visual cues goes back to the Greeks who use it to remember long stories before there was much printed word, and one of the arguments is that it’s because visual imagery developed much earlier than the use of language and the things you described in terms of things that might eat you may be tied back to our evolutionary adaptation, and so we’re tying into things that evolve fairly early in the evolution of human beings.

Rebecca: …and it also is just that emotional response.

John: Right, when you trigger an emotional response and fear is a major one…

Rebecca: Yeah. mm-hmm

John: Do the wolf’s chase the Princess Bride? I don’t remember that…

Alex: Well no…. So, the Princess Bride lecture is where I talk about decision rules using discounted cash flow techniques…. and so this sort of standard playbook for any finance professor anywhere as you talk about net present value and internal rate of return and payback period and maybe a few other rules as well, you just sort of go through the… whatever they have in the book in the order in the book, and you talk about each one in turn. But, it’s a lot more fun if you can personify those.. and so that’s what I use the characters from Princess Bride to do… and so the main character, which is net present value, is personified as Wesley / (there’s a spoiler alert here – so you haven’t seen the movie Wesley is the Dread Pirate Roberts). I always show my kids my lecture slides and when I showed them that one my older daughter was mortified that I would ruin the Princess Bride movie for students by revealing that Wesley and the Dread Pirate Roberts are one and the same… and so from there, I personify the different rules based on the different characters in Princess Bride… and so internal rate of return which is sort of flashy and useful but has some flaws as Inigo Montoya; the payback period which is sort of a blunt instrument is Fezzak the Giant, and so forth… modified internal rate of return is the Cliffs of Insanity.

[LAUGHTER]

Rebecca: So, the important thing to ask then is do you have the spoiler alert at the beginning of your lecture.
AEX: I do now.

[LAUGHTER]

John: How old was your daughter at the time when she reacted to that?

Alex: About ten years old.

John: Okay. So, by the time students are in college, they’ve probably either seen it or they may be less likely to, so it probably doesn’t do quite as much damage.

Alex: I hope so, and in fact I even now encourage them to watch the movie before that lecture without really telling them why. So some of them do. Some of them ask around and figure out why, but that makes it more fun.

Rebecca: That was gonna be my follow-up… is that homework?

Alex: It’s the best homework they’ll ever have.

John: So, how have students responded to this? Do they remember this later?

Alex: Yes and no. for the Princess Bride lecture I believe that the students sort of uniformly enjoy that…. but I don’t have a good sense of how much that actually impacts the depth of their learning. For the marmots, it’s a completely mixed bag… some of the students really love it, and I really do lean hard on the fact that this is the mascot… this is the thing that ties it all together… and this is the visual cue so you will now recognize that when we do bond pricing “oh, here’s a marmot, that means it’s just time value of money…” “oh, when we do net present value, here’s a marmot showing up alongside our Princess Bride character it’s just time value of money” and so forth. Many of the students really enjoy that and grab on to it and some students, they’re not having any of it. They want boring… they want dry… and I’m afraid they’ve come to the wrong place. Been there and done that. I don’t do that anymore.

John: But that use of imagery is really common. People who work on developing memory… the memory palace type things where you tie specific concepts and bundles of concepts with chunks of item to key images, helps people remember things long term. I can see how it would be really effective.

Rebecca: I also could see that one of the things that students often struggle with when information is new to them is making those connections. So providing that visual cue like “here’s time to make a connection…” it’s actually really helpful, because those are the kinds of things that might seem really abstract and very separate if you’re not making an explicit. So, I like that you don’t even have to say explicitly like “this is the thing” you have by putting the image up there, and you’re prompting the students to predict what that connection is or challenging them to think of it on their own before you reveal what that connection is.

Alex: That’s right, and so what started off as just a fun way to get attention: “Hey, class is starting let’s all key in… Here’s a scary wolf.” So, now we’re all paying attention to the scary wolf because that’s how our brains are hard-wired. It’s now grown into this entire thing where throughout the entire class is a continuing callback.

Rebecca: It sounds to me like you must have some pressure every semester to have to have something new that you introduce into some sort of lecture so that there’s some anticipation.

Alex: I’ll be the first one to say that I shamelessly recycle all of my jokes…

[LAUGHTER]

… and so I’m always sort of terrified when I have a student who started the class last year but had to drop at the 6 week mark or the 8 week mark because whatever was going on their life, and then they’re back the next year. Like “oh man, you’re gonna get the exact same jokes with the exact same timing, the exact same patter.” It mostly seems to work fine, and I do continually try to incorporate new things. One of the nice things about teaching finance, although some of it is very static… the basic concept of time value of money is going to be there for basically forever and I won’t need to change the actual examples in those slides really ever… but a lot of the other material changes very rapidly. So, the notion of what does market efficiency mean? who can beat the market and win? what is the evidence? But for topics like market efficiency, for topics like financial markets, for topics like investment banking, those areas transform rapidly… and so I’m continually changing those lectures year after year because one of the big topics that I cover in my course that is not really traditional for an undergrad business finance class is how firms raise external capital. The reason why is because it’s one of my main research interests, and so I have lots of ideas of what I want the students to know, and lots of research that I can tie into the lectures that I give. For the past five or ten years financial markets, the regulation of financial markets, firms’ ability to raise external capital, has changed tremendously as regulation has changed and so I’m continually revamping that portion of the course basically every time that I’m breaking it out.

Rebecca: One thing that I can’t help but think is that you and John have some similar backgrounds in terms of content, so the time value of money that you keep referencing probably makes sense to you but I feel like our listeners might not know exactly what that is so maybe we should just take a minute and give a quick cap of that so people know.

Alex: Yeah, absolutely. So, if you have $100 and you invest it earning an interest rate of 5% per year, in a year how much money will you have?

Rebecca: One hundred and five?

Alex: One hundred and five dollars.

[LAUGHTER]

Rebecca: It’s not a trick question, right?

Alex: No, No, it’s not…

[LAUGHTER]
… and so, as you invest money it earns some rate of return and so then money in the future you have more, because it earns some positive rate of return, and conversely, would you rather have $100 today or $100 in a year? Well, the answer is $100 today, because you could invest that for that year and have the hundred dollars plus some additional return. So, you’d have a hundred and five dollars one year from now instead of the: “oh, I’m going to give you a hundred dollars a year from now” and that’s all the time value of money is.. and then it’s application of that over multiple cash flows and multiple periods where people start to lose track that it’s just math.

John: That’s the fun part.

Rebecca: Yeah. Yeah.

John: I also ask my students have they ever burned their mouth on a slice of pizza, and why don’t they just wait? ..and it’s one way of introducing the notion that we prefer things now to later. It’s a fairly important concept in economics and finance and it’s at the basis of finance.

Alex: Oh, absolutely… and I like the way of framing it as how patient you are. Are you willing to wait or are you impatient? …and so the way that I sometimes describe that in class is if you have a low discount rate that means that the future is worth about the same as the present and so you are patient. If you have a very high discount rate well then the future and the present are very different, so high discount rates → impatient, low discount rates → patient.

Rebecca: You realize that the irony of this whole thing is that I have these conversations with my husband all the time because he’s in finance, right? Yeah….

[LAUGHTER]

John: Ok, well, does that help explain it?

Rebecca: Yeah, yeah. I’ve had it explained to me many times.

[LAUGHTER]

I figured our listeners would need it.

Alex: Well, since we’re talking about the things that our students do, I’ll just share one example that I use in class that I continue to use even though it completely does not work with undergraduate students. It’s when I’m trying to introduce the concept of sunk costs… and so a sunk cost is some amount that you have paid… a cash flow that has happened in the past perhaps… and so once that is paid, you can’t get it back …and so it’s like the notion of “should he throw good money after bad” is another way of phrasing it… and so, what I used to say is, “Well, you go to a movie, you pay your fee to go in, and you decide it’s a terrible movie, should you keep watching to get your money’s worth or should you leave? and then a student pointed out to me one year that well, actually, if you go and you complain you can get your money back from the management.

[LAUGHTER]

Okay, different example. This is the one that never works on the undergrads. You go to an all-you-can-eat buffet and you have a choice of do you buy one plate for this amount or do you pay a little more to get the all-you-can-eat buffet and you decide to go for the all-you-can-eat. You pay the $10 for the all-you-can-eat instead of the $7 for the single plate… and you go and you fill up your plate and you eat… but you decide you’re full after just your first plate… but you’ve already paid that all-you-can-eat price. Should you go back for more? …and the intellectual scholarly answer is “Well, no, because the marginal benefit of eating more food is negative at this point because you’re full. Even though the marginal cost is zero.” But, for the students, the undergraduates, the marginal benefit of eating more food is always positive.

[LAUGHTER]

They view it as a sport. Dinner’s not over until they kick me out.

Rebecca: I think that’s really important to consider your audience and what works. So, I think that’s a really good demonstration of knowing your audience and why something might not work for a particular audience.

Alex: Yes, nonetheless, I still use that example… but I tell them ahead of time “I know this is not going to make sense to you. You’re going to push back on me. Nonetheless, I want to talk it through with you so that we can think about who has marginal benefits and marginal costs of what amount.”

John: Just last semester I used exactly the same thing of asking them how many of them would leave a movie theater if it was clear that they weren’t enjoying the movie and this time I had some people argue “I do that all the time.” So, yeah…

Rebecca: Movie theaters? Don’t you just have Netflix?

[LAUGHTER]

John: Okay, that’s actually a relevant point. In terms of the Princess Bride, we’re moving into a generation where many of your students might not have seen that. Do you have any other movie references that you might substitute in the future?

Alex: You know, I haven’t found the right one yet and, honestly, even if I found a good substitute, I’d really want to keep the Princess Bride, just to give them the incentive to watch that movie.

[LAUGHTER]

It is such a classic and such a gem. I really would like for them to all see that movie.

Rebecca: It’s an investment you’ve made.

[LAUGHTER]

Alex: That’s right.

Rebecca: How have faculty responded to the methods that you’ve been using?

Alex: Faculty are not surprised that I do the things that I do.

[LAUGHTER]

John: They’ve known you for a while.

Alex: They’ve known me for a while and I really lean into the corny pun bad jokes big time in class…. and I really I’m quite shameless about it.

John: That’s not new… I seem to remember that back at Duke when you were still in college.

Alex: …and it’s just a question of do I try to suppress that innate desire or not?

[LAUGHTER]

…and I’m at the stage in my career where “nope, not anymore.” You’re just gonna let it all out. My colleagues are not really surprised, but interestingly, one of my colleagues has adapted her lectures quite a bit to embracing this notion of adding images into the slide decks… and that’s my wife. She teaches business law and regulation of business. She was an attorney by training and I described her as a born-again economist because when she started teaching regulation of business she had to teach herself basically all of public choice economics. So, she got a couple of high-level textbooks and worked through them all but she and I regularly discuss teaching techniques and so she now has gone down the rabbit hole of finding that perfect image to highlight the point that you want to make… to have that really stretched metaphor that you can then call back throughout your course. So, it’s been a lot of fun to have her as a sounding board to go back and forth with.

Rebecca: I appreciate that you’ve brought design into the process.

Alex: Yeah.

John: Excellent. It’s something we all should probably do more of and think about more and certainly much more effective than those PowerPoint bulleted lists that are so common.

Alex: Yeah, a little bit of both actually goes a long way.

Rebecca: You’ve won some awards for your research. How do you maintain a balance between teaching and research?

Alex: It’s difficult. There are some ways in which research can feed directly into the teaching… and so my research that touches on market efficiency, some of that can come into the classroom. My research that touches on how firms raise external capital, some of that can come into the classroom. Now when I teach my PhD courses, those are heavily flavored by my research interests and preferences, but when I teach the undergraduate core finance class there’s not a whole lot that can flow back from that teaching into the research. So, that’s one of the downsides of teaching that class as opposed to perhaps a class that is more specialized or an elective or something that’s a little bit further downstream from the core class.

Rebecca: But at the same time, if it’s something that you teach routinely then the prep isn’t as difficult.

Alex: Absolutely. That’s true. It is hard to overstate how useful it is to teach a class a second time, or a third, or a fourth, or twentieth. Nonetheless, every single year, even though it’s the same class and mostly the same topics, I go through every slide, every lecture from the beginning every year.

Rebecca: But, I remember teaching a bigger selection of classes and one of the things that I’ve liked about my position at Oswego is that that suite of classes has gotten smaller… and then there’s a little less I’ll keep on top of to make sure that you have all that fresh information and what-have-you for classes.

Alex: Absolutely, and I’ve done the same thing. I’ve taught a variety of courses over the years and it’s been nice… as you describe it that suite of classes narrows, so that you have the same core group of classes that you’re teaching over and over again and you can start to specialize. You can really invest the time to get over the fixed costs of finding all those right images for the slides… to going through and taking time to invest in the design aspects of the lectures. That if you had four different courses every term, it would be incredibly difficult to find the time to do that effectively.

John: While you’re working with PhD students, you must do quite a bit of mentoring of them. How do you see the role of a faculty member as a mentor for graduate and undergraduate students?

Alex: Faculty vary widely on their views of how much mentoring PhD students should have… and so you have one model where it’s sink or swim… the PhD students are some of the smartest people in the world… they’re good students… they’ll figure stuff out… just point them in the right direction and let them go, and they’ll get there. That’s not the view that I subscribe to. Because I think we frequently overestimate just how much the doctoral students know, particularly about how the profession works. There’s no book for that. They can’t just go down to the library and find a textbook on how to be a good assistant professor or… there are books on how to write a dissertation but that only gets you so far, because it really needs to be very field specific. So, I tend to go very much the other way, which is a lot of sort of high-touch mentoring. I write co-authored papers with many PhD students, one of whom is now your colleague there at Oswego in the Finance Department in the School of Business, and it’s enjoyable for me. It is a good learning experience for the students and I think it helps them to learn how the profession works much more efficiently, because when it comes time to write a paper and they might put together some tables and say I want to structure the introduction this way. Oh, no, no, no, no, we can’t do that… because if you do that then it’ll make people be concerned about this issue here. So, instead, we need to twist it around this way and start with this… start with the big picture, not what your paper does but what your paper’s about. That sort of thing. That’s hard to learn on your own.
PhD students are PhD students because they are extraordinarily good students and they’re really good at learning. Though, that’s not the job for academics. The job is not the learning, the job is creating knowledge… and the transition from being a consumer to a producer of knowledge is scary, and it is the road that has very few signs or roadmaps to help them get down. It’s a transition of going from a consumer to a producer of knowledge… is very profound for a lot of people.

Rebecca: What’s interesting about what you’re hearing is you know my field the terminal degrees in an MFA a Master of Fine Arts, and the undergraduate degrees are really professional degrees. But, it’s the creators of cultural content ,and so that struggle happens at the undergraduate level too, of going from being that consumer of culture to a producer of culture. It’s really not that much different… just what they’re creating is a bit different.

Alex: That’s right.

John: I remember when I was working on my PhD, one time, where up until that point I had been meeting with my advisor every month to talk about my research, and at first I was just asking him questions… and then he was asking me questions and I realized suddenly that I knew more about the topic than he did… and that’s I think that sort of transition that’s sometimes difficult… because when you’re working on your research you’re mostly going out and finding all these earlier studies and so forth but you get to some point where suddenly you become the expert in the field and that’s a tough transition to make. It’s scary, as you said.

Alex: It is. It’s quite the watershed moment when you realize… when you are presenting your research to a room of 30 presumably learned scholars that maybe collectively they know more about the topic than you, but you know more than any individual person in that room… and becoming that expert and then owning it, so that you can write confidently is, I agree, a very tough transition

Rebecca: I like the emphasis on the owning it part. I think that’s key.

Alex: Yeah, imposter syndrome is real.
[LAUGHTER]

John: Everywhere.

Rebecca: So, we usually wrap up by asking “what are you gonna do next?”

Alex: Well, that’s a good question. I really wanna keep doing what I’m doing, making my class better year on year, teaching PhD students and training them year on year and working on research, hopefully research that people will actually find interesting and useful.
I have a sabbatical coming up.

Rebecca: When’s your sabbatical? What’s the countdown?

Alex: Well, the sabbatical is a year from now and in between then, one of my colleagues with whom I co-teach a doctoral seminar, he’s on his sabbatical. So that PhD course, that normally I teach half of, I’m now teaching the entirety of next fall. So, I think the first thing I’m gonna do is prep the rest of that class.

John: Excellent.

Rebecca: Deadlines make a difference.

Alex: Yes, indeed.

John: That procrastination thing… and that time preference…

Well, thank you!

Rebecca: Thank you so much.

Alex: Thanks.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.