42. Flipping the classroom

Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode, Dr. Dominick Casadonte, a Chemistry Professor at Texas Tech University, joins us to discuss research and best practices related to flipped classrooms.

Show Notes

  • Camtasia
  • Mediasite
  • Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement. International Society for Technology in Education.
  • Sams, A., & Bergmann, J. (2013). Flip your students’ learning. Educational leadership, 70(6), 16-20.
  • Bowen, J. A. (2011). Rethinking technology outside the classroom. Journal of Music History Pedagogy, 2(1), 43-59.
  • Bowen, J. A. (2014). The Teaching Naked Cycle: Technology Is a Tool, but Psychology Is the New Pedagogy. Liberal Education, 100(2), n2.
  • Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve student learning (ed.). Jossey-Bass
  • Belford, R. E., Stoltzfus, M., & Houseknecht, J. B. (2015). ConfChem Conference on Flipped Classroom: Spring 2014 ConfChem Virtual Poster Session. Journal of Chemical Education, 92(9), 1582-1583.
  • Stoltzfus, J. R., & Libarkin, J. (2016). Does the room matter? Active learning in traditional and enhanced lecture spaces. CBE—Life Sciences Education, 15(4), ar68.
  • Stoltzfus, Matthew (2016). Engaging Students in the Flipped Classroom (video)
  • Coats, H. J. (2016). A study on the effect of lecture length in the flipped classroom (Doctoral dissertation).
  • Casadonte, D. J. (2016). “The Effectiveness of Course Flipping in General Chemistry – Does It Work?” ACS Symposium Series, “The Flipped Classroom”, December 2016 (Book Chapter) The Flipped Classroom Volume 2: Results from Practice, Chapter 2, pp 19–37, Chapter DOI: 10.1021/bk-2016-1228.ch002, ACS Symposium Series, Vol. 1228, ISBN13:9780841231627eISBN:9780841231610,
  • POGIL.org

Transcript

Rebecca: Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode we discuss research and best practices.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is dr. Dominick Casadonte, a chemistry professor at Texas Tech University. Dr. Casadonte is recognized as a global leader in flipped learning by the Flipped Learning Global Initiative.
Welcome.

Dominick: Thank you very much.

Rebecca: Today our teas are:

Dominick: Well, my usual afternoon beverage is an iced green tea with three pumps of raspberry, but since I’m getting over a cold, today I’m drinking a hot green tea with vanilla, lavender, and honey.

Rebecca: That sounds really nice. Finally, a tea drinker.

[LAUGHTER]

John: I’m drinking Irish Breakfast tea.

Rebecca: …and I’m drinking chai today.

John: You’re recognized as a an expert on flipping the classroom, and you’ve been doing it for a while. Could you tell us a little bit about what a flipped classroom is?

Dominick: Sure. The usual paradigm, if you think about it with regard to teaching, is lecture, homework, lecture, homework, lecture, homework, tests, lecture, homework, lecture, homework, and so on… and the problem with that format is that it’s not conducive to deep learning… with the exception of maybe very bright students or very bright asynchronous learners. Trying to keep up with a lecturer is often difficult. A student, for example, may not have gotten down everything that they wanted to write down… they may not have gotten it down correctly… or it might be fragmented… or perhaps they just didn’t understand well enough to write it down in a meaningful way, or a way that’s meaningful for them. Then they try to go home and they try to do homework, but since they didn’t really understand the lecture in the first place, the homework is again difficult and they might only be able to work part of it. So then, the very next day, there’s a new lecture and a new topic. The normal sort of didactic approach to education often leads to what I’ll call fragmented learning, in the sense that there isn’t enough time in class to practice many of the active learning strategies that work so well because faculty are so concerned with getting the lecture done. In the flipped model, on the other hand, we flip the homework-lecture-homework paradigm so that the students’ homework is to watch a lecture online. Now it can either be PowerPoint or a video and different people use different techniques. It can be done on Camtasia, it can be done through Mediasite, it can be done through a variety of different platforms. It could be done on YouTube for example. So, they’ll watch usually a video or a lecture online before they come to class, and then they may or may not do online homework prior to coming to class as a part of the pre-class experience. Then, once in class, time can be spent, for example, checking the students knowledge… clearing up muddy points or any misconceptions they might have… working advanced problems. But now, in this particular case, class becomes more of a discussion and you can use a lot of the active learning strategies that really engage the students in learning the material. By flipping the classroom, we freed up time to really engage and interact with our students and it helps with their learning in significant ways which I can talk about later.

John: So, in a traditional class, you tell students what the content is in class, and they come in with very different backgrounds and some of them are able to pick it up, others get lost along the way, and then they’re sent out to do homework… where students seem to have the most difficulty on homework or tests… and we’re pretty much leaving them alone to do that. But, here in the classroom, you’re able to work with them and help them through some of the issues.

Dominick: Right… and so, for example, in the classroom with homework, now they have a mentor who can help them work through the more difficult problems to help guide them along. There are models of the flipped classroom where there are peer mentors in the class who can help and walk around and mill around as well. So, there are opportunities for group work. There are opportunities for group discussion. You can engage the topics in a much deeper way than you can by just simply lecturing at students. So, it’s a very very non-passive way of teaching.

Rebecca: What prompted you to get into a flipped classroom model?

Dominick: Desperation, I would say. [LAUGHTER] I first started teaching an online class in the fall of 2007 as part of a multidisciplinary science master’s program of which I was a part here at Texas Tech… and getting everyone on the same platform at the same time, my thinking was that lecturing to them live online was a waste of time, basically. The question that motivated me was the same one that motivated John Bergmann and Aaron Sams around the same time at the high school level up in Colorado. Namely, what could I do to use my classroom time more effectively? What could I do to use my online time more effectively? …and I thought, “Well, let’s take the lecture out of the classroom.” So, I pre-recorded my lectures. I had the students watch the lectures before we got together in the online environment, and then we spent the majority of our time having discussion and working problems…. and these were teachers at the time and they were very very enthusiastic about the ability to actually discuss the material that they were trying to learn… and so then I thought “Well, hmm, if that works really well on an online format why don’t I try it face to face?” …and so then, starting in 2008, I adopted this to my general chemistry class. The following summer, a high school teacher that I had as part of a workshop said “Oh, I see you’re flipping your class” and I said “Flipping, what’s that?” ‘Cause at the time, there were a lot of different terms for flipping. It was called: time-shifted instruction, reverse instruction, blended instruction, all sorts of things. Flipping is a term that’s really sort of stuck.

So, I started out of a sense again of “How can I engage my students more effectively in the classroom?” and once I realized that it worked swimmingly in the online environment and then I said “Well, okay, will this work in the face-to-face?” …and it worked even better there. By that point, there are pockets of people around the country who are doing this, and they used all sorts of interesting terms. Jose Bowen at SMU, who’s an art professor, used the term “naked teaching” because it can be unsettling when you walk into a classroom and not have the comfort of your lecture notes to be able to project or read to the students. So, very often I will walk into the classroom and say “What are we going to talk about today? or “What would you like to talk about today?” You can’t do that unless you feel comfortable about the subject material and you have some expertise, but it’s a great way, in the long run, to really, really impact students.

Rebecca: You mentioned both doing this in an online environment and also in the classroom. Is there a difference between your experience in both?

Dominick: Yes, it’s so much easier face-to-face. To be able to walk around to gauge what’s going on in the classroom, their level of understanding, and how learning is happening. The power of direct peer-to-peer contact should never be underestimated, I think. Now, in the online format, my focus was more on the development of a learning community working on a particular topic, rather than in peer-to-peer mentoring and things like that. So, they’re very different approaches, but you do develop a sense of community in both, I think.

John: Now, with your online classes are they synchronous or asynchronous?

Dominick: My online classes are synchronous, in the sense that everybody meets in one place at one time. We have various software programs that allow us all to be in the same place. Yeah, so they’re all together at one point, and that’s kind of interesting because I have people from all over the country who are taking these classes. They’ve never actually physically met, at least for the first class that I teach. It’s interesting trying to, in the discussions, get a sense of the personalities of the people who were providing discussion.

John: Are they participating in video formats or is it just audio or text?

Dominick: I’ve done both. I’ve done video and audio and depending on the bandwidth and the number of students in the class both can work well.

John: How large are your classes?

Dominick: Well, my online class had 24 students in it. So, a fairly large, I think, from an online perspective class. It wasn’t one of these very very large classes like you might see at MIT, for example, but 24 is a good class in terms of bandwidth… trying to get everybody in the same room at the same time… and my face-to-face classes… they’ve ranged anywhere from a low of 25 to a high of 150.

Rebecca: You mentioned two different techniques that you use both in online versus in person. So, online you mentioned community formation and then in person you mentioned peer to peer. Can you expand upon each of those?

Dominick: Yeah, my first adage for teaching is know your audience, and in the face-to-face environment I was working largely with in-service teachers who were trying to develop a more significant content knowledge of chemistry, and so there the idea was the development of a supportive community where these teachers could bring their ideas to the table in terms of not only the content that I was teaching them, but also how they could then apply that content in their own classroom settings… and in that regard they were able to help teach each other techniques that they could use in the classrooms based on what they were learning in terms of the content. There really was a community focus, sharing knowledge as opposed to just gaining knowledge. In the face-to-face classroom, it’s more a sense of the students trying to understand the content at a deep level. There I found that it is true that when you teach something you really hopefully really understand it and so peer-to-peer mentoring is much more effective there. So, we tend to work in groups… we tend to work with dyads… two people working next to each other… and then sometimes I will just have people go to the board… but once again it was in that context of community because I think it’s very important, if you’re going to do flipping, and do it well, that it’s an active, encouraging, engaging classroom experience. If my students go to the board, whether they get the problem right or wrong, the class gets into a habit very very quickly of applauding the student for their attempt… whether or not it’s right or wrong… and then we debrief. We talk about what works… what doesn’t work… So, it really is both peer to peer and community building there as well. But, the emphasis is more on the individual in the face-to-face classroom, I think.

Rebecca: It sounds to me a little bit like the choices that you’re making online and in-person aren’t necessarily because of the medium, but rather who’s in those particular classes. Am I hearing that correctly?

Dominick: That’s right, and one of the nice things about flipping is that it is such a rich environment in which to work. As I mentioned earlier, the pre-class videos can be video lectures, they can be audio lectures (if that’s appropriate), they can be PowerPoint presentations, they can be any number of things. In the in-class experience, it’s an active learning environment. so you tailor it to the people in the classroom. For example, if you’re trying to teach in a flipped environment of the class of 24, there you can do all sorts of things that promote individual learning in ways that it’s a little bit more difficult to do in a class of 300, for example. But, I have a good friend, Matt Stoltzfus at Ohio State, for example, who routinely flips like general chemistry class of 600 students, and he’s able to give them as close to a personal learning experience as one can, I think. The point that I’m trying to make is that the flipped environment is a very rich one and it allows you to tailor the learning experience to your class specifically.

John: Do you create your own videos or do you use ones created by other people?

Dominick: I actually create my own videos. I have my own recording studio in my office… I have a Mediasite setup…. I have a video camera… I have a document camera… and I have a wonderful microphone…. and so every couple of years or so I re-record all of my videos. Now, I do use a mixture of other formats. So, for example I do have post-video homework that the students have to do online before they show up to class… and that’s done using an online learning platform that we have here at Texas Tech through a national distributor. The advanced problems that we work in class come both from the textbook that we use and also from problems that I develop. So, it’s sort of a hybrid. But, the pre-lecture videos I actually produce…… and one of the things that studies have shown is that students develop a certain sense of identity with regard to the person teaching the class. Some people, when they’re starting to flip, might want to just use Khan Academy videos, for example, or things like that. But the studies show that the class wants to see folks on the videos (or your voiceovers if you’re using PowerPoint with voiceover)… they want to see the professor who’s teaching the class, because that’s their professor… and so they develop ownership, if you will. Plus if you’re using other media, for example a Khan Academy, they may not be teaching it exactly the same way that you want to teach it, and so then you have to either reteach or undo. There’s a quality control issue there. So, it’s just easier, if you’re gonna do flipping, to make your own videos… and there are so many different ways of doing that now that there’s really no excuse.

Rebecca: That had been my experience as well when I’ve done videos in my classes. The students really liked the quirkiness or knowing that it’s the same person that they had in their classroom and if you try to slip in something else occasionally they really didn’t like it.

Dominick: …and a lot of people have a perfectionist tendency and really want their videos to be really super perfect. Well, once again, studies have shown that that’s not really what students want. They want to see the foibles. They want to see you as you are. If they know that you’re going to say “um” or “uh” in the classroom then if you don’t say “um” or “uh” in your videos, then they’re gonna say “Is that a robot teaching the class that looks like my professor?” So, it’s okay to be human when you’re doing the pre-lecture videos. But, I think one of the things that often hangs up people when they’re starting to do flipping is this notion that it has to be perfect, and it really doesn’t.

Rebecca: I think they really appreciate when you make mistakes and things, too. I know that my students did when I do like a coding mistake, I’m like “Whoops, I made a mistake..” and go back and fix it and explain what I did and why it was wrong. We all make those kinds of slips and errors and things and we would do it live. So, it’s kind of nice to do it in videos, too.

John: It makes you seem more human by doing that.

Dominick: One of the nice things about them seeing you make mistakes is that it gives them permission to make mistakes. They don’t have to be perfect. I’ve had experience with students in classrooms where they come in and they’re intimidated… they’re shy… they’re just afraid that they can’t master the materials. So, seeing somebody make a mistake who is an expert gives them permission to make mistakes… and one of the things that that really does is it empowers them to learn, because at the end of the day when somebody’s trying to learn they’re going to make mistakes and I give my class permission to make mistakes. In fact, I tell them you have permission to make mistakes while you’re learning. After you’ve learned something, that’s a little different. If you’re an engineer, for example, and you’re building a bridge, I don’t want you making mistakes. But while you’re learning… absolutely, make mistakes. Part of education is this movement from novice to expert and in that process one makes mistakes. So, I told my class “You have the right to make mistakes” and I use an example. So, let’s say you’re a five-year-old and you’re trying to learn how to ice-skate you fall down, what does a five-year-old do? They laugh, they brush themselves off, they might giggle a little bit, they get up and then they just skate some more. Now, imagine you’re an eighteen year old and you’re learning to ice skate and you fall down. Most of the time, people stand up and say “Did anybody see me?” and they worry about what people are gonna think, instead of just getting up, laughing, and moving on. So, what often happens is the 18-year old never learns to ice skate, whereas the five year old, who’s willing to make mistakes, learns. So, I tell them it’s okay to make mistakes while they’re trying to learn… and also it’s about empowering students to be able to have confidence in themselves… and we talk about this a lot. We did a study of what motivates students in the flipped environment, and part of it is the confidence… the autonomy… that they develop in the flipped environment. When they really think they’ve got and they really understand, there’s a certain level of “Geez, I understand this. I can understand the next thing…” and so on and so forth. That sense of confidence really improves their educational experience.

John: How long are your videos? Do you tend to have very long ones or do you chunk them up into smaller chunks? and what would you recommend in terms of video length?

Dominick: We did a study a few years ago on the optimum length for flip videos, and it came about because our book dealers: McGraw-Hill, Prentice-Hall, Cengage… those are the top three that we have here at Texas Tech… were telling us that they were creating videos and they’re creating these five to seven-minute videos… and I said why are you making five to seven-minute videos?” and they said “Well, everybody knows that the Millennial generation has a five- to seven-minute attention span and so they want five to seven minute videos.” I said: “But, what is best for learning and improving learning outcomes? I don’t care what their preference is. What’s best for them to learn?” He said “Well, we don’t know.” I said “Well, do you have any data or evidence that shows the five- to seven-minute videos are really great for learning?” and they said “No, but we would be interested if someone would do a study and tell us.” So, I had a graduate student who embarked on a study of lecture video length. We set up essentially short video people… those are people to watch five to seven-minute videos… and we did this with master videos, so each video was probably 40 minutes to an hour long. We put stop signs in the video and when they hit a stop sign they would stop. They would work some online homework and then they could pick up again… or they could go off and do something else. but the short videos were five to seven minutes and then we had a long view group, as it were. We allowed them to decide which videos they wanted to do and not surprisingly, 62% decided to do short videos. So, I have no problem with the book dealers’ notion that Millennials prefer shorter videos. But, then we let the semester go on and we didn’t force them to stay into one group or the other. We let them move and we watched their video habits. We weren’t video stalking them, but we could watch their lecture habits using the Mediasite analytics down to the millisecond.

What we found was that 60% of that short video group switched to the long video length, which is very surprising to us, and we did also a variety of assessments. We looked at online homework grades. We looked at quiz grades. We looked at exam grades. We looked at final exam grades. We looked at the American Chemical Society standardized tests that we give as a pre-post and what we found was that there was a subgroup of the long-view group that watched the video as a long video, but then they stopped at specific points to either have a snack with a friend, go to the bathroom, whatever they needed to do… and we called them the long pause people. It turned out that in every assessment that required global understanding… so final exams, ACS exam individual exams… the long-pause viewers actually scored one standard deviation higher than the short viewers. In fact, the short viewers had the worst learning outcomes of all three groups. Then we gave them Likert scale questionnaires, and we also gave them open-ended questions, and we said “Why did you make the switch? What do you find?” and they said “Well, it just got too fragmented to look at these short videos and we couldn’t take what we saw in video A and connect it to video B and so on and so forth, especially if several hours had gone by, because we then had to go back and watch video A again to remember what we forgot. But, if we did the long videos we were able to just put it all together.” Also in that we found that the best optimum video time. So, what constitutes a short video versus a long video for Millennials is 20 minutes or less is short, 30 minutes or more is long. So, 20 to 30 minutes is the sweet spot for video length. That’s what the study showed us and we’ve just submitted that for publication.

Rebecca: Have you adjusted how you’re teaching based on that information?

Dominick: Yes, my videos are roughly in the range of 30 to 40 minutes, but I tell my students “Take the time you need. Take breaks. It’s going to help your learning outcomes.” I also share with them the results of our studies, because I think if you’re going to, in my case if you’re gonna be a scientist, you should be data-driven. So we want our students to know that we’re not just telling them this because it’s something anecdotal, but rather it comes from data that we’ve collected.

John: Have you thought about controlling for self selection and randomly putting students in groups? Because one concern I’d have with that is that it could be the case that those students who select the short videos might have done less well no matter which group they were in or vice versa… although you do have switchers in there.

Dominick: Yeah, I’ve actually been very lucky in my studies that I teach our honors general chemistry sections… and so I look at the SAT scores… I look at their previous class performance… and it’s a very very, as much one can have, a homogeneous group. So there’s really not much of a selection bias, I think, as far as the study goes.

Rebecca: You’ve also done some research on the flipped classroom approach in general, not just the video length. Can you share some of your findings?

Dominick: Sure. We found out a lot about the flipped environment over the past 10 years or so. As I mentioned a minute ago, I’ve been very lucky in that I tend to teach very bright students. I did a five-year longitudinal study on the effect of flipping which has been published in American Chemical Society monograph and we found that the average exam grade increased by 9.2 percent over that five-year period, and that the largest gains in learning came during summative assessments (for example during final exams and externally developed independently normed exams like our ACS exam). We also have done work on what motivates students to do well in the flipped classroom and we just recently presented at the biennial conference on chemistry education regarding the effectiveness of peer mentoring during the flipped classroom. The results there were very astounding to me. It shouldn’t have been because peer-led team learning has been around for more than 15 years, but nonetheless, in trying to tweak the classroom, the addition of the peer mentor took what was already much better than had been before and improved it dramatically. I’ll give you an example. On the American Chemical Society end-of-term exam, without the peer mentor present… So, I give my students incentives if they score above the 95th percentile on that exam, and it’s a challenging exam, they get an automatic “A” in my class. They don’t have to take the final because this exam is normed against thousands of students around the country in a variety of different university settings. Pre-flipping, I had zero to one student scoring above the 95th percentile; post-flipping the average was roughly nine. So, it increased tremendously. With the presence of the peer mentor, the number went from 9 to 34 and so almost a six-fold increase in the number of students scoring above the 95th percentile. The only difference being, since I used to run the discussion sections that I subsequently allowed the peer mentor to do, that was really the only difference in the environment as far as I could see in terms of controlling for the different factors, except for the obvious one of different students. But she did this in the fall of 2016 and the fall of 2017 and there were 34 in one in the fall of 2017, 33 in the fall of 2016. The percentages of the class actually increased. This is kind of reproducible, if you will. So, we’ve looked at a variety of different things, and with regard to motivation we looked at that from self-determinacy theory and found essentially three things that really sort of motivate people: one is autonomy, second is pace, and the third is responsibility.

In the flipped environment, this sense of autonomy… the sense that “Oh yeah, I am really learning something” is very important to the students. The pace, the fact that they could watch the videos in their pajamas, for example, was very important to the students as a motivating factor… and responsibility, the fact that they had to take responsibility for their own education as opposed to being sort of spoon fed in a lecture format was something that motivated them as well. That student just graduated in December, so we’re now preparing that for publication. So, we looked at a variety of different aspects of flipping.

Rebecca: Can you clarify about your peer mentor model? The students are watching videos outside of class, and then they’re coming to a class with you, and then also recitation session with another student?

Dominick: They watch a pre-lecture video. They do online homework. We have a class discussion… work problems… and then there’s a separate recitation section… and historically I have done all of that. I had a very bright student who really wanted some teaching experience and she said “Would you let me run your recitation?” and I, like many faculty perhaps, don’t really want to give up control of my classroom environment. So, it took a lot of cajoling on her part to get me to do that…. and I said “Well, let’s look at it and see what happens.” She took over the recitations and… talk about a motivated young lady… she would provide review sheets for them… she would do all sorts of things that I would do, but the way she would do it, I think, spoke to the students so much more effectively than I was able to do. I think that’s one of the real reasons why their scores went up.

Rebecca: She knew how to meet them where they were at in a way that, as we become more of an expert in our field, we lose touch with that.

Dominick: Yeah, that’s exactly right. So, one of the things that I’ll be interested in following is that there was a recent study that Prentice-Hall published about the Millennials versus the Gen Z students… and one of the things they noticed is that, while there was about a sixty or so percent “like” amongst the Millennials for using computers and computerization and video and things like that, that number was almost half for the gen Z population. It’s almost as though it’s such a normal part of their life, the gen Z population, that there’s nothing special… there’s nothing unique about it… there’s no value-added to doing videos… it’s the sort of normal expectation. So, I’ll be very curious to see how this sort of flipped environment works over the next 10 to 15 years when the expectation is that they’ll be seeing videos at some point either prior or post classroom experience.

Rebecca: …definitely an interesting question.

John: You’ve organized several symposium on flipped classrooms. What are some of the biggest takeaways from those symposia when you bring people from many different disciplines together? Are the results pretty similar across disciplines or do they vary substantially?

Dominick: Well, I think the first thing that struck me with regard to these symposia is how diverse the flipped environment really can be. Since active learning occurs during the classroom time, there’s almost as many different active learning strategies as there are teachers, and so no two flipped classrooms are the same… and that’s the first thing that I learned. The other thing that I learned is that you have to be committed to flip. You can’t do it half-baked. If you try to do flipping, there was one example of a professor who said “Well, all this is is what we’ve always done. You tell the students to read the material before they come to class and then we have a discussion.” So, he said “Okay, I’m going to ‘flip my classroom’ and just have them read the textbook before they come to class and then we’ll have a discussion.” He found very quickly that the students weren’t reading the textbook, so there was very little discussion going on, which frustrated the professor and frustrated the classroom and set up more of an adversarial relationship.. and it was the worst teaching experience he ever had and he said “I’m never going to flip my classroom again.” So, one of the takeaway messages, and this was reported at one of the symposia, is that if you’re really going to flip your classroom, you’re in for a dime, you’re in for a dollar. You do it as well as you can and be very concerned about what you’re putting in and what you’re expecting to get out or it can be a very very bad experience. Now, I will say this, that almost everybody when they get up on board the flipped bandwagon, especially if you’re using technology prior to classroom, it’s hard at first. That’s the other real take-home message. It takes a lot of time to flip your class the first time. But, once you do it, it actually is much more enjoyable. It’s actually easier, I think, than the regular didactic approach. Those are some of the take-home messages.

Rebecca: Can you talk a little bit more about some of the other challenges faculty might face if they’re doing it for the first time?

Dominick: Sure, yeah. The first and biggest challenge that people have is time. Don’t decide you’re gonna flip your classroom two weeks before the start of a semester. It can be the most horrible experience you’ve ever had as a teacher, because students have an expectation that each lecture is going to be there for them. As we all have various things come up or university requirements… meetings… things like that… and you may find yourself at 3 o’clock in the morning trying to record a video for dispersal at 8 o’clock in the morning. So, don’t wait till the last minute. That’s the biggest sort of thing. That’s one of the reasons why people are sometimes a little bit risk-averse with regard to flipping their classroom.

Other things that have come up: saying “Oh, I can just use Khan Academy videos or videos that are on the web” …and they haven’t previewed them, for example, and then they find that the students have a very different concept of the material than the professor has and then you end up spending a lot of time: “Well, which one do we believe? Do we believe the video that you showed us? or do we believe what you’re telling us?” and so it can create an environment which doesn’t propagate trust in the learning experience. Other issues are: online homework versus homework that you put together for the students… what are you gonna do during class time? how are you gonna fill that time. If you’re not familiar with active learning strategies, that can be very daunting. For example, you walk into a classroom and the students have already watched a video… they’ve done some homework… So, what’s your value added? If you’re not used to active learning strategies, that can be very difficult the first time that you’re doing it…. and then do I need to change my assessment strategy based on the fact that now I’m using a different kind of pedagogy? So, there are a lot of different moving parts and I think putting all those moving parts together can somewhat be inhibitory for people who are trying to flip for the first time.

John: What types of active learning strategies do you use? You’ve mentioned some group work on problems, but could you give a few examples of types of activities you use during the class sections?

Dominick: Sure, there’s group work first of all. I send students to the board. I’ll pass out file cards, for example, and ask the students to put down one thing that was unclear in the video or something that they really would like to learn about in addition to the video. So, it’s not just muddy points, but it’s also how do we expand and extend. Because in my case it’s an honors class and I want to give them a little bit more than the normal amount of material and experience, and so those are some of the things that we do. We have to make molecules and structures and do all sorts of things, and so I use human atoms. I have volunteers come up to the front of the class and they have to then make molecules. They have to develop particular structures. They have to show how they bond, how they vibrate, how they move. What they do. So, they have to actually sort of insert themselves into the molecular dynamic, if you will. They’re trying to understand that, as an atom won’t understand its environment. The first couple of times, students aren’t used to that level of kinesthetic learning, but once they get it, then I usually have a fair number of students who are willing to volunteer and come up. Because, while they’re doing that, we’re also discussing, and I’m asking the class: “Okay, so why is it that this particular atom won’t bond here? What’s wrong with this?” and so it’s a real discussion… but now using human beings as the models rather than just making stick figures or things like that.

So, I try to move students through a variety of different learning environments that engage them not only visually, auditorily, but also tactilely. and kinesthetically, and that’s somewhat unusual, I think, in chemistry classes. Because, once again, most people don’t think of chemistry as a visceral activity in many cases.

Rebecca: Actually, it sounds like a lot of fun.

Dominick: Well, and it is, and I’ll tell you I walk into my class the very first day and they’re all very respectful, because Texas students typically are very respectful and they’re honors students. So, they really want to make a good impression. I usually start my class laughing, telling them this is the last time this class will be quiet. A noisy class is a learning class, a quiet class can be a sleeping class, I don’t know. So, yeah, my class is always very, very engaging, I hope.

Rebecca: Have you ever had the moment when you’ve asked your students “What do you want to talk about today?” and then not have anything they want to talk about?

Dominick: Yes, I think every faculty member has that “Oh, my gosh” moment. I always come prepared with questions, so if they don’t have anything to talk about, then I’ll ask them “Well, what did you think about this particular part of the video?” or “Did you really understand this?” or “Let’s take this concept and move it farther” because one of the things I never do in my classroom is just do a rehash of my video. I figure they’ve watched the video. So, I might say “Well, okay, you saw this but how could we apply this in this other setting?” …and then if they really didn’t understand it, then I’ll be able to tell in a heartbeat whether or not there are real questions because they really don’t have a good sense of understanding… and then I can go back and say “Okay, at what point is this breaking down for you? How did this not work?” Silence kind of tells me that there is usually a breakdown somewhere… and so I try to address what that breakdown is and try to correct it. So, that’s what I meant earlier when I say one of the things I do during my class is try to clear up misconceptions… try to address muddy points… and just make sure that they really understand the lecture part as well. That can take anywhere from five minutes to 45 minutes. My classes are an hour and a half long. Depending on how difficult a lecture was… on what their level of understanding is… So, I want them to have deep understanding of the content and so if they’re coming in silent then I worry that that depth is not there.

John: When they see the videos, you have them take tests. Are the tests the same for each student? or do you vary the questions? Is there some randomization there?

Dominick: Well, no. So, we give departmental exams, so the exams are the same for every student, basically. I know many universities have a format and they can vary the numbers that are put in, for example, but no, my students really… they’re fairly separated when it comes to the exam, so I don’t really feel the need to give each one a separate numerical set of calculations.

John: …and it sounds like many of your students are honor students where that might be less of an issue.

Dominick: Right, and we have an honors code through the Honors College, and I tell them the first lecture if you’re cheating you’re out of here. If you’re cheating you’re out of the University… and I’ve had the unfortunate occasion to have students suspended from universities… not at Texas Tech, but at other places I’ve been… and so they understand that I’m very serious about that.

John: On those tests, do they have multiple attempts or just a single take on the test?

Dominick: Well, my tests are usually set up so that they’re half multiple choice, half free response. Because at the end of the day, I tell my students “Qe don’t live in a multiple-choice world… A. agree, B disagree… you know, we don’t… and so I need to know how you’re thinking. While multiple choice exams are expedient in terms of grading, they don’t let me know what you know. So, I give them both because some students like multiple choice exams. They think they’re really good at them. Some students really want the free response. So, it’s a mixture of both. I try to give them a rich assessment environment as much as possible.

Rebecca: So, to follow up on what John was asking, are your homework assignments kind of a multiple attempt to help learning? or is it a one attempt kind of thing?

Dominick: It’s a one attempt. Yeah. Now, what I do allow them to do after they’ve provided their answer, is that I’ll allow the question to be open so they can go back and review it, especially if the answer is wrong…. and I allow them to do that to help them review for the exams as well.

John: So, they can go back and retake it, but only the first attempt counts towards their grade?

Dominick: Right. So, they have to think about what they’re putting in there before they put it in. Because in some cases, I’ve heard stories of students who put a wrong answer in purposefully and then the online learning environment gives them hints or tells them how to work a problem just like that… and then they go back and they have another problem with just different numbers but they’ve already been coached essentially in terms of how to answer the problem. Once again, perhaps it’s because I think you have to do things right eventually at the end of the day, I really want them to get it right… and these are… once again… they’re relatively straightforward pre-class questions. Their designed as just-in-time or warm-up questions. They’re not multifaceted. The questions we’re getting in the class are really challenging problems. They’re challenging problems using an honors book. So, hopefully they differentiate between those.

John: Are they graded on the problems they do in class as well?

Dominick: They’re not. Because part of that environment, once again, is to have groups… have community… have a mentoring process… and so, the ultimate goal in that whole process is the solution of the problem. So, I’ve already tested them in the online learning environment. I’ll test them on the exams. I’ll test them on quizzes. But, in the class I want the process of how to solve the problem come forth and not the grade be the most important thing.

Rebecca: So, we always end or wrap up our podcast with the question: what are you gonna work on next?

Dominick: Well, I have a pretty active and diverse chemical education research group and with regard to flipping, specifically, I mentioned that I had a recent PhD who looked at motivation in the classroom and what we found there were there were basically three reasons that motivates students to want to do well: autonomy, pace, and responsibility. Through the flipped environment they learn how to develop confidence in their ability to learn, and secondly they liked that the class is largely self-paced and they get to watch the videos in their pajamas if they want to, for example… and finally they really appreciate the fact that they are responsible for their own learning. So, we’re going to be looking at the role of metacognitive intervention as an autonomy motivator in the flipped classroom. That is to say, if we help them think about how they’re thinking during the early parts of the flipped classroom, do they proceed to confidence in their ability to learn that much faster… and we’ll also be looking at how the flipped classroom, especially with community building activities and community building learning strategies can improve the learning outcomes among historically underrepresented communities in the sciences… though especially communities where the notion of family and community is so important in their lives… that are not necessarily in the classroom. So, those are the two areas that we’re going to be moving into with regard to flipping… and I have a number of other projects that are not related to flipping as well. So, it’s a very diverse group of questions that we’re trying to answer, but once again I think that the flipped environment is a very value-added environment for both the students and the faculty… and so I think it’s a mature pedagogy in the sense that we talk about process oriented guided inquiry learning (POGIL) being mature and peer led team learning activities (PLTL) as mature pedagogy. Service learning is another mature pedagogy that has matured over the last 20 years or so… and I think it’s now safe to say that flipping is a mature pedagogy. In fact, at the biennial conference on chemistry education, there was a wonderful paper doing a meta-analysis on flipping and the presenter showed that in terms of looking.. I think he looked at 18 or 19 different papers on the flipped environment… and he found that, in general, there’s about a 30% improvement in student learning outcomes and it’s even better in organic chemistry than general chemistry (which was surprising to me). But, nonetheless it really does improve learning for students and that, in the final analysis, is what we’re trying to do.

Rebecca: It sounds like some really interesting projects. We’ll be looking forward to finding out what you find out.

Dominick: Thank you.

John: Well, thank you. this has been fascinating.

Rebecca: Yeah, thanks so much for spending some time with us today.

Dominick: Thanks, I’m gonna have another sip of tea. Thank you.

Rebecca: Thank you
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

41. Instructional Communication

There is often a misperception that being a well-liked, kind and caring faculty member comes at the cost of rigor or high expectations. In this episode, Dr. Jennifer Knapp, an expert in the field of instructional communication, joins us to discuss strategies we can employ to make the classroom a positive and productive learning environment.

Show Notes

  • National Communication Association instructional resources
  • Mottet, T.P., Richmond, V.P., & McCroskey, J.C. (2006). Handbook of instructional communication: Rhetorical and relational perspectives. London: Routledge.
  • Chesebro, J.L., & McCroskey, J.C. (2002). Communication for teachers. Boston: Allyn and Bacon.
  • The journal Communication Education also contains many useful articles.

Transcript

Rebecca: There is often a misperception that being a well-liked, kind and caring faculty member comes at the cost of rigor or high expectations. In this episode, we turn to an expert in the field of instructional communication to provide us with strategies we can employ to make the classroom a positive and productive learning environment.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Dr. Jen Knapp, an associate professor of communication studies and an associate dean in the School of Communication, Media and Arts at SUNY Oswego. Welcome, Jen.

Jen: Thanks, John. Thanks, Rebecca.

Rebecca: Thanks for coming. Today, our teas are:

Jen: Black raspberry green tea.

John: Tea Forte black currant tea.

Rebecca: I’m having Prince of Wales tea.

John: We’ve invited you to join us today to discuss your primary research area, instructional communication. What does research in instructional communication tell us about creating a productive classroom environment?

Jen: So, I’ll start by telling you exactly what instructional communication is… and what we do. Essentially we’re talking about communication between instructors and students that enhances learning or perhaps in some way affects the learning process negatively. We’re more interested in how messages are delivered than the actual content of the course. So, we’re talking purely about communication behaviors by instructors and students and how that affects what goes on in the classroom, which should be learning.

Rebecca: Is your area of research focus only on in-classroom communication or does it expand beyond the classroom?

Jen: One of the things I research is out-of-class communication and I think maybe at some point we will talk a little bit about that, but primarily I focus on what is going on in the classroom – specifically what instructors are doing in terms of communication and how that affects students.

John: What can instructors do to create a better classroom environment?

Jen: There are a lot of communication variables related to instructional comm. The primary instructional comm bread-and-butter concept is this idea of immediacy – and immediacy has to do with increasing physical or psychological closeness between instructors and students… and the bottom line is, if you, as an instructor, engaged in these verbally and non-verbally immediate behaviors, there’s going to be more positive outcomes in the classroom for your students… specifically learning… but ultimately, what I think is really interesting, is that even on a nonverbal level, you can influence what’s going on with your students and how they are perceiving your messages… but also how they’re wrestling with the content. So, it comes in two flavors: verbal and nonverbal immediacy. We were talking about nonverbal communication… we’re talking about everything but the words. People will commonly refer to it as body language, but it’s also your tone of voice and how you use space and touch and things of that nature. E ven something as simple as eye contact can make a difference in terms of what’s going on between instructors and students in the classroom… engaging in vocal variety… but also using humor… calling students by name… all of these things can help increase the connection between students and instructors. Most people believe that the instructor-student relationship is an interpersonal relationship, or a type of an interpersonal relationship, which means you’re connected to each other in some sort of meaningful way. All the things that you value in terms of how you communicate with your friends and your family… a lot of that plays into what goes on in the classroom as well. People want you to make eye contact. People want to be around people who are funny. So, there’s a lot of research that suggests instructors that use humor in the classroom tend to get more positive evaluations, but also there’s more learning that occurs in the classroom if an instructor is using humor effectively.

Rebecca: Does that shift with culture?

Jen: Yes. All communication occurs within a context. Culture is our biggest context. Immediacy, in particular, is very culturally based. It is something that you need to be careful of. Most of the research that I do and that I’m familiar with has been conducted here in the United States with traditional college-age populations, but certainly if you were to travel abroad and perhaps you were to teach a semester away then these rules may not apply.

John: …and it might not also apply if we have foreign students here who have not adjusted to U.S. classroom climates.

Jen: Of course. Yes.

Rebecca: So, what are your biggest secret secrets? [LAUGHTER]

John: …related to teaching.

Rebecca: …related to teaching.

Jen: Oh… no one warned me that I had to divulge my… my biggest secrets today.

Let me go back to immediacy for a little bit and talk a little bit more about that and why that essentially is a positive thing. I don’t think I listed the outcomes. You’re perceived as more approachable… you are perceived as more student-centered… more responsive… you’re friendly… you’re open… and you are essentially inviting communication. So, if you engage in these types of behaviors you are going to invite communication. If you are an introvert, I don’t recommend that you try to be overly immediate because students are going to pick up on that and then they’re going to think: “Oh, well this person is friendly. This person is a good listener, so I want to spend time with them. I’m gonna visit with them. I want to get to know them.” So, you are inviting communication when you engage in these behaviors. But something you should also keep in mind, in terms of immediacy, and this is probably more of a personal choice for me… and other people may not agree… is that it decreases the status differential between you and your students. You are trying to give the perception (hopefully it’s not just a perception and it’s reality) that you care for your students… you are engaged… you are enthusiastic… they see that you’re passionate about your content… you’re moving around the room… you kind of work the room when you engage in these physical behaviors… and so it decreases the status differential between you and them. For me, I like that in my classroom. I don’t want to give the air of being the professor who has all the knowledge and the expertise and I’m looking down on everyone and being condescending. For me, I like to have… not an equal partnership… but I want my students to feel like they are a partner in what is going on in the classroom and anyone can share an idea. I can share an idea. It’s open. It’s friendly… and that’s important when you’re teaching something like interpersonal communication. You’re talking about relationships. Sometimes that class turns into a self-help class and everyone’s talking about their problems with their partner or their family. Everyone’s telling personal stories. You can’t not tell personal stories when you’re in that class. You don’t want anyone to feel like you’re being judged or that you are judging other people. So, I like to have low status differential… low power distance between me and my students… and I can get to that point by engaging in these types of behaviors. I don’t know if that’s a secret, necessarily.

Rebecca: …maybe a secret if you don’t know about it.

Jen: …it could be…

Rebecca: …not a secret anymore,

Jen: …it could be… but I think a misconception… and if you think of it in terms of power differential or having low power distance between you and your students… and some instructors might be uncomfortable with that setup…

Rebecca: Is there a difference in gender, related to this low power difference perception?

Jen: I don’t know if there’s a difference in perception but female instructors and feminine communicators… so those are two different things… are more likely to engage in immediate behaviors than more masculine communicators.

John: You talked a little bit about how instructors can create more of a sense of immediacy by walking around the classroom, by maintaining more eye contact, and by using humor. What else can faculty do to help create the sense of immediacy?

Jen: So, remember that it’s psychological closeness or also physical closeness… if you ever had a student approach you after class and they want to talk to you, and the desk is between you and that student… or the teacher station… or something like that. Something you can do in order to create that perception of closeness is to come out from behind objects. You don’t want to stand in front of the classroom. You don’t want to stand behind the little desk. If you’re in Lanigan 101 and you’ve got that teacher station, but you also have a couple of tables in the front… the student approaches, you don’t stand behind the table. You can move out from behind the table… trying to make eye contact with people in the room… smiling goes a long way in terms of just coming across as approachable and friendly… and the idea is, if people find you to be approachable and friendly, they’re going to engage in something like out-of-class communication. You’re not going to go to your instructor’s office hours if you feel like they’re an evil troll, but you will go to their office hours if it feels like “You know what? I got this thing that’s going on in my life. I need some extra time on an assignment. I feel like if I were to go see Rebecca, she seems like the type of person who would understand or who would at least listen to me” and you can do all of that just by modifying your behavior in the classroom.

Rebecca: What happens when that openness gets to a point where those conversations move beyond class-related conversations like you just mentioned?

Jen: Yeah.

Rebecca: So, that particular example is “There’s something in my life but it’s related to the class.” What happens when it goes past that?

Jen: Sure. That is definitely a risk. If you are engaging in this behavior and you are giving the impression that you are approachable and friendly and someone that listens, as I mentioned earlier, that invites communication. So, you will have students show up at your door for reasons completely unrelated to the class… and maybe it is to seek help or advice about the relationships because they’re in your interpersonal communication class… or it just might be they think you’re a friendly person to talk to. That has happened to me and I’ve sat through very awkward conversations or heard things from students that I felt like I had no business hearing. But, you know what? Maybe if you can be a force of good… or if they are disclosing something to you… if it’s something like a sexual assault or something like that, then obviously it’s much better… you don’t ever want to hear that type of message… but it’s better for them to feel as if that’s someone they can talk to you and they can confide in and then you could help them get connected to resources, or something like that. But, then there are also, on a much less serious note, students who are just looking for a friend and they’re hangers on… and they don’t understand leave-taking cues. So, you might be packing up your things to teach your next class and trying to give the signal that it’s time to go, and they might not realize that. Sometimes you have to have very direct conversations at that point: “I have to go. I’m sorry, I can’t talk to you about this any longer.”

Rebecca: You had mentioned a physical closeness, but you also said that there was verbal immediacy as well?

Jen: Right… psychological closeness… the verbal messages would be: using students’ names, using humor, telling personal stories, engaging in self disclosure. Those would be all examples of verbal immediacy… and then the nonverbal immediacy would be: moving around the room, using vocal variety, decreasing space between you and the students, using eye contact. That would all be examples of nonverbal immediacy… and ultimately this leads to affective learning… and my goal as an instructor is always to create more communication nerds. So, I did not start as a communication major, but once I fell into it, I absolutely fell in love with it and thought I cannot live my life without this… and everything I was learning in the classroom I could immediately apply outside of the classroom. Every day in the classroom that is my goal with my students: to get them to know something… be able to do something… to better their lives… better their relationships… find an internship… whatever it might be… and I just love helping to produce comm nerds… people who are quoting comm theories to me… who are analyzing their conversations or the relationships and then telling me about it… or having them explain how they taught their father about cognitive dissonance theory and then how they used it in a work situation or something like that. That’s something that I love… and ultimately affective learning, I feel, is really one of the best outcomes of immediacy and something that’s important to instructional communication: getting students to learn because they like what they’re doing… they see the value in it… they develop a positive attitude to what’s going on in the classroom and the content that you’re teaching them… and also a lot of these behaviors… instructor behaviors… Frankly, if you like your instructor, there’s a good chance you’re going to work harder for that instructor and that you’re going to do well in the class. You might get to a point where you don’t want to disappoint your instructor… but I’d actually like to ask you a question: if you could talk about some of your favorite professors and the types of behaviors that they engaged in that you really liked?

Rebecca: That’s a good question. I need a minute to think. It’s funny, but the first thing I can come up with are all the behaviors I don’t like… [LAUGHTER]

John: Yeah… a strong emotional reaction, either way…

Jen: Sure. Absolutely.

John: I think, thinking back to my college career, which was a while ago… sometime last century… many of the professors that had the most impact on me did exhibit these behaviors. They interacted with you outside of class a bit and they demonstrated some sort of passion for the subject.

Jen: …and I think students want you to care about them… for sure. I start all my classes by asking them how they’re doing? What’s going on? So, many are in clubs and organizations, so I say “What are you promoting right now? What is your organization doing? What’s important to you?” and then finally “Does anyone have any good news?” I just like to hear good news and students appreciate that… and they sometimes, maybe once a month, remember to ask me how I’m doing, which is a win I think… to get that at least once a month? [LAUGHTER]

Rebecca: If you model it and eventually eventually it’s reflected back, right? [LAUGHTER]

Jen: Yeah, Eventually. I guess that’s the theory behind it.

Rebecca: The faculty that I remember the most, or that I had really good experiences with, are the ones that I had, probably, interactions with outside of class. Those are the faculty that I felt like I could go talk to. Who maybe pushed me harder because they got to know me a little bit, to know how to push me in a way that was positive rather than pushing in a way that would have a negative impact on me. They always got more out of me. So, I think everything you’re saying was completely true for me.

Jen: Yeah. That out-of-class communication piece is really important, and before we were studying it in communication and calling it out-of-class communication, people in education were calling them out-of-class experiences. There’s a whole program of research in education devoted to this… and they studied more the outcome of those events. In comm, we study what leads to out-of-class communication more than anything else. In education, they were saying “But here’s the good news… here’s all the good stuff that happens if students are communicating with you outside of the classroom.” So, whether it’s during office hours or whether they run into you at Price Chopper, the first time you see an instructor outside of the classroom can be a bit daunting or jolting. Students think that we just get put away in a closet overnight and brought back out the next day to teach. The first time they see you it might be a little bit weird, but ultimately if they see you, they see you as human and you stop and you say “Hi Rebecca. Hi John. What’s going on? I know you’ve been playing your bass lately. What are you working on? What are you excited about?” In those little things, like you mentioned, Rebecca, they add up and they definitely make students feel better about themselves. It really helps with their development of sense of self and can also help with motivation in the classroom.

John: How would this work in a larger class setting? Can these behaviors scale very nicely? Certainly walking around can, but what else can you do if you have a class of 400 students or so?

Jen: Sure. All of this can certainly be scaled up. Now I don’t recommend if these types of behaviors or being immediate does not come natural to you, that you launch right into trying to do all these things, because students will sniff out that…

Rebecca: inauthentic…

Jen: Yeah …lack of authenticity. They will definitely sense that. The same with verbal immediacy; using humor is an example of verbal immediacy, but if you’re not funny do not try to be funny. It will not go well. But, certainly you can scale this to larger classes. Whether you’re teaching Micro at 400 or I used to teach Comm 100 to over 200 students and I want to say (I’m sure it’s not true…)… I want to be able to say that my teaching style was not that different, whether I was in front of 20 students for a capstone or 200 students for a large introductory course, because ultimately I’m still teaching the way that I think students should be taught. I’m still engaging in these behaviors. I’m still aware of other instructional variables like clarity… like credibility. All of those things are still important. It doesn’t matter necessarily the size of your audience. We typically say “the bigger the audience, the more formal your communication needs to be.” But, I think there are exceptions to that as long as you are still being authentic in some sort of way. Any of our instructional variables that you might learn about can certainly be applied in a large lecture room. There’s no set of categories that “here’s what you do in a large lecture versus here’s what you do in a smaller studio level class.”

John: I know when I teach the large class I generally get in somewhere between 30 and 50 flights of steps every class and usually two or three miles of walking, because it’s a big ways around.

Jen: Oh my gosh. Yeah, Lanigan 101 is a big room. It’s a hard room to work too, because there’s a whole sea of people in the middle that you can’t get to. That’s where eye contact really makes a difference. You just try to make eye contact with them because you can’t physically get that close to them, but you still want them to feel as if you are speaking directly to them, and you’re not trying to be everything to 200 people in the room.

Rebecca: Other than immediacy, are there other theories or principles that we should be aware of as instructors?

Jen: There are a lot of instructional variables, and I think I’ll share some resources that maybe your listeners would be interested in taking a look at later on. Something else that is important to me is credibility. Credibility is essentially believability, and if you are a professor you should be in the business of being believable. It’s important to remember that communication is about messages, but at the end of the day meaning is in the mind of the receiver, and so you can do your absolute best to craft what you think is the perfect message. However, whoever is getting or receiving that message in decoding that message… it’s going through their personal filter. It might be a very benign message, but maybe they’re having a bad day… maybe they’re really hungry, so they’re not quite paying attention. You don’t have complete control over how people decode your messages. You have to remember that meaning is in the mind of the receiver. What you might find credible is going to be different than what John feels as credible. Credibility is a perception. Whether or not I am truly credible doesn’t matter. As long as you think I’m credible, I win. I might be a complete moron, but if you think I’m credible then it doesn’t matter because then everything I say is going through that credibility filter.

We usually talk about credibility as the three C’s: competence, character, and caring. …and for some people different elements are more important. Some people (who perhaps are more logically based) competence or that perception of expertise or knowledge rules the day, always. For some people, they just want to feel like you have some level of goodwill, and you have their best intentions in mind, and that’s the caring aspect of it… and for some people it’s character or it’s honesty and trust that you are being honest and your being truthful with them, and nothing else matters other than that character piece or that trust piece. For different people, different things are important, or they’re gonna pay attention to different aspects of the message based on what they value more… whether it’s the competence the, character or the caring. So, credibility is an instructional variable and it’s not just instructional it goes across different communication contexts. But, that’s something that I think would be interesting for people to know about and to learn about power… how you influence what’s going on in the classroom… also something that can be studied across communication contexts. But how ultimately are you influencing your students? Are you getting them to do what you want them to do because you are rewarding them? …’cause you’re punishing them? or are they doing it because they feel like it’s the right thing to do and they are internalizing your message and they believe in the value of the work? …and there’s some other types of power as well… and then just plain clarity. Clarity is another instructional variable that’s important, in terms of how you structure your messages for your students in the classroom.

John: The next thing we should probably talk about is: what might go wrong or what should faculty avoid doing that might create a negative environment?

Jen: There’s a program of research in the 90s that investigated teacher misbehaviors. So, I thought it’d be fun to ask you what some of those categories are. I bet you can come up with a lot of teacher misbehaviors. So, what are things that instructors do that students don’t like? Just rattle them off at the top of your head.

Rebecca: I’m thinking. I’m a thinker.

Jen: Don’t overthink it.

Rebecca: I know, but I have to still think. They don’t like it when when you’re condescending or like a know-it-all.

Jen: Sure.

John: …especially if you’re not only condescending but wrong. So, that competence is kind of important as a factor there.

Jen: Yeah. I do want to add a fun fact… yet, also our cross to bear as people who study communication. I love producing communication nerds. I love people who are analyzing their conversations. They are putting into practice positive conflict management strategies. However, you can often get accused of applying your communication knowledge in a less than savory way. So, some people get really upset because they feel like you’re Jedi mind tricking them with your communication skills. …something to keep in mind… that as comm majors, we often get yelled at for actually using what we’re learning in the classroom… because people don’t wanna fight fair. They want to get below the belt and say mean things when you’re like “Let’s be constructive. We don’t want to be verbally aggressive. Let’s try to just be argumentative… we’ll stick to the arguments.” That doesn’t go over very well when you’re having a fight with your girlfriend. [LAUGHTER]

Rebecca: I think being late…

Jen: Yup, that’s a big one.

John: …or not being prepared at the start of class is another thing.

Rebecca: I hate when the technology doesn’t work or there’s serious user error.

Jen: For sure. Anything else on your mind?

Rebecca: They don’t like it when you don’t know their name or… that extends to… it’s not just name but gender pronoun… pronunciation. There’s a whole slew of things that probably snowball onto that.

Jen: Absolutely. You got some good ones. I thought I would touch on a couple others that maybe you hadn’t been thinking about. You did mention being condescending… but sometimes being sarcastic and using put-downs is a problem for students, naturally. Unreasonable or arbitrary rules… If you think about your syllabus and what’s in there. Your syllabus sends a message on day one. You want to think about ultimately what you’re sharing with students based on your syllabus. Inaccessibility… Students want to be able to see you out of the classroom. They want to visit you during office hours. Being late… definitely. But one I think that’s interesting, that we probably don’t often think about, is information underload. Students want to be challenged. Most students want to be challenged, and this ties into something that we’ve been talking about previously. There’s this misconception that if you have a classroom that seems to be open and friendly and you are approachable as an instructor, that that means you are the easy instructor… and I have a major problem with that. I think it’s absolutely possible for you to do all of those things to be liked as an instructor, but to also have high standards… and frankly, if you set a bar for your students and they exceed it then you should continue to raise that bar. …and ultimately having or doing tasks that the students don’t feel like are getting them to the end goal of the course is actually considered by them a misbehavior. That’s something that you would want to avoid.

Rebecca: It was a good one that it’s most definitely overlooked… and you definitely hear those conversations: “Oh, take this class because so-and-so is easy. All we do is talk.”

Jen: Yeah, there’s certainly that misconception too… in comm studies, in particular, like “What do you do in that major? …and I come from what we call “communication and social interaction” or “communication,” “communication studies.” We’ve had different names over the years. We thought CSI would be super cool and hip and turns out people are like “I don’t get it. I don’t know that is.” [LAUGHTER] We’re changing it back to “communication,” but if I tell someone “Oh, I’m a journalism professor or public relations professor or a broadcasting professor” like everyone has an idea of what that means… and if I’m the communication professor they’re like “So, you just talk all the time?” I’m like like “No, there’s actually more to it than that.”

John: Well, you do talk all the time, but it’s about something. [LAUGHTER]

Jen: We’re communicating about communication. So, it’s all very meta. Yes. [LAUGHTER] It’s a good time.

Rebecca: It’s very deep.

Jen: Yeah, it is. Of course it is, all the time.

John: Where can faculty go to find more information about instructional communication?

Jen: Penfield [Library at SUNY-Oswego] does own the handbook of instructional communication. We asked them (we being the Comm Studies department) a few years ago to purchase that so people can check that out of the library. The National Communication Association has some great links in terms of instructional communication and what to do in the classroom and how to enact certain behaviors. That is a great resource. There’s another book that I like a lot called Communication for Teachers which summarizes a lot of instructional communication literature and also talks about how to apply that to a classroom… whether it’s K through 12 or in a college classroom.

John: We’ll share links to some of these materials in the show notes.

Rebecca: So, we mentioned earlier on about talking about communication that happens outside of the classroom and we’ve hinted at a couple things here and there, but could you talk a little bit more about those out-of-class experiences and that impact on learning?

Jen: It impacts student motivation, positively. So, they have those moments…and it can just be passing in the hallway or walking through the breezeway in Marano and it’s just a simple “Hello” to a student. That’s something that they can take with them, put it in a little pocket and store that. “Oh, Professor Kane remembers my name” or whatever it might be that makes a difference. But, ultimately it gives a student an opportunity to connect with you on a different level… in a different sort of time-space continuum, if you will. Everything is crazy before class… after class… lots of people want a piece of you… If they take the time to come visit you during office hours and that’s more that’s one-on-one time that they get to spend with you to develop those relationships and certainly that can help them. Students who engage in more out of class communication tend to do better in their classes than students who do not engage in out of class communication. But, it also has… besides classroom outcomes… has better outcomes for them personally. Networking, which you were alluding to earlier… as you met with your professors, you got to know them… they got to know you… now, when they get a call that someone needs an intern or needs someone who can do graphic design work, well you and I were just talking an hour ago in my office and I know that you have this skill set, so now I’m gonna pass this opportunity on to you… because I know that you’re interested and I know that you can do the work. So, that’s a tremendous outcome for students if they take the time to get to know their professors and their professors know them, when those opportunities come past, they can give those to the students that they’ve met and they’ve spent time with… and it just gives students another way to practice their interpersonal communication skills.

John: We always end with the question: What are you going to do next?

Jen: Something that is important to me, as someone who studies communication, 1. is to always correct people who say “Communications” instead of “Communication.” No “s” just “Communication” but also to show people the value of what we study, in what we know as communication scholars. One of the committees I sit on is the Title IX committee, and I’m also a Title IX investigator. One day, Lisa Evaneski was describing some of the cases that she was seeing as Title IX investigator and she said “These aren’t necessarily Title IX cases. We’re not talking about instances of interpersonal violence or sexual assault or anything like that. They’re just, I don’t know, messy breakups…” and I’m like “Ah, we can help with that.” So, in communication, and those of us that study interpersonal communication, we talked a lot about how to treat people positively… how to breakup constructively… how to just be a good human during those difficult times… and so there’s been a group of us that are working in comm studies to create a workshop that Lisa can potentially direct people to that maybe need a little bit of coaching about how to treat people or how to be in a relationship or how to break up… but also we would open it to the campus in general. So, anyone who’s going through a nasty breakup or thinking about “maybe it’s time for me to dump this person and move on. How can I do that in a healthy positive productive way?” …how to use social media or not use social media during during those those times… So, we’re working on building a workshop on messy breakups… which will maybe eventually have a different title, but so far we’re just stuck on messy breakups.

Rebecca: I think it works.

Jen: Yeah, and our goal would also then be to turn that into some type of research as well. Something that we could could share with our discipline, in terms of how we are applying and using our knowledge as communication scholars to help solve a problem on campus… something of that nature… A dream that I’ve always had, and that I know John knows about, is to develop some sort of instructor boot camp. It would go nicely with your badging program if we could have something where people would learn ultimately how to teach… or how to best employ some of these instructional communication variables, in order to get the best out of their students. We can also talk about how to build a syllabus… how to write a syllabus… how to structure assignments… how to ensure that your messages are clear to your students… those types of things. So, one real thing that I’m working on and one thing that I would like to at some point…
JOHN… an aspirational goal…

Jen: Yeah… actually launch…

John: oI think we’d like to see something along those lines to here.

Jen: …and I do think it’s important to say I’m not the only person that knows about this stuff and that studies it so I’ve got colleagues in Comm Studies Katherine Thweatt and Mary Toale, all three of us graduated from the same doctoral program in instructional communication, so there are a handful of us that are interested in this and that are dedicated to it, along with some other great interpersonal scholars in Comm Studies.

Rebecca: I think that what’s really exciting about your workshop idea… that hopefully is not just an idea real soon… is that students will see a discipline in action… and the more ways that we can do those sorts of things on campus, the more real it is for students about how these things that seem like they’re not applicable or they’re not applied somehow…

Jen: Right.

Rebecca: …in action. Some fields are maybe more obvious than others and so the more we can be visible as scholars in the community and sharing that knowledge with the community, I think, is always really nice.

Jen: Yeah, instructional communication is a great example of an applied field.

John: Very good. Well, thank you.

Jen: My pleasure. Thank you both very much.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

40. Design Thinking

When we design our classes, we often focus primarily on the learning objectives that we determine for our students. Might our classes be more effective if we focused more on our students’ needs, objectives, goals, and the barriers they face? In this episode, we examine how we can use design thinking to make our classes better serve students’ needs.

Show Notes

Transcript

John: When we design our classes, we often focus primarily on the learning objectives that we determine for our students. Might our classes be more effective if we focused more on our students’ needs, objectives, goals, and the barriers they face? In this episode, we examine how we can use design thinking to make our classes better serve students’ needs.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca:This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca:Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Allison Rank joins us again today as a guest host. Rebecca has been once again displaced and she’s in the guest chair this week. Welcome Allison and Rebecca.

Rebecca:Thanks.

Allison: Thanks.

John: Today our teas are:

Allison: Cold water.

Rebecca:…once again.

John: …and my tea today is a ginger tea

Rebecca:…mine’s English afternoon. I almost thought that with today’s episode we should have made it water day or something, in honor of Allison.

Allison: I will keep coming, but I’m not drinking tea.

[LAUGHTER]

Rebecca:We’ll have to get you sick one more time.

[LAUGHTER]

John: There’s been a lot of discussion in various groups about the importance of introducing design thinking and we’ve also heard bits of that discussion from you on previous podcasts. So today we’re going to talk a little bit about design thinking. What exactly is design thinking?

Rebecca:Design thinking is a methodology that is probably familiar to most people in creative fields, because it’s something in common that most creative fields have. So, it’s not necessarily unique to design, and it’s also a process that’s common with innovators. But, really it’s an idea that you’re empathizing or getting to know your audience and seeing the process or the solution through their eyes. We’re not just coming in with the idea that “I know the solution. I know what the outcomes gonna be already…” but being open to the idea that it could be something else… and not having my preconceived idea… and that’s where the innovation actually comes in… and so you use that empathy to help define a problem. Then there’s a big process of ideation. You’re really breaking out of the normal, or the quick or ready-at-hand, answers or solutions that we might have from the ideation stage. Then you prototype. You’ll try something out at a small scale, then you test it see how it actually works…. and then you go back and revise it and do it again and again and again… When you’re using design thinking you may end up ultimately with a solution, but as a faculty member the way that I implement it, it’s never probably fully finished… that implementation stage just goes again and again.

John: Well, isn’t that just the notion of recurring reflective practice? A useful thing to do in any case, right?

Rebecca:Yeah, I think the key piece that tends to be missing from more traditional ways of developing curriculum or being in the classroom is the piece of looking at it through the students perception or lens… and it’s not just thinking about “what classes have they taken before” or “what don’t they know” but rather: “Where are they having fear? Where are they struggling? Where are they concerned? Where do they have some delight or surprise in the subject matter? What are their goals?” …and actually starting with that rather than starting with “Here’s the content you need to have.” So, I think that that’s a big difference between more traditional practices… and obviously there’s backwards design that starts with the end goal, but the design thinking mixes more of the students perception than would be otherwise.

John: So, more focus on audience?

Rebecca:Yeah, definitely.

Allison: Can you actually distinguish between backwards design and design thinking for a minute?

Rebecca:Yeah, I can try. [LAUGHTER] Backwards design… I’m sure many people have heard in relationship to curricular development… and that’s generally thinking about what you want the student outcomes to be… and you start with the outcomes and then you essentially work backwards from there. so how do you get students to end up at those outcomes… but, it’s usually more from implementing evidence-based practices and based on that science that’s what you would do… and it doesn’t necessarily take into account this particular group of students, or that audience piece as much. There’s definitely things in common.

John: Just to elaborate a little bit, you’d start with the goal, and then you’d work back to how you would measure that goal, in terms of designing assessment, and then you would build the learning objects… but it’s a somewhat different focus. It’s more just on getting students to the goal without as much focus on their needs or their motivation or interest, perhaps?

Rebecca:Yeah, I would say that it aligns really well with more traditional design practices, rather than user-centered design practices… where let’s say, if I’m doing an actual design work for a client, like a graphic design piece, and someone comes with their business goals or outcomes and I’m just focusing on that… that’s very related to backwards design and that we’re thinking about our curricular goals that’s aligned. This piece brings in that user centeredness or the audience.

John: Are the two necessarily exclusive though? Because I would think that you could use a design thinking approach as a way you design the learning approach, or am i misinterpreting that?

Rebecca:No, I think all the design thinking does is adds audience into backwards design. It’s backwards design but there’s a much bigger focus on audience as a result. But the other pieces are in common.

John: Maybe if you could provide an example of how design thinking might affect the way you structure instruction…

Rebecca:Okay. I’ll use an example from one of the classes that I’m working on right now, because it’s in my head. One of the things that I’ve been struggling in the classes that I teach is getting students to understand designing with accessibility in mind… and accessibility means making sure that whatever product you’re designing is available to all people including people with a wide variety of disabilities… from maybe a more traditional backwards design approach I might identify: they need to understand the accessibility principles, apply them in a design (and that would be the outcome that I’m looking for), and then I would figure out how I would measure that using a rubric on a project, and then move back from there and figure out some learning activities that they might need to do to practice those skills and have retrieval practice and that kind of thing… but if I’m thinking about it from a design thinking perspective, I’m not assuming that the outcome is gonna be this particular design with this particular thing in mind. It’s not a project, necessarily, but rather I’m gonna start with the idea: “Why do students struggle with the idea of accessibility in the first place?” Part of it is they think that it doesn’t apply to the work that they’ll be doing, so they don’t see how it’s relevant. Part of it is they may not understand the wide variety of disability that’s there… or how people with disabilities might use a website. They’re not familiar with the assistive technologies… how they work, etc… in part because a lot of existing examples are at a level much higher than a beginner. So, it might be hard to relate to. They get frustrated because there’s a technical component to it and that’s also new on top of the design piece. See, if you keep going through that whole list of activities… or maybe they feel like they can’t talk about disability because they don’t have the language to do that. Now, there’s a whole pile of other potential problems that I need to solve, rather than just assuming I need to teach the accessibility principles. For me to be able to do that now, I’m realizing. I need to teach a little bit about disability… I need to teach about talking about disability,… as part of that integrated process to actually reach that goal… and not just the content that would reach that goal.

Allison: If, as a faculty member, and I’ve been struggling to do this a little bit… there was a workshop that you offered at the start of summer that a number of faculty went to, what would you recommend when you’re trying to first start thinking about making a shift to designing classes using this strategy?

Rebecca:I think the best place to start is… one small thing. Maybe there’s one thing that students are struggling with in your class. I’m not sure what that might be. Maybe it’s writing an argument, so we can be a more specific. That’s the instructional objective. Now you need to start thinking about “how does that relate to a student’s needs?” or “where do they get stuck or where do they get frustrated?” or “what do they already know?,” “what are their misperceptions?” and you start from their perspective and some of that’s through observation. So, you can probably answer some of those questions just based on your own observations from having taught writing in your classes.

Allison: Sure.

Rebecca:Some of it might be you need to ask some questions of that population to better understand why they feel like they get stuck. It might be interviews. It might be surveys or questions in class.

Allison: Yeah.

Rebecca:You get a feel for that, then you start trying to figure out “Okay, if I want them to write arguments better, but this is their barriers, hurdles, and goals, where can you find some… essentially… synergy between those that you could focus on? …and then define an actual problem that you want to solve. You might initially start thinking “Oh, the problem I’m trying to solve is students don’t know how to write an argument,” but actually if you’re using design thinking you would be open to the idea that that’s not the problem you’re trying to solve… and you allow that exploration to allow you to gather more information to come up with a better problem statement that puts the user or the audience (or in this case, the student) at the forefront as opposed to your objective at the forefront.

Allison: I think the place where for me, even in the initial explanation, as a faculty member I get tripped up is the immediate knee jerk that is “Sure, but then by the end of the semester I need them to write an argument.” So, at some point I’m gonna refocus to the objective being writing an argument. When in the process does that come back?

Rebecca:What you’re assuming is that that ever goes away… and it doesn’t. Rather, you’re just reframing the problem. If the hurdle is “I don’t know what an argument is…” that there’s two sides to an argument perhaps…. at least…

Allison: Yeah…

Rebecca:…or there’s multiple perspectives that are involved in the argument, then it might start with that, in overcoming those perceptions… in doing some exercises to move towards… it’s not that you won’t get them to do that in the end…. but maybe you’re always thinking it has to be a paper… and I’m just putting words in your mouth….

Allison: Sure.

Rebecca:…it might not be that…

Allison: Sure.

Rebecca:…but maybe your assumption is that the only way I can get there is they have to write a paper… but maybe there’s some other kinds of ways that they can practice doing an argument that’s not a paper first… that might take advantage of some of their strengths or their perceived strengths that could help them be more confident to do the thing you want them to do ultimately. But, I think it’s really about the starting point… the journey to get to that place is very different if you’re thinking about the student first, rather than my goal of writing an argument first.

John: When you talk about focusing on the student first, would you focus on what preconceptions they have? what barriers? and what’s preventing them to get to that? …and then focus on designing ways of getting them to achieve that goal.

Rebecca:Yeah, definitely, and also what some of their goals are. They might have goals that are related, that you could bring to the forefront or make that the lead… that’s the hook to get them where you want them to go… Use the things that they care about as a way to get there. we’ve talked in the past from some of the reading groups and things that we’ve done on our campus about the big questions that you can surround your class around. That’s one of the strategies that might help you get more student focused. That’s a strategy that you could use.

So, you were saying that some of the things that we need to focus on would be misperceptions and that sort of thing, but I would expand that… and this is where most academics get nervous. I know Allison does… [LAUGHTER] ..is that we start getting into the severe qualitative space.

Allison: Yes.

Rebecca:…and feelings… [LAUGHTER]

Allison: Yes.

Rebecca:We know that fear, though, prevents learning. There’s certainly evidence about that. Understanding fears could be really important… understanding aspirations… understanding what their experience is like… which are all things that don’t have hard facts necessarily associated with them. They’re more squishy.

Allison: There are also things that change from class to class.

Rebecca:It’s true.

Allison: …it’s part of what makes me very concerned about trying to implement this as part of my syllabus design.

Rebecca:Yeah.

Allison: …because what works really well for a class may not for the next one and it strikes me as difficult to tell until you are three, four, or five weeks into the class that it’s gone awry.

Rebecca:Yeah. I think that you want to use design thinking on individual small things and, if you’re a design thinker, you’re flexible… in that you can recalibrate… “Oh, this is off course,” that’s the iterative part… that you’re not married to some solution. It’s not so precious… If you’re thinking about developing these things, part of what you want to focus on is the idea that: a) it’s not precious… the learning is precious… but the way that we get the learning done isn’t. That sometimes helps… just remembering that… but it doesn’t have to be perfect all the time.

Allison: Right. Sure.

Rebecca:So, I would focus on… there’s some things that are gonna work, in general, most of the time… and those are probably things to say like “Okay, I’m close enough on that” but then there’s always gonna be the one thing… it hasn’t been working for awhile… and those are the things that I would focus on using a design thinking method to get you outside of standard solutions so that you might actually find something that works…

John: It sounds like you’re also suggesting maintaining flexibility so that if you’re trying an approach and it’s not working, go back to the drawing board and redesign it. I seem to remember hearing something about a case where someone was doing something in a class and it wasn’t working that well and they brought in the three little pigs.

Rebecca:Yeah, I think something like that might have happened… [LAUGHTER]

John: …and that was in an earlier episode of our podcast… you discussed that… and that seems like a really good example of this, where the approach that you thought would work based on past experience and so forth just wasn’t working that well. So, you changed it to something that did work better with that group of students. Is that correct?

Rebecca:Yeah, as you teach over time, and if you’re teaching the same kinds of things over time, what you end up with is a repertoire of things that you can use or a repertoire of assignments or experiences or modules or whatever that you can mix and match as your student population changes… and one thing that I struggle with is the mix of my students changes very drastically between semesters…

Allison: Right.

Rebecca:…and that’s actually why I have to mix things up. If I have all majors one semester but then it’s like a hodgepodge…

Allison: Sure.

Rebecca:…another semester, then you really have to approach things differently…

Allison: Yeah.

Rebecca:…or it’s just not gonna work. As you develop these tools for one population you don’t abandon them forever if it didn’t work this time around, but that might be the thing that you bring back in another time… recognizing like “I see these patterns again…” I think, over time, then you end up with that repertoire, so it’s not a big workload issue. You can’t think that you’re gonna solve every problem… every semester… all the time. That’s not a workload related thing… but over time, you have the ability to solve problems on the fly much easier.

Allison: Are there particular resources you would recommend that faculty go to if they’re trying to figure out how to start doing this for a class?

Rebecca:There’s a few colleges that have really embraced the idea of design thinking for populations of students outside of design. One of them is the “D” school which is at Stanford, which is kind of a hodgepodge between design and business, I think… kind of an interesting strategy based program, but they have a virtual crash course in design thinking online. There’s a scenario… and how to facilitate… in a playbook for how to facilitate it. You could go through an exercise like that. It’s all free, It’s a creative commons license… to just figure out how to design think before you start trying to apply it to your own context. That would be one way of doing it. There’s also IDEO which is a design company who’s best known for design thinking and working with pretty major brands doing pretty innovative things…. and some of the founder names are in a lot of the literature on design thinking. One of them is Tim Brown and the other is David Kelly. David Kelly is the founder of the d.school as well… but both of them are founders of IDEO. They have recently set up online classes in design thinking and design thinking for leadership and creativity, and what have you… and they have a wide repertoire of them. That would be another resource. You have to pay for those courses, but those are the absolute experts in design thinking. You know it’s a design thinking workshop when you see a lot of post-it notes.

John: ..and are there any books or other references that might be useful in addition to these?

Rebecca:Yeah. Those same two people have a couple of books that might be worth checking out. One is by Tim Brown called Change by Design and David Kelly and his brother Tom wrote a book called Creative Confidence that is also pretty good. That has a lot of design thinking and creative thinking for leadership in it. Both of those books provide a good frame for how to use some of these methodologies in context outside of design. The key thing to remember about design thinking, and this goes back to one of your earlier questions Allison, is that it’s not linear… [LAUGHTER] It’s a completely nonlinear super messy process… and so it makes people from certain disciplines really anxious. There’s not like a beginning, middle, and end. It’s a spiral that gets mixed and turned over and over again. it’s important to remember that. There is some science involved, because we certainly want to be using evidence-based practices and things as we’re coming up with solutions… but there’s a little bit of intuition based on experience that comes through. There’s a little bit of emotion that’s there… and really thinking about it holistically rather than just from one perspective is really key.

Allison: How have students responded to units that you designed using this? or do students know? Is this one of those processes that ideally students don’t see? or is it a process that, particularly for your field, ideally students do see?

Rebecca:I’ve never pointed it out. It’s an interesting question. I tend to point it out more when I’m working with teachers… because I just can’t help myself but explain my process comes from my discipline. That’s why I do what I do, but it’s related to these other design practices that are related to curriculum. It just brings this other piece in. In general, I can tell the ones that I’ve spent time designing versus things that I haven’t as much. Those work better and what John was referring to before when I stopped everything because things weren’t working and I recalibrated and really thought about the students and where they were at… that worked fantastic. I’m really hoping… I guess we’ll know after this little accessibility experiment that I’m doing right now whether or not using the design thinking method is gonna solve this particular problem… but I think the key is when it doesn’t, that’s okay… because it’s not precious and I can iterate and learn from what I tried… and that didn’t work… so why didn’t [it] work and then try something else. I think ultimately it does end up working, but it might take a couple tries and that’s in part because you can’t tell the future… and you don’t know the future based on the past… but it’s really focusing on the present… This is what I can observe… This is what I know about the students… This is what they can tell me… and I can only really make decisions based on that…

John: How do you assess how well it’s working at any given time?

Rebecca:The same way that we assess student learning… my student learning outcomes are better than whatever I did must have worked or I don’t really care it worked… if they’re doing well, let’s keep it. [LAUGHTER]…make a thumbs-up….

John: What I meant to ask is do you monitor this when you try something new as it’s going to see how it’s going or do you wait until you see the final stage?

Rebecca:I’m watching and observing during the process to see whether or not students are catching on and I will intervene if it’s not working… if I’ve tried something and it’s like “ah yeah. I can foresee this crashing and burning in the next couple of classes…” or whatever then I’ll circle back and do the iteration before the end of the semester sometimes it’s between semesters that I do iterations and sometimes I just recalibrate in the middle of a semester on something and make some minor tweaks to something so it works better for the students.

Allison: I think the thing that’s interesting there is I suspect that most faculty would say “oh yeah, I can feel when something’s going off in my class” and I think many of us at least try to stop and say “okay, what do I need to do to fix this?” What seems different is what question are you asking. What are the series of questions you ask when it’s time to say “Oh, this has gone off the rails in some way…” and I think it can be easy to say “with this group of students it’s gone off the rails” without actually thinking about “Is there something different about that group of students that’s why it’s gone off the rails?” Which seems like the insight that the design thinking may really provide. For a lot of faculty would already say but I do design my classes really carefully… or I have lots of things that I run that are different from class to class that it’s really about where do you start….

Rebecca:Yeah.

Allison: …the questions.

Rebecca:Yeah.I think you’re right. The series of questions that I asked might be quite different. I still start with those same goals, but I start with all those questions about students and then in the middle of the semester I revisit those questions often. Did I make a good choice about that? I have these non-design students in my class, do they seem like they’re getting the design piece or not? If they’re not, then I did something wrong. I need to fix that because that’s not okay. I certainly do that and sometimes I just ask them: “What do you need? This is not working. I can see it’s not working. I’m sure you can feel it’s not working. Do you know what you need? Because if you know what you need then I’ll start there.” …and I think it’s a willingness sometimes to be willing to have a conversation… and it does have to be with all students… but if you have a couple students who maybe are a little more forthright, or you have a good relationship with, I’ve done that… “What do you think is going on here?”

Allison: Yeah, and I think to me that gets back to the question I asked earlier, which is “Do you make it transparent to your students the same way that I think we’ve talked about in other contexts… that for assignment sheets, you want to say “The purpose of this assignment is to get you from point A to point B and this is what I’m trying to do is to actually say “Hey, we can all feel that this went wrong” beyond just coming in and saying “Hey, we can all feel that this isn’t going great, here’s what we’re gonna do…” instead it’s “Hey, we can all feel that this isn’t going great. I’d really like to hear from your perspective what’s going wrong, but also what do you need for me so that I can make sure to adjust in a particular way.”

Rebecca:Yes, I make that part transparent.

Allison: OK.

Rebecca:If things go wrong I certainly…

Allison: Yeah.

Rebecca:…whatever. [LAUGHTER] I’m an open book. I make mistakes. I’m human… but I don’t always hold to the forefront that I’m using a process that’s user centered or audience centered.

Allison: OK.

Rebecca:That might be obvious when I asked them for their perception…

Allison: …what they’re looking for. Yeah.

Rebecca: I just don’t name it.

John: …and that’s what I was thinking about when I was asking that. That I would think that getting some feedback from them, if you’re going to focus on the needs of the students it would be really important to make sure that you’re meeting those needs as you move through.

Rebecca:Yeah, and I always start my classes getting to know students. I do an exercise the first day of class that’s called “hopes and fears” that brings out some things that I may or may not be aware of… Often I am… there’s certain things that bubble up every time, but every once in a while there’s something there that I wasn’t expecting to be there… and then you can kind of ask about it and get a feel for it and that’s right at the beginning of this semester. “Oh, okay something new’s here, I should be aware of that.

Allison: That’s one we’ve talked about before… that I am actively planning to steal from many of my classes.

Rebecca:It works really well.

John: Now, that approach can work really well in a smaller class of the sizes that you normally teach. How might that scale to larger scale classes?

Rebecca:I think that that can work. Probably the amount of flexibility you can have in the middle of this semester might not be there. You can make some shifts, or whatever, but there’s a lot more students to deal with, so you’re not quite as mobile or nimble. There’s that. You may have to do more of the iteration between semesters rather than within this semester. If you’re steering a big ship, you can’t make a drastic turn. You can kind of steer it in a slightly different direction and make minor corrections, but I don’t think you could do a major correction in the same way that I could do in a smaller class. I just don’t think it’s that feasible or advisable. [LAUGHTER]

John: But, you could still try to get feedback from the students on how it’s working… what’s working well… what’s not working well… It could even be just from surveys even. Because it might be harder to get that small group feedback result.

Rebecca:Yeah, and I recommend doing that every semester… getting a feel for that… something that’s separate from standard course evaluations… not at the same time as course evaluations… not at the end of the semester when students are stressed out… at different points in this semester, you can get that feedback and there’s a wide variety of ways that you can collect that information.

John: …and responding to that could be useful too… letting students know that you do hear their voice and that you are responding and making adjustments where you can, or at least being transparent, and letting them know why perhaps some of the things that they think might help might not work as well if it’s not consistent with evidence-based teaching or something similar.

Rebecca:Yeah, and I think you can also say “Well, thanks for your feedback. I can’t make all the adjustments I’d like to this semester, but I’m gonna use it for next semester just like I used the previous semester’s to make this class better for you.” I think if you indicate that then students are more interested or invested in giving you useful feedback.

John: They know their voices are being heard and their needs are being addressed, even if it’s not going to immediately benefit them.

Rebecca:Yeah, yeah… and depending on how big your major is maybe I can’t adjust it in this class, but you might have another class with me, and I might take this information under advisement for that other class that you might end up being in.

John: One of the things we did with our reading groups is we had faculty from many different disciplines getting together and talk about problems they experienced in their courses and then we had people from different disciplines respond with techniques that they found useful. Would that be a good way of trying to encourage faculty, perhaps, to do more design thinking? by talking with colleagues from other disciplines as well as their own?

Rebecca:Yeah. I think one thing that you could do in a setting like that is to remind faculty to investigate who there audiences… to gain empathy for their audience… really their students… but then to take advantage of those opportunities to interact with other faculty, ideally faculty not from your own discipline because different disciplines think different… and use those group opportunities for that ideation piece… because the solutions that come to you most naturally are aligned with your own pattern of thinking… but if I have a conversation with Allison, who’s in political science and I’m in design… something she does in her class may not directly apply to what I’m doing, but all of a sudden it gets me to think differently about what I’m doing and the students that I have… or maybe there’s similar issues that students are struggling with. I find all of those interactions with other faculty to be most valuable for that ideation. I can’t ideate in a room by myself… Really all that ideation, even if I’m not sitting with sticky notes and brainstorming or doing a specific brainstorming activity, those interactions with other faculty feed my ability to come up with ideas that I wouldn’t have had otherwise.

Allison: I think they also tend to feed empathy. It’s that sort of experience of sitting in rooms with other faculty and hearing them throw off language… that it gives you that moment that you don’t have as much anymore, maybe… that you had an undergrad when you were suddenly in a class and were like “Oh, I don’t know what this is about… at all” and it’s clearly just the base vocabulary of somebody else’s discipline and sort of be the person who has to say “This is super interesting, but I don’t know where we are in this conversation right now…” can, I think at least for me, often help with that feeling of “Oh, this is what the students that take my intro to political science gen ed class… that’s what they’re feeling when they’re sitting in the classroom…” and I think that’s… in terms of just being in a headspace to think empathetically with our students… can be very helpful.

Rebecca:Yeah, and I think that happened a lot when we were doing our syllabus workshop this past spring. There were faculty from a wide range of disciplines that were in the room. I think there was only overlap between two faculty. When we were having conversations or were talking about different pieces of the workshop would be like “Wait a second, I thought it was like this…” We have our own misperceptions of each other’s disciplines and it came out in those conversations.

Allison: Right, or people would raise “This is the prompt I would use in my poetry class and we all had a “Oh, we don’t think that means what you think it means.”

Rebecca:Yeah.

Allison: At all. I thought that was a very valuable part of doing that workshop.

Rebecca: Yeah, definitely… and that’s the ideation piece. You got to get yourself out of thinking in the way that you normally think, and that’s where you come up with the innovative ideas.

John: Well, at this point, we normally ask our guests what are you going to do next? So, Rebecca, what are you going to do next?

Rebecca:Vacation…. I’m going on vacation.

John: Where are you going?

Rebecca:I’m going to Iceland.

John: Excellent.

Rebecca:I just need a break. I’m gonna come back and work on the accessibility stuff that I had started. I have a grant actually to support that work through Teach Access.

John: Congratulations! I saw that you had tweeted that.

Rebecca:Thanks. That funding will help me do some of the things that I’ve wanted to be able to do for a long time, which involves inviting the disability community into helping me develop some of the exercises that I do with my students.

Allison: Great.

John: Okay, well thank you for serving as a guest, and I guess I’ll see you when you get back from Iceland and I get back from North Carolina.

Rebecca:Yeah, which is about the same day.
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca:You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

39. Video feedback

Have you spent hours writing comments on student papers only to see them end up in the trash can as student file out of class? In this episode, Dr. Jessica Kruger, a clinical assistant professor in the Department of Community Health and Health Behavior at the University at Buffalo joins us to explore how providing video feedback may help motivate students to hear, see, use, and understand your feedback.

Transcript

John: Have you spent hours writing comments on student papers only to see them end up in the trash can as student file out of class? In this episode, we explore how providing video feedback may help motivate students to hear, see, use, and understand your feedback.
[Music]
John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
Rebecca: This podcast series is hosted by John Kane, an economist…John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today our guest is Jessica Kruger, a clinical assistant professor in the Department of Community Health and Health Behavior at the University at Buffalo. Welcome, Jessica.

Jessica: Thanks, glad to be here.

John: Glad to have you here. Today our teas are:

Rebecca: Jessica are you drinking tea?

Jessica: I am drinking tea.

Rebecca: Yes! Score one for us. [LAUGHTER]

Jessica: Today, I chose actually a chamomile tea from Yogi, and the Yogi tea proverb today is “not sharing is not caring.”

Rebecca: Sounds perfect. I’m drinking chai today.

John: …and I have blueberry green tea.

Rebecca: So, Jessica, can you tell us a little bit about the types of courses you teach?

Jessica: Sure. I teach in a brand new undergraduate public health program, and the courses that I currently teach: introduction to public health, which spans the gamut of information… anything can be public health… and we talked about it all in that course. I also teach a class called Social and Behavioral Aspects of Health and Methods and Mechanisms in Public Health.

Rebecca: What are your class sizes? or the kinds of things that you do in classes? What do they look like?

Jessica: My smallest class is 75 students and my largest is about 250. This upcoming semester it may be over 300. My class, when you walk into it, you’re going to hear music playing at the very beginning (probably pretty loud) to start the class. After the music stops, we get into the material, and if the students can guess it, the song usually actually relates to the topic that we’re covering, or something that’s going on in the world. I think my most apparent connection between music and academics and the world was, we had an ice storm here in Buffalo and, of course, the song was, you guessed it, Ice Ice Baby. [LAUGHTER]

John: What is the purpose of the music to start up the class.

Jessica: I like to get people comfortable and it usually fits with my personality… pretty fun… pretty open… pretty outgoing… and so when they come into class they should sit there, get acclimated to the environment, and then get ready to go for the day. Once the music stops, they know to stop their chatter, put away their phone, and get down to business.

John: Do you talk about how it relates to what you’re going to be doing or is that something that just flows naturally?

Jessica: Sometimes it flows naturally and other times the students will say “So, why did you play that song?” and it takes them a moment and I was like “didn’t you hear the reference to cigarette smoking in it? Obviously, we’re talking about tobacco today.” [LAUGHTER]

Rebecca: One of the reasons why we invited you to talk to us today is about how you give students feedback. Can you talk about the kinds of assignments you give students and the kinds of feedback that you want to be able to give them?

Jessica: Yeah. In my class of 75, the small one, we get to know each other very well. But, in that small class, we actually write a paper… and this paper isn’t just a 15 to 20 page paper. They do it in sections, so the students never know in the beginning… and I actually never give them a page number… of the paper that they’re going to write… and it’s all about the social and behavioral aspects of a specific health topic. For example, some students write about maternal mortality… infant mortality… tobacco use… and they have to describe the public health issue, but also what behavioral,or what social parts of the puzzle, fit into this health issue. They’re writing this paper in small chunks. They’re turning in about a page every other week, and with that they get written feedback on some of their papers. The typical written feedback where you say “Oh, you know, you need to cite this. You need to add an apostrophe here” or “Hey, could you explain this further?” But what I also do is, twice during the semester, as they’re putting their papers together to build it into a larger document, I provide them with video feedback… and this video feedback is something new. I tend to make videos to describe things for friends, families… and I said “why not just make videos for my students?” They can see my emotion… my face… how tired I am after reading all 75 papers, and they can also see the excitement when I say “Oh my goodness, that’s such a good point. I’m so glad you made that.” They can also hear, in my voice, how I’m feeling towards their paper or what’s going on, and I can even show them “This is actually how you do a citation” because they can see my screen.

Rebecca: That sounds really great. I can imagine that modelling behaviors or modelling how you think through writing could be really useful for students.

Jessica: It seems to be effective.

Rebecca: Can you talk a little bit about how it works, what your process is, in terms of how you’re recording it?

Jessica: Sure. Students turn in their paper via Blackboard and when I go in to grade the papers, I pull it up and I also pull up a program called Panopto. Panopto is a screen capture software similar to Screencast-o-Matic and it allows me to take a screenshot or a screen grab of my computer screen, and also you see a little picture of me in the corner. I enable that just so students know that “Hey, I’m there.” In a class of 75, you may not get that personalized attention in class, but you are getting a video made directly by the professor with your paper.

After I pull up the paper, I start going through it. I usually scroll through it once or twice to point out and start thinking and distilling my thoughts about it, and then I go as if I was just reading the paper. I start going through and saying “Oh, you know, you need a transition sentence here” and highlighting it or highlighting an area where I say “I think you need another citation here” or “That’s a really great point, I’m really glad you added that. Could you also add some more?” and so this allows students to: 1. visually see their paper and any sort of modifications they need to make, but also to see that I’m actually reading their paper and giving them some hopefully quality feedback that they can help improve upon.

John: There’s a lot of research out there where people are more likely to misperceive a negative tone or read a negative tone into written text, and they’re much less likely to do that when they actually see facial expressions and so forth. Have students taken the feedback a bit more appropriately?

Jessica: I actually think that’s a good point because I assess this and one of my students comments really struck me. They said: “I like that I can watch you read my paper and see where I went wrong. I feel videos helped increase the trust with the student and the professor. Sometimes written feedback makes you feel attacked or that you feel the professor’s just being unfair and biased towards your writing.” So, I feel that this does help break down some of those walls, and they can see the emotions, but also realize I’m also using a rubric to grade. It helps them better understand the rubric and how I’m figuring out their score in a fair way.

John: So, do you grade the whole paper with the video feedback? How long are these videos?

Jessica: So, because they’ve turned their paper in in multiple chunks in my class, I usually know what I’m looking for in each of their papers. I remember their topics. I know their writing styles, and so my videos for some of the students who are very good writers only last maybe about one to two minutes. For those who may need more correction and more additional support, I’ve had videos up to 10 minutes long. But the average is about 3 to 5 minutes.

Rebecca: Do you pre-read and then record or are you doing the whole process being recorded?

Jessica: I slightly pre-read, just so I’m not caught too off-guard. But, sometimes some of the most real and genuine comments come from when you see something and you’re like “Wow, I can’t believe they actually picked that up from lecture! That was awesome.” That point really struck a chord with them and they put that into their paper.

Rebecca: I really like that what you’re describing is annotating their papers but doing it live. Not only are you providing the feedback, but you’re also providing a model for having a conversation with a piece of writing, which is probably how we want students to read… which i think is really cool.

Jessica: I would agree. I never thought I would be doing this. It mostly came out of the fact that I have pretty terrible handwriting and my spelling is not always the best. So, instead of sitting down and typing out comments, which I found took me actually a lot longer than video grading, I feel like this adds that connection aspect, models some behavior, and also give some information that they can use versus my chicken scratch.

John: How do you share the information back to the students?

Jessica: I’m able, with Panopto, to get a live link and that link can actually be emailed to the student directly through the program. The student gets an email in their inbox saying: “Hello, there’s a video that’s being shared from Dr. Kruger. Please click on this link to watch it.” I also embed the link into our Blackboard shell, so that they can go back and look at it at a later time.

John: That’s a very convenient way of doing it. It’s much easier than having to render the videos and then send them out, and that is one of the nice features of Panopto. We use it at Oswego as well, but I don’t know of anyone who’s using it right now for this purpose, but it sounds like a great idea.

Rebecca: You’ve already talked about some of the advantages and disadvantages, but can you suss out a couple more details related to that?

Jessica: Most definitely. Because this was new to me and actually it’s quite new to the literature. Video grading compared to written feedback and audio feedback actually hasn’t been studied that much, and if it has been studied it’s been done in fairly small classes… not as small as mine, even smaller classes, about 14…16 students… and so I wanted to actually compare the students perception of the written feedback that I give them within this class and the video feedback that they’re receiving. So, they received written feedback about eight times in the class. They received video feedback about twice because their paper was assessed about ten times through the semester… and so at the end of the class I gave them a survey and I was looking at the quality that they felt (if it was better than written, about the same as written feedback, or worse than written feedback). their ability to understand information, the helpfulness of the information they were given, the accessibility… so, were they able actually get the information through the link that I provided… and also the ease for them to make changes within their paper. Overall, almost all of the students felt that this was better than written or about the same as written feedback. For example, 75% of the students that responded to the survey said that the quality was better than written feedback, and a quote that supports this from a student is that they said “Video seems to give a better chance to explain what you want fixed faster to say then write all the comments. So, I noticed you tend to include more information…” which was a neat observation of the student, even though they probably got their paper back covered in purple pen and stickers. But, they still felt there was more information from that video.

Rebecca: I wonder if some of the information that they’re perceiving is the facial expressions and stuff, so it’s not actually the commentary but how to interpret the commentary.

Jessica: Most definitely, and actually I didn’t tell the student that that’s a little face in the corner. I didn’t want to point it out, but actually most of the students noticed it, and the most common comments I’m seeing with all this information is “This information is really clear, but it also makes me feel really connected to my instructor.” One student even says “University of Buffalo makes me feel small, but the one-on-one interaction is really appreciated and makes me feel more personal and more connected to the instructor.”

John: So, it sounds like it benefits you by giving you a little bit less work, perhaps, and the students. Are there any problems in terms of students getting the feedback? Have they voiced any concerns? or are they as likely to read the feedback as when you give it back to them on paper?

Jessica: I wish I could sit down with all of them and say “So, tell me the truth, did you actually go through this?” What I’ve found is actually, within the students that I surveyed, about 67 undergraduates out of the 75 completed the survey, and 85 percent of them had actually watched the video feedback, which i think is pretty good. I’ll take 85 percent any day. With the papers, they continually got them back. So, I’m not sure if they actually read them. I hope… because they needed to incorporate those changes in their final paper in order to improve their grades. But that’s one thing with paper we actually don’t know. With Panopto, I can see if they’ve actually clicked on the video to watch it. Some students actually watched it multiple times, which I found to be interesting. They went back to review before they turn their paper in again to make sure that they made all those changes.

It’s actually neat. When I sent out links I would see the views spike up and then when their papers were due again I would also see the views spike. [LAUGHTER]

Rebecca: That seems like a good trend to have seen, right? Did you have any other interesting findings?

Jessica: So, with this very exploratory research in this, with a larger group of students, I was wondering about the users making change. When I’m making changes when I get comments from a reviewer. they’re usually on paper, right? Reviewer 2, you probably wouldn’t want video feedback from them. But, I thought it might be more challenging for them, because they’re watching a video and watching their paper, versus having a piece of paper next to them making the changes, but 81% of the students said it was better than the written feedback. One student said “It was much easier to understand what exactly you wanted fixed. With written feedback you have to first understand their handwriting and then understand exactly what went wrong, which is often hard.” So, I think that the voice and the emotions and the pictures helped them synthesize that information faster and easier than maybe that written feedback. I did have another student said something surprising. Although they knew I was going to do this in class, I had a student say “I got this email link about a video. I didn’t know that that was real, so I didn’t click on it. I didn’t watch it at all.” [LAUGHTER]

Rebecca: Sounds like a good excuse.

Jessica: Yes, exactly.

Rebecca: But that does bring up some issues related to accessibility and access. So, what do you do for students who maybe video isn’t the best method for them? Maybe they need a transcript or they need captions or something? Do you have a different method that you use for those students? or how would you approach that, if a student needed something like that?

Jessica: I always allow students to meet in person and discuss their paper so if the video option… if they need transcripts or other accessibility option, Panopto does have a built-in feature that may or may not be enabled for people’s campuses. Our captioning will be enabled soon, but with that does come a barrier of accessibility. So, I always give students the option of meeting with me. You can also upload the video to YouTube and create your own transcripts, which also have some challenges…. and there is actually, in the offices of accessibility, the ability to send them a video and they can send you back transcript as needed. But, in this case, if a student needs any sort of accommodations, I’m happy to meet them where they are and figure out a different way for that student besides the video feedback.

Rebecca: I think it’s always just important that as we’re thinking about these new methods that we’re not losing track of the fact that electronic documents may or may not be accessible naturally unless we take some extra steps. So, I think it’s always good when we let students know upfront that like “Hey, we’re trying this thing out, but if you need something else, let me know.” So, I’m glad that you’re doing that, but also that you’re making people aware of some of the tools that are available as well. This is such a great technique.

You said you were working on a paper, right?

Jessica: Yes, that’s where I was pulling the stats up. I’m like “Okay, I gotta pull that up.” [LAUGHTER]

Rebecca: I look forward to reading that when that’s out.

Jessica: Yeah, I have to figure out what journal I’m gonna send it to, which is always a fun thing to figure out, right?

John: One thing that I think might be worth mentioning is that for people who are using Blackboard who have really bad handwriting, as I do myself, we used to have that Crocodoc option to mark up text. It’s probably more work than video feedback, but at least you could embed comments and notes and other things. Now that Blackboard has lost that and they’ve replaced it with a New Box View, providing comments and providing feedback in Blackboard is so much more difficult, cumbersome, and so much less flexible than it was even with earlier releases. I think video feedback is a really good way of doing it, and Panopto is a really nice tool because it’s all stored on the server, you don’t have to store their videos, you don’t have to somehow transmit them to the students, or give them links to things in your own cloud services. So, it just seems like a really good approach.

Jessica: I think there’s a lot of utility for this for people who are teaching online. That’s how you can make that connection with the students. They can see who you are, if you’re comfortable with that, and also get to know them a little bit. I have colleagues who use tools such as Flipgrid, and one of their first assignments is: “Show me where you study” to kind of get to know each other. But, this is getting to know each other on an academic level, and knowing that I’m just not some person sitting behind a desk with a pen marking up your paper to bring you down… I’m here to help you learn and to help you grow as a writer and a future public health professional.

John: I used Voicethread in my online class last semester and it provided a similar experience. It got to the point where, and students commented on this too… some discussions were done in Voicethread and others were done in text… and when I was reading and some of the students noted that when they were reading text comments and text responses from students they could then hear the voice of the students. They could match the people to voices and it gave it a much deeper sense of connection then I had experienced before… because I had just used primarily text discussion forums.

Rebecca: I also want to note… 75 students in doing a writing assignment… like wow… that’s crazy. [LAUGHTER]

Jessica: Yeah. I think it’s important to help meet our learning objective for the course overall… and some would say I’m a glutton for punishment. But the students actually don’t complain about the paper as much as they complain to me about other things. Because they really don’t know that it’s going to be this long paper and grading it in these digestible chunks is actually much easier for me and the students. So, don’t tell my students next semester, but they’re still going to have to write the paper… and I still not to a page count, but the average page count for last semester’s class was just 15 to 20 pages at the end.

John: That’s a lot of grading.

Rebecca: It’s pretty incredible. How long does it take you to do each chunk? I mean you must have a down to a science because you’ve been doing it for a while.

Jessica: With the written feedback it takes me a few minutes per paper because they have a pretty solid outline that they follow and students have similar topics and I encourage them… I’m like “Work with a friend if you’re both writing about tobacco in Pacific population your background is going to be similar. If you both have the same source, be sure that you’re not plagiarizing.” But, you can work together and still have very separate papers… and so part of it becomes very repetitive at the beginning of the paper discussing a background on a public health topic… which is pretty straightforward, but where they get creative is actually where they get to health design and intervention that they think will work to help solve their public health problem.

Rebecca: At least it sounds like they get interesting.

John: It sounds like you scaffold it very nicely. How many times do you provide them with feedback over the course of the semester?

Jessica: About 10 times. Next semester, I’m going to do it a little bit less because it’s a lot of work on my part, but the smaller the chunks to grade, the faster it is to grade them… and the sooner I can realize that they might not understand something then we can cover that in more depth during the class. So, I use this project as a way to help them improve writing, but also it’s a nice check of “Oh, they actually don’t understand that concept so well” or “They’re applying it in a way that really isn’t the best, so let’s go back… let’s talk a little bit more about it and we can all be on the same page…” and I really, really like the way that it’s broken down and I don’t think I would ever wait til the end of the semester for one giant paper. Ever. [LAUGHTER] Because it causes a lot of anxiety for the students and it takes a long time to read such a long paper if you haven’t been working on it in chunks. I think my favorite thing that I hear from students is “You know, I never thought I would write like this” or “I didn’t like the idea that we had to turn in small pieces of paper but, you know what? I’m done with your paper in my class and I was procrastinating all the other papers. So, I’ve learned a new technique in writing and reducing procrastination.”

Rebecca: That must make it so that is a lot better to read at the end too.

Jessica: Oh, most definitely. If they make minor mistakes throughout it, they can fix it for when they turn it in for more credit, so that they’re not losing out on points.

Rebecca: So, we generally wrap up by asking what’s next? So, what’s next for you, Jessica?

Jessica: So, what’s next is I get to have a fun summer but coming back to the school year, I think I’m going to be writing a textbook with my students. I’m very much into the ope educational resource movement and so very passionate about it. Public health is a little bit behind other disciplines in adopting open educational resources… and I think one way to get students excited about this, but possibly other faculty, is to write a textbook… and like all of my classes… they’re small. [LAUGHTER] So, I’m going to attempt to do this with 75 students next semester. I’m either crazy, delusional, or ahead of the curve. I’ll let you know, hopefully on another podcast.

Rebecca: Sounds great. Sounds like it’s quite the adventure that you have planned for yourself for the fall.

Jessica: Life is always an adventure. [LAUGHTER]

I’ll let you guys in on a secret. I’m going to take a hundred students on a field trip this next semester to do some experiential learning… which will be really fun… and also terrifying to take that many students on a field trip. but I think experiential learning is so important in any academic field… and so, I’m this huge advocate and always trying to integrate it in my classes. One thing that I do is I’m highly involved in student-run free medical clinics and have been my whole career. As a student, I had to start my own medical clinics and now I want you just to suffer and do it like that. [LAUGHTER] I want them to have the same opportunities that I had and learn about the different disciplines and working together and helping out the community overall. So, each week I take undergraduate students with me to a free medical clinic and their job is to screen and advocate for patients and it’s so great to see how passionate the students are about helping these folks that they’ve never met… and also how invested they can get. The other day, when we went to the clinic, one patient needed eyeglasses and we actually didn’t know of any free places to get eye exams… and so the student took the initiative… got on their phone… called probably 10 eye exam places and then made it his mission to actually create us a document and call corporate offices to see if we could get eye exams for these patients that are in need. So, it really allows them to take charge… take off the training wheels… and become an advocate and use those public health muscles for good.

Rebecca: Sounds like a lot of rewarding days.

Jessica: Very much so.

John: That sounds like a lot of busy and long days as well. [LAUGHTER]

Jessica: I think the most common question I get is “Do you sleep?” and I do… eight hours a night. Sleep is very important for overall health. [LAUGHTER]

John: …and, in your field, that’s something especially to focus on.

Rebecca: Well, thanks so much for your time, Jessica. It’s been really interesting to hear what you’re up to… and I think you’ve offered a lot of fruit for thought for most of us.

John: Yes, thank you very much.

Jessica: Thanks so much. Happy to be here.

[MUSIC]
John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

38. Reflective practice

Now that we have been on summer vacation for a while, we thought it would be useful to take a break from our usual interview format to reflect on the previous semester and our plans for the fall. We also provide some recommendations on summer reading related to professional development.

Show Notes

  • Sue, D. W. (2016). Race talk and the conspiracy of silence: Understanding and facilitating difficult dialogues on race. John Wiley & Sons.
  • Guffey, E. (2017). Designing Disability: Symbols, Space, and Society. Bloomsbury Publishing.
  • Evans, N. J., Broido, E. M., Brown, K. R., & Wilke, A. K. (2017). Disability in higher education: A social justice approach. John Wiley & Sons.
  • Hogan, Lara (2016). Demystifying Public Speaking. A Book Apart (https://abookapart.com/products/demystifying-public-speaking)
  • Hoffman, Kevin H. (2018). Meeting Design: For Managers, Makers and Everyone.  Rosenfeld Media.
  • Openpedagogy.org
  • Schwartz, D. L., Tsang, J. M., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. WW Norton & Company.
  • Cavanagh, S. R. (2016). The Spark of Learning: Energizing the College Classroom with the Science of Emotion. West Virginia University Press.
  • McGuire, S. Y. (2015). Teach students how to learn: strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Stylus Publishing, LLC.
  • Parkes, J., & Zimmaro, D. (2016). Learning and assessing with multiple-choice questions in college classrooms. Routledge.
  • Lewis, M. (2016). The undoing project: A friendship that changed our minds. WW Norton & Company.The Undoing Project – Michael Lewis
  • Tea for Teaching podcast: 15. Civic Engagement – a discussion with Allison Rank about the Vote Oswego project.
  • DeRosa, Robin (2017). “OER Bigger than Affordability” Inside Higher Ed. November 1.
  • Tea for Teaching podcast: 30. Adaptive Learning
  • Videoscribe
  • Flipgrid
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Learning How to Learn MOOC
  • Oakley, B. A. (2014). A mind for numbers: How to excel at math and science (even if you flunked algebra). TarcherPerigree.
  • Oakley, B. (2017). Mindshift: Break through obstacles to learning and discover your hidden potential. Penguin.
  • Oakley, B. (2018). Learning How to Learn: How to Succeed in School without Spending all your Time Studying; a Guide for Kids and Teens. Penguin.
  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc..
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Teaching in Higher Ed – Bonni Stachoviak
  • Teach Better – Doug McKee and Edward O’Neill
  • Email addresses: john.kane@oswego.edu and rebecca.mushtare@oswego.edu

John: Now that we have been on summer vacation for a while, we thought it would be useful to take a break from our usual interview format to reflect on the previous semester and our plans for the fall.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Today our teas are:

Rebecca: …a mix of seven different kinds of tea, and it’s not really describable at this point.

John: After I’ve had many different types of tea today, I have Twinings’ Wild Berries herbal tea.

Rebecca: Finally dropping the caffeine after a long day?

John: …after many teas earlier in the day, yes.

Rebecca: So, I start my reflective practice while grading during finals week and for me it’s a really effective and productive procrastination technique. As I’m reading assignments or looking at projects and making notes about things that clearly did not work or “Wow, I really should cover these skills better” or “This really worked…” and I have a running dialogue with myself while I’m grading them and I use that for planning for the fall. What are your practices like, John?

John: I’d like to do that a bit during grading week but during grading week I’m generally busy working on the workshop schedule for our workshops here…

Rebecca: What?

John: … and also working on plans for various presentations at the SUNY Conference on Instructional Technology and so forth… and then getting ready for my trip down to North Carolina for the summer. So, I try to do it as I’m going during the semester so that I keep in my blackboard folder for each course a hidden folder where I list any problems… and I’ll do that for the course overall, as well as within individual modules. That way, when I go to refresh the course in the future I’ll have a list of things in general I want to do differently as well as specific recommendations in specific components of the course.

Rebecca: Have you ever accidentally made one of those hidden files not hidden?

John: I have not, no. [LAUGHTER]. I’m much more likely to leave something hidden that the students have as an assignment, but they’re usually pretty good at reminding me of that as we go through.

Rebecca: I think my greatest fear of having notes like that would be that I would make them really public and then probably have some sort of snarky comment in my hidden files. [LAUGHTER]

John: So, we thought maybe we talked a little bit about our lists of plans and then make some general recommendations of things that we found useful. So, Rebecca would you like to go first?

John: Sure, I think both of us have a fairly aggressive reading reading dream list. I don’t know how much either of us will get through that list, but my list includes Race Talk and [the] Conspiracy of Silence by Derald Wing Sue… which jDerald Wing Sue’s coming to our campus in the fall to give a talk based on this book… and we’re gonna have a reading group again. So, I want to make sure I’m on top of that.

John: That’s also on my list. I started reading it earlier, but I got buried in the semester, so it’s on the top of my summer reading list.

Rebecca: Yeah, I read the first chapter but then that’s as far as I got. I’m also planning to read… I started reading but I didn’t have time to finish a book called Designing Disability: Symbols, Space, and Society by Elizabeth Guthrie. It’s a really interesting book about the history of the wheelchair symbol. So, it’s related to design, obviously, which is my area of teaching… but also my interest in accessibility, which I’ve been working on a lot on campus. Related to that, I also am planning to read Disability in Higher Education: a Social Justice Approach by Nancy Evans. I started reading that during this semester and read a few chapters here and there but didn’t get all the way through. It’s a pretty hefty read. So, I’m hoping to get through a lot of that this summer… and then I have two other books that are not so much teaching related but come out of the design field. One of them is Demystifying Public Speaking, by Laura Hogen, which is from a series called A Book Apart… it’s made for designers, so I’m hoping to read that book and pull out some nuggets that might be helpful for students who get a little nervous about public speaking… or see whether or not it’s a good recommendation for our advanced students in our program… and then the other one that comes from a designer is Meeting Design for Managers, Makers, and Everyone by Kevin H. Hoffman. I’ve seen Kevin speak and have had some conversations with him in the past about designing meetings, so that meetings are actually productive and useful rather than unproductive and something that could maybe have happened in an email. So, I’m looking forward to reading a fuller version of his process. What are you hoping to read, John?

John: Well, several these I’ve already started again but haven’t gotten too far but they’re enough so that they’re on my Kindle or I have the books very handy… and I plan to read them as soon as I can. One is The ABCs of How We Learn by Daniel Schwartz. I actually made it, I believe, through letter L before I had to put it down to get caught up on some other things.

Rebecca: Yeah, I remember getting some updates in the various letters and it did kind of fizzle out.

John: So, I will finish that fairly soon, I believe. The Spark of Learning is a book I’ve heard wonderful things about from Sarah Rose Cavanagh. I’m hoping to read that this summer. It’s also on my Kindle app. The Teach Students How to Learn book by Saundra Yancy McGuire and Thomas Angelo is a really good book that talks about ways of improving student metacognition. Again, I’ve read a little bit of that just to see that it is something I really want to continue with. Another thing I’d like to look at, since I teach large classes where I use a lot of multiple-choice questions, is a book that I heard about on a couple of other podcasts on teaching and learning… in particular, the Teaching in Higher Ed podcast, which is Learning and Assessing with Multiple-Choice Questions in College Classrooms by Jay Parkes and Dawn Zimmaro. That’s something I haven’t started yet, but I do have a copy of that and I’m looking forward to reading it. Another book that somewhat on the border between teaching and learning and my work in economics is The Undoing Project by Michael Lewis. It’s a book on the early development of behavioral economics by Kahneman and Tversky, and the reason why it’s on the border of economics and teaching is that behavioral economics explains why people don’t always behave as rational agents… and certainly that’s important in trying to understand how people work from an economics perspective… but when we’re dealing with students and faculty we observe that people don’t always behave, perhaps,in an optimal fashion. We don’t see people engaging in activities that are in their long-run self-interest, and they often will prefer short-run benefits over long term benefits, even though they know they’d be better off doing their work a bit earlier and so forth. So, it overlaps between those two interests. I’m looking forward to that I guess that’s it for my books.
So, what are your plans for redeveloping or redesigning some of your courses?

Rebecca: Well, I have a new class that I’ll be offering in the fall that’s related to some other special topics I’ve taught before on experience design… and in that class we’re gonna do two community projects: one is called “recollections storytelling through mementos“ which is the design of an interactive exhibit that will travel to multiple adult care facilities in central New York. It’s the second exhibition in a series. The last one we did was a couple of years ago… and so the design and development of that will happen partially through the summer and then in my class in the fall… and then the exhibit will go up and travel next year in 2019… and then the other project that we’re gonna work on is our very famous [LAUGHTER] regular guest Allison Rank, who’s talked about her project Vote Oswego. My students will be working on that project as well, doing some design work with her class. We scheduled our two classes so that they would be at the same time slot, so that they could collaborate a little bit easier this time. so I’m looking forward to working with Allison a little bit this summer to make some specific plans for that for the Fall. So, I’m doing that and then revisiting my web design courses like I do every year: a) the content generally changes because standards and things and web change but I’m also… I had my little list, as I was grading, of things that I want to make sure that I’m doing and some of that means integrating more reflective practice opportunities I think it’s really important and I always plan on doing that and then somehow it gets cut. So, I decided I really need to just actively decide to cut something else out, so that there is actually that room and that’s not what gets cut in the future.

I’m also working on some new accessibility modules and I’m also really thinking of… I’ve been doing a lot of quizzes based on our reading groups and things that we’ve been talking about for retrieval practice… but I’m really thinking about switching to trying some in-class polls even though my class is relatively small and mixing in some practical exercises and I was doing both of those kinds of things in the quizzes and I think spreading those out a little bit will actually help with engagement, and also make it so it doesn’t take up as much class time.

John: In terms of the use of polling in small classes… for the last five or six years now I’ve been using polling in classes that I teach at Duke where generally there are between sixteen and twenty students, and it works just as well in small classes as it does in large ones. In some ways it works a little bit better.

Rebecca: Yeah, I can imagine that and I know that you’ve talked about that in the past, so you’re wearing on me. [LAUGHTER]

John: It’s a good practice.

Rebecca: Yeah, how about you?

John: Well, I’ve got a number of things planned. One is, I’ve been wanting to adopt an OER for a long time, but I’ve been somewhat tied to the adaptive learning tools and so forth provided by publishers, as well as the array of materials they provide… but, I want to explore some OER options for my large introductory class.

Rebecca: For those that aren’t familiar, what’s an OER.

John: Open educational resources… basically things that are released under Creative Commons licenses… and there’s two major advantages of that: one is that it would be free for students… students would also have access from the first day of class, and we’ll be talking about that more in future episodes… and another thing I’d like to do more is explore some alternatives to publisher provided adaptive learning tools so that it might be possible to find some ways of integrating OER with it, or to investigate ways in which OER materials can be used with adaptive learning systems that can work in classes where you want to have enough variety in the question so students can’t just look them up on the internet…

Rebecca: …and if you’re a little more interested in OER and the kind of big impact that that can have on students, you may want to check out Robin Derosa’s article in Higher Ed “OER Bigger than Affordability.” …and then we also have a previous episode that’s about adaptive learning that people might want to check out if they’re curious about that.

John: I believe was episode 30. Another thing I’d like to do, along the same lines, is I had written an econometrics text that I’ve been using in class for a while. I’d like to rewrite that as an OER text, and one of the things I need to do is update some of the old videos I’ve created. Last winter, when I was at the OLC conference in Orlando (at Disney World) I saw a presentation on Videoscribe and I had seen some videos created by that and it just looked really really cool and so I purchased a subscription to that and now I actually have to actually learn how to use it… and it does involve a bit of work… and there’s a bit of start-up costs in that, but it’s a very powerful tool and it looks like a really good way of presenting technical material.

I’d also like to explore a little bit of Flipgrid just because i’ve used voicethread now and I keep hearing really good things about Flipgrid, so I’d like to look at that and compare the benefits of the two systems.

Rebecca: What’s a Flipgrid?

John: Flipgrid is very much like Voicethread except the videos are provided in a grid. In many ways, it’s very similar to Voicethread except your class shows up as an array on the screen. You can click on any of the boxes for the students and hear or see their responses.

Rebecca: So, it sounds like the interfaces may be the big benefit there.

John: I believe so. I need to explore it more. It’s something I’ve been hearing a lot about from a lot of people who do some really good work, so I’d like to see how it compares.

Rebecca: You know all your talk of OERs and open education resources reminded me that one of the key things I have on my to-do list is to explore all the available resources that are available on openpedagogy.org. After hearing Robin DeRosa talk about it at CIT, the conference that John and I were at in late May, I got really excited about some of her teaching techniques and I just really want to see what else is out there and what’s available. So, who knows, it might really overhaul something.

John: I was at the same talk and we were both so impressed by it we went down and we talked to Robin at the end and we’ve invited her to come back to Oswego in the fall to give a presentation here, and there’s a good chance that she will appear as a guest on a future episode of the podcast. So, there’s also some things we’d like to recommend to others: books and tools that we found really useful. So, would you like to start?

Rebecca: Alright, so most of our recommendations are publications that have highly influenced our show. So one of those is Minds Online by Michelle Miller, a great cognitive psychologist. The book is about being online, but all the things she talks about works in in-person classes too, so I highly recommend that book.

John: Michelle Miller, after I had read her book, so impressed me that I invited her to come up to campus to give a workshop here… and people were so impressed by that that we created our reading group series here. Our first one was Michelle Miller’s Minds Online and participants were so enthused about that they insisted that we bring her back again at the end of the reading group and she was a wonderful speaker as well as a very good author.

Rebecca: Yeah, and a great facilitator too. We also want to recommend Barbara Oakley’s Learning How to Learn MOOC. It’s a great way to learn the basic cognitive science behind the evidence-based practices. So, if you’re not familiar, that’s a great way to follow along and get involved and her videos are fantastic.

John: It’s also the most popular MOOC in the world…

Rebecca: …and it’s the biggest one too, right?

John: and it’s the biggest one and she’s got hundreds of thousands of students taking it. It’s a four-week experience and I encourage all my students to take it.

Rebecca: …and if you’ve never done a MOOC, what a great experience to take one of the best MOOCs in the world.

John: It also provides very good examples of effective practice for online teaching that are very scalable. So, there’s a lot of good reasons to do it.

Rebecca: She also has some other great books including: A Mind for Numbers, Mind Shift, and Learning How to Learn: How to Succeed in School without Spending all your Time Studying; a Guide for Kids and Teens. That last one is a new one that’s directed specifically at middle school and high school students.

John: Another book, I think, that we’d both strongly recommend is Make it Stick. We used that as our second reading group here at Oswego a couple years ago, and Peter Brown came up and presented on that. but it’s by Brown, Roediger, and McDaniel. Peter Brown is a novelist and Roediger and McDaniel have done a tremendous amount of work in studying how people learn.

Rebecca: We can’t go without mentioning Carol Dweck’s Mindset book as well. We often see who we might traditionally think of as being quote unquote good students, “A” students maybe who hit something in college where they realize that they have to struggle a little bit and they don’t know what to do, because everything’s always come easily to them… but they struggle because they don’t have a growth mindset. So, this is a great way to learn more about the differences between fixed and growth mindsets and maybe put some strategies in place to help all of our students move more towards a growth mindset in the courses we teach.

John: The next thing we recommend is Jim Lang’s Small Teaching. it covers much of the same material as Minds Online and Make it Stick but it does it in a somewhat different way. It focuses on small techniques that you can change in your classroom that pay off very substantially. So, for people who don’t want to substantially revise their courses, it’s a very effective way of making small modifications… activities that take five to ten minutes in a class… that have a very large impact without requiring a dramatic overhaul or restructuring of your course.

Rebecca: Yeah, and the faculty here have responded very well to this book and have made a lot of small changes to their classes in the last year and had big success.

John: Another thing we’d like to mention are some podcasts that we listen to that have some really good coverage of topics related to higher education. The first one is Teaching in Higher Ed by Bonnie Stachoviak. The other one we want to recommend is Teach Better by Doug McKee and Edward O’Neill and you might remember Doug McKee from a previous episode.

Rebecca: So, we usually conclude by asking what’s next, but if you really want to know you could just listen to this episode again. We made a lot of references during this episode to a lot of great material and I can’t imagine that you wrote it all down, especially if you’re driving in your car, right? So, remember to check the show notes will have specific links and details so that you can find all these resources so that you can also enjoy some of these during your summer.

John: If any of you have any recommendations for topics for the show, please write to either of us. Our email addresses will be in the show notes.

Rebecca: We also wanted to take a couple minutes and just reflect on the podcast itself. We really appreciate the community of listeners that we’ve gained. We never expected this to even go on this long. It was a little experiment that we had that we wanted to try out in the fall and now we’re on Episode… oh, I don’t know what episode we’ll be on.

John: We’ve been really impressed by how many listeners we’ve reached across the U.S. and throughout the world. We were expecting we’d mostly get people listening from our institution and perhaps some of our colleagues in other places. So, we very much appreciate all the support you shown.

Rebecca: …and please let us know if there’s other things that we can cover that you’re really interested in or really need some professional development in.

John: We hope you’re enjoying your summer vacation. Enjoy the rest of your summer!

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

37. Evidence is Trending

Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode, Michelle Miller joins us to discuss recent research that will likely shape the future of higher education.

Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.

Show Notes

Rebecca: Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode we talk to a cognitive psychologist about recent research that will likely shape the future of higher education.
[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Michelle Miller. Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.
Welcome, Michelle!

Michelle: Hi, I’m so glad to be here.

Rebecca: Thanks for joining us.
Today’s teas are:

Michelle: I’m drinking a fresh peppermint infused tea, and it’s my favorite afternoon pick-me-up.

Rebecca: …and it looks like it’s in a really wonderfully designed teapot.

Michelle: Well, thank you… and this is a thrift store find… one of my favorite things to do. Yeah, so I’m enjoying it.

John: I have Twinings Blackcurrant Breeze.

Rebecca: …and I’m drinking chai today.

Michelle: Pretty rough.

John: We invited you here to talk a little bit about things that you’ve been observing in terms of what’s catching on in higher education in terms of new and interesting innovations in teaching.

Michelle: Right, that’s one of things that I really had the luxury of being able to step back and look at over this last semester and over this last spring when I was on sabbatical… One of the really neat things about my book Minds Online, especially now that it’s been out for a few years, is that it does open up all these opportunities to speak with really engaged faculty and others, such as: instructional designers, librarians, academic leadership, educational technology coordinators… all these individuals around the country who are really, really involved in these issues. It’s a great opportunity to see how these trends, how these ideas, how these innovations are rolling out, and these can be some things that have been around for quite some time and just continue to rock along and even pickup steam, and some newer things that are on the horizon.

John: You’ve been doing quite a bit of traveling. You just got back from China recently, I believe.

Michelle: I sure did. It was a short visit and I do hope to go back, both to keep getting involved in educational innovations there and, hopefully, as a tourist as well. So, I was not there for very long but I had the opportunity to speak at Tsinghua University in Beijing, which is a really dynamic institution that’s been around for about a hundred years. For a while in its history it specialized in things like engineering education polytechnic, but now it’s really a selective comprehensive university with very vibrant graduate and undergraduate programs that are really very relatable for those of us in the United States working in similar contexts. My invitation was to be one of the featured speakers at the Future Education, Future Learning Conference, which was a very interdisciplinary gathering of doctoral students, faculty, even others from the community, who were all interested in the intersection of things like technology, online learning, MOOCs even, and educational research (including research into the brain and cognitive psychology), and bringing all of those together… and it was a multilingual conference. I do not speak Chinese but much of the conference was in both English and Chinese and so I was also able to really absorb a lot of these new ideas. So yes, that was a real highlight of my sabbatical semester and one that I’m going to be thinking about for quite some time.

I should say that part of what tied in there as well is that Minds Online, I’ve just learned, is going to be translated into Chinese and that’s going to come out in May 2019. So, I also got to meet with some of the people who were involved in the translation… start to put together some promotional materials such as videos and things like that.

Rebecca: Cool.

John: Excellent.

Rebecca: So, you’ve had a good opportunity, as you’ve been traveling, to almost do a scavenger hunt of what faculty are doing with evidence-based practices related to your book. Can you share some of what you’ve found or heard?

Michelle: This theme of evidence-based practice, and really tying into the findings that have been coming out of cognitive psychology for quite some time, that really is one of the exciting trends and things that I was really excited to see and hear for so many different quarters I visited in different institutions… and so I would say definitely, this is a trend that is continuing and is increasing. There really does continue to be a lot of wonderful interest and wonderful activity around these real cognitively informed approaches to teaching, and what I think we could call scientifically based and evidence-based strategies. One form this has taken is Josh Eyler’s new book, called How Humans Learn: The Science and Stories behind Effective College Teaching. This is a brand new book by a faculty development professional, and a person coming out of the humanities, actually, who’s weaving together even from his humanities background everything from evolutionary biology to classical research in early childhood education to the latest brain-based research. He’s weaving this together into this new book for faculty. So, that’s one of the things that I’ve noticed and then there’s the issue which i think is another great illustration of best-known practice which is the testing effect and retrieval practice.

John: One of the nice things is how so many branches of research are converging… testing in the classroom, brain-based research, and so forth, are all finding those same basic effects. It’s nice to see such robust results, which we don’t always see in all research in all disciplines.

Rebecca: …and just breaking down the silos in general. The things are all related and finding out what those relationships are… exploring those relationships… is really important and it’s nice to see that it’s starting to open up.

John: We should also note that when you visited here, we had a reading group and we had faculty working on trying to apply some of these concepts, and they’re still doing that… and they still keep making references back to your visit. So, it’s had quite a big impact on our campus.

Michelle: This wasn’t true, I don’t think, when I first entered the teaching profession… and even to the extent when I first started getting interested in applied work in course redesign and in faculty professional development. you would get kind of this pushback or just strange looks when you said “Oh, how about we bring in something from cognitive psychology” and now that is just highly normalized and something that people are really speaking across the curriculum… and taking it and running with it in a lasting ongoing way, not just as a “Oh, well that was an interesting idea. I’m going to keep doing what I’m doing” but really people making some deep changes as you mentioned. This theme of breaking down silos… I mean I think if there’s kind of one umbrella trend that all of these things fits under it’s that breakdown of boundaries. So, that’s one that I keep coming back to, I know, in my work.

So, the idea of retrieval practice, drilling down on that one key finding which goes back a very long ways in cognitive psychology. I think of that as such a good example of what we’re talking about here… about how this very detailed effect in cognition and yet it does have these applications across disciplinary silos. Now when I go to conferences and I say “Okay, raise your hand. How many people have ever heard of retrieval practice? How many people have ever heard of the testing effect? How many people have heard of the book Make it Stick (which really places this phenomena at its center)?” and I’m seeing more hands raising.

With retrieval practice, by the way, we’re talking about that principle that taking a test on something, that retrieving something from memory actively, has this huge impact on future memorability of that information. As its proponents like to say, tests are not neutral from a memory or from a learning standpoint… and while some of the research has focused on very kind of stripped-down laboratory style tasks like memorizing words pairs, there are also some other research projects showing that it does flow out to more realistic learning situations.

So, more people simply know about this, and that’s really the first hurdle, oftentimes, with getting this involved disciplinary sometimes jargon riddled research out there to practitioners and getting it into their hands. So, people heard of it and they’re starting to build this into their teaching. As I’ve traveled around I love to hear some of the specific examples and to see it as well crop up in scholarship of teaching and learning.

Just recently, for example, I ran across and really got into the work of Bruce Kirchhoff who is at University of North Carolina – Greensboro and his area is botany and plant identification. He has actually put together some different really technology-based apps and tools that students and teachers can use in something like a botany course to rehearse and review plant identification. He says in one of his articles, for example, that there just isn’t time in class to really adequately master plant identification. It’s just too complex of a perceptual and cognitive and memory test to do that. So, he really built in from the get-go very specific principles drawn from cognitive psychology… so, the testing effect is in there… there’s different varieties of quizzing and it all is about just getting students to retrieve and identify example after example. It brings in also principles such as interleaving, which we could return to in a little bit, but has to do with the sequencing of different examples… their spacing… So, that’s even planned out exactly how and when students encounter different things that they’re studying. It’s really wonderful. So, for example he and his colleagues put out a scholarship of teaching and learning article talking about how this approach was used effectively in veterinary medicine students who have to learn to identify poisonous plants that they’ll see around their practice. This is something that can be time-consuming and very tough, but they have some good data showing that this technology enhanced cognitively based approach really does work. That’s one example. Coincidentally, I’ve seen some other work in the literature, also on plant identification, where the instructors tagged plants in an arboretum… they went around and tagged them with QR codes… that students can walk up to a plant in the real environment with an iPad… hold the iPad over it… and it would immediately start producing quiz questions that were are specific to exactly the plants they were looking at.
So, those are some of the exciting things that people are taking and running with now that this principle is out there.

Rebecca: What I really love about the two stories that you just shared was the faculty are really designing their curriculum and designing the learning experiences with the students in mind… and what students need and when they need it. So, not only is it employing these cognitive science principles, but it’s actually applying design principles as well. It’s really designing for a user experience and thinking about the idea that if I need to identify a plant, being able to identify it in this situation in which I would need to identify it in makes it much more dynamic I think for a student… but also really meets them where they’re at and where they need it.

John: …and there’s so many apps out there now that will do the plant identification just from imagery without the QR code, that I can see it taking it one step further where they can do it in the wild without having that… so they can build it in for plants that are in the region without needing to encode that specifically for the application.

Michelle: I think you’re absolutely right once we put the technology in the hands of faculties who, as I said, they’re the one to know: “Where are my students at? Where are the weak points? Where are the gaps that they really need to bridge?” and that’s where their creativity is giving rise to all these new applications… and sometimes these can be low-tech as well… or also things that we can put in a face-to-face environment… and I’d like to to share just some experiences that I’ve had with this over the last few semesters.

In addition to trying to teach online with a lot of technology, I also have in my teaching rotation a small required course in research methods in psychology which can be a real stumbling block… the big challenge course… it’s kind of a gateway course to continued progress in our major. So, in this research methods course, some of the things that I’ve done around assessment and testing to really try again to stretch that retrieval practice idea… to make assessments really a more dynamic part of the course and more central part of the course… to move away from that idea that tests are just this kind of every now and again this panic mode opportunity for me to kind of measure in sorts of students and judge them… to make good on that idea that tests are part of learning. So, here’s some of the things that I try to do. For one thing, I took time out of the class almost every single class meeting as part of the routine to have students first of all generate quiz questions out of their textbook. So, we do have a certain amount of foundational material in that course as well as a project and a whole lot of other stuff is going on. So they need to get that foundational stuff.

Every Tuesday they would come in and they knew their routine: you get index cards and you crack your textbook and you generate for me three quiz questions. Everybody does it. I’m not policing whether you read the chapter or not. It’s active… they’re generating it… and also that makes it something like frequent quizzing. That’s a great practical advantage for me since I’m not writing everything. They would turn those in and I would select some of my favorites I would turn those into a traditional looking paper quiz and hand that out on Thursday. I said “Hey, take this like a realistic quiz.” I had explained to them that quizzes can really boost their learning, so that was the justification for spending time on it and then I said: “You know what? I’m not going to grade it either. You take it home because this is a learning experience for you. It’s a learning activity.” so we did that every single week as those students got into that routine.

The second thing that I did to really re-envision how assessment testing and quizzing worked in this particular course, was something inspired by different kinds of group testing and exam wrapper activities I’ve seen, particularly coming out of the STEM field, where there’s been a lot of innovation in this area. What I would do is… we had these high stakes exams at a few points during the semester. But, the class day after the exam, we didn’t do the traditional “Let’s go over the exam.” [LAUGHTER] That’s kind of deadly dull, and it just tends to generate a lot of pushback from students… and as we know from the research, simply reviewing… passing your eyes over the information… is not going to do much to advance your learning. So, what I would do is… I would photocopy all those exams, so it has a secure copy. They were not graded. I would not look at this before we did this… and I would pass everybody’s exams back to them along with a blank copy of that same exam. I assigned them to small groups and I said “Okay, here’s your job. Go back over this exam, fill it out as perfectly as you can as a group, and to make it interesting I said I will grade that exam as well, the one you do with your group, and anything you get over 90% gets added to everybody’s grade. This time it was open book, it was open Google, it was everything except you can’t ask me questions. So, you have each other and that’s where these great conversations started to happen. The things that we always want students to say. So, I would eavesdrop and hear students say “Oh, well you know what, I think on this question she was really talking about validity because reliability is this other thing…” and they’d have a deep conversation about it. I’m still kind of going back through the numbers to see what are the impacts of learning? Are there any trends that I can identify? But, I will say this: in the semesters that I did this, I didn’t have a single question ever come back to me along the lines of “Well, this question was unclear. I didn’t understand it. I think I was graded unfairly.” it really did shut all that down and again extended the learning that I feel students got out of that. Now it meant a big sacrifice of class time, but I feel strongly enough about these principles that I’m always going to do this in one form or another anytime I can can in face-to-face classes.

Rebecca: This sounds really familiar, John.

John: I’ve just done the same, or something remarkably similar, this semester, in my econometrics class which is very similar to the psych research methods class. I actually picked it up following a discussion with Doug McKee. He actually was doing it this semester too. He had a podcast episode on it. It sounded so exciting, I did something… a little bit different. I actually graded it but I didn’t give it back to them because I wanted to see what they had the most trouble with, and then I was going to have them only answer the ones in a group that they struggled with… and it turned out that that was pretty much all them anyway. So, it’s very similar to what you did except I gave them a weighted average of their original grade and the group grade and all except one person improved and the one person’s score went down by two points because the group grade was just slightly lower… but he did extremely well and he wasn’t that confident. The benefits to them of that peer explanation and explaining was just tremendous and it was so much more fun for them and for me and, as you said, it just completely wiped out all those things like “Well, that was tricky” because when they hear their peers explaining it to them the students were much more likely to respond by saying “Oh yeah, I remember that now” and it was a wonderful experience and I’m gonna do that everywhere I can.

In fact. I was talking about it with my TA just this morning here at Duke and we’re planning to do something like that in our classes here at TIP this summer, which i think is somewhat familiar to you from earlier in your academic career.

Michelle: That is right we do have this connection. I was among, not the very first year, but I believe the second cohort of Talent Identification Program students who came in, I guess you would call it now, middle school (back then, it was called junior high) and what a life-transforming experience. We’ve had even more opportunities to talk about the development of all these educational ideas through that experience.

John: That two-stage exam is wonderful and it’s so much more positive… because it didn’t really take, in my class, much more time, because I would have spent most of that class period going over the exam and problems they had. But the students who did well would have been bored and not paying much attention to it; the students who did poorly would just be depressed and upset that they did so poorly… and here, they were actively processing the information and it was so positive.

Michelle: That’s a big shift. We really have to step back and acknowledge that, I think. that is a huge shift in how we look at assessment, and how we think about the use of class time… and it’s not just “Oh my gosh, I have to use every minute to put such content in front of the students.” Just the fact that more of us are making that leap, I think, really is evidence this progress is happening… and we see also a lot of raised consciousness around issues such as learning styles. That’s another one that, when I go out and speak to faculty audiences, 10 years ago you would get these shocked looks or even very indignant commentary when you say “Ok, this idea of learning styles, in the sense that say there are visual learners, auditory learners, what I call sensory learning styles (VAK is another name it sometimes goes by). The idea that that just holds no water from a cognitive point of view…” People were not good with that, and now when I mentioned that at a conference, I get the knowing nods and even a few groans… people like “Oh, yeah. we get that. Now, K-12, which I want to acknowledge it’s not my area, but I’m constantly reminded by people across the spectrum that it’s a very different story in K-12. So, setting that aside… but this is what I’m seeing… that faculty are realizing… they’re saying “Oh, this is what the evidence says…” and maybe they even take the time to look at some of the really great thinkers and writers who put together the facts on this. They say “You know what? I’m not going to take my limited time and resources and spend that on this matching to styles when the styles can’t even be accurately diagnosed and are of no use in a learning situation. So, that’s another area of real progress.

Rebecca: What I am hearing is not just progress here in terms of cognitive science, but a real shift towards really thinking about how students learn and designing for that rather than something that would sound more like a penalty for grade like “Oh, did you achieve? Yes or no…” but, rather here’s an opportunity if you didn’t achieve to now actually learn it… and recognize that you haven’t learned it, even though it might seem really familiar.

John: Going back to that point about learning styles. It is spreading in colleges. I wish it was true at all the departments at our institution, but it’s getting there gradually… and whenever people bring it up, we generally remind them that there’s a whole body of research on this and I’ll send them references but what’s really troubling is in my classes the last couple years now, I’ve been using this metacognitive cafe discussion forum to focus on student learning… and one of the week’s discussions is on learning styles and generally about 95 percent of the students who are freshmen or sophomores (typically) come in with a strong belief in learning styles… where they’ve been tested multiple times in elementary or middle school… they’ve been told what their learning styles are… they’ve been told they can only learn that way… It discourages them from trying to learn in other ways and it does a lot of damage… and I hope we eventually reach out further so that it just goes away throughout the educational system.

Rebecca: You’ve worked in your classes, Michelle, haven’t you to help students understand the science of learning and use that to help students understand the methods and things that you’re doing>

Michelle: Yes, I have. I’ve done this in a couple of different ways. Now, partly, I get a little bit of a free pass in some of my teaching because I’m teaching the introduction to psychology or I’m teaching research methods where I just happen to sneak in as the research example will be some work on say attention or distraction or the testing effect. So, I get to do it in those ways covertly. I’ve also had the chance, although it’s not on my current teaching rotation… I’ve had the chance to also take it on as in freestanding courses. As many institutions are doing these days… it’s another trend… and what Northern Arizona University, where I work, has different kinds of freshmen or first-year student offering for courses they can take, not in a specific disciplinary area, but that really crossed some different areas of the student success or even wellbeing. So, I taught a class for awhile called Maximizing Brain Power that was about a lot of these different topics. Not just the kind of very generic study skills tip… “get a good night’s sleep…” that kind of thing… but really some again more evidence-based things that we can tell students and you can really kind of market it… and I think that we do sometimes have to play marketers to say “Hey, I’m going to give you some inside information here. This is sort of gonna be your secret weapon. So, let me tell you what the research has found.”

So, those are some of the things that I share with students… as well as when the right moment arises, say after an exam or before their first round of small stakes assessments, where they’re taking a lot of quizzes… to really explain the difference between this and high stakes or standardized tests they may have taken in the past. So, I do it on a continuing basis. I try to weave it into the disciplinary aspect and I do it in these free-standing ways as well… and I think here’s another area where I’m seeing this take hold in some different places… which is to have these free-standing resources that also just live outside of a traditional class that people can even incorporate into their courses… if say cognitive psychology or learning science isn’t their area… that they can bring in, because faculty really do care about these things. We just don’t always have the means to bring them in in as many ways as we would like.

John: …and your Attention Matters project was an example of that wasn’t it? Could you tell us a little bit about that?

Michelle: Oh, I’d love to… and you know this connects to what it seems to be kind of an evergreen topic in the teaching and learning community these days, which is the role of distracted students… and I know this past year there just have been these one op-ed versus another. There’s been some really good blog posts by some people I really like to follow in the teaching and learning community such as Kevin Gannon talking about “Okay, do you have laptops in the classroom? and what happens when you do?” and so I don’t think that this is just a fad that’s going away. This is something that the people do continue to care about, and this is where the attention matters project comes in.

This was something that we conceptualized and put together a couple years ago at Northern Arizona University with myself, and primarily I collaborated with a wonderful instructional designer who also teaches a great deal… John Doherty. So, how this came about is I was seeing all the information on distraction… I’m really getting into this as a cognitive psychologist and going “Wow, students need to know that if they’re texting five friends and watching a video in their class. It’s not going to happen for them.” I was really concerned about “What can I actually do to change students minds?” So, my way of doing this was to go around giving guests presentations in every classes where people would let me burn an hour of their class time… and not a very scalable model… and John Doherty respectfully sat through one my presentations on this and then he approached me and said “Look, you know, we could make a module and put this online… and it could be an open access within the institution module, so that anybody at my school can just click in and they’re signed up. We could put this together. We could use some really great instructional design principles and we could just see what happens… and I bet more people would take that if it were done in that format. We did this with no resources. We just were passionate about the project and that’s what we did. We had no grant backing or anything. We got behind it. So, what this is is about a one- to two-hour module that, it’s a lot like a MOOC in that it there’s not a whole lot of interaction or feedback, but there are discussion forums and it’s very self-paced in that way… so one- to two-hour mini MOOCs that really puts at the forefront demonstrations and activities… so we don’t try to convince students about problems with distraction and multitasking… we don’t try to address that just by laying a bunch of research articles on them… I think that’s great if this were a psychology course, but it’s not. So, we come at it by linking them out to videos, for example, that we were able to choose, that we feel really demonstrate in some memorable ways what gets by us when we aren’t paying attention… and we also give students some research-based tips on how to set a behavioral plan and stick to it… because just like with so many areas of life, just knowing that something is bad for you is not enough to really change your behavior and get you not to do that thing. so we have students talking about their own plans and what they do when, say, they’re having a boring moment in class, or they’re really really tempted to go online while they’re doing homework at home. What kinds of resolutions can they set or what kind of conditions can make that that will help them accomplish that. Things like the software blockers… you set a timer on your computer and it can lock you out of problematic sites… or we learned about a great app called Pocket Points where you actually earn spendable coupon points for keeping your phone off during certain hours. This is students talking to students about things that really concern them and really concern us all because I think a lot of us struggle with that.

So, we try to do that… and the bigger frame for this as well is this is, I feel, a life skill for the 21st century… thinking about how technology is going to be an asset to you and not detract from what you accomplish in your life. What a great time to be reflecting on that, when you’re in this early college career. so that’s what we try to do with the project…and we’ve had over a thousand students come through. They oftentimes earn extra credit. Our faculty are great about offering small amounts of extra credit for completing this and we’re just starting to roll out some research showing some of the impacts… and showing it in a bigger way just how you can go about setting up something like this.

Rebecca: I like that the focus seems to be on helping students with a life skill rather than using technology is just a blame or an excuse. We’re in control of our own behaviors and taking ownership over our behaviors is important rather than just kind of object blaming.

Michelle: So, looking at future trends, I would like to see more faculty looking at it in the way that you just described, Rebecca, as this is a life skill and it’s something that we collaborate on with our students… not lay down the law… because, after all, students are in online environments where we’re not there policing that and they do need to go out into work environments and further study and things like that. So, that’s what I feel is the best value. For faculty who are looking at this, if they don’t want to do… or don’t have the means to do something really formal like our Attention Matters approach, just thinking about it ahead of time… I think nobody can afford to ignore this issue anymore and whether you go the route of “No tech in my classroom” or “We’re going to use the technology in my classroom“ or something in between… just reading over, in a very mindful way, not just the opinion pieces, but hopefully also a bit of the research, I think, can help faculty as they go in to deal with this… and really to look at it in another way, just to be honest, we also have to consider how much of this is driven by our egos as teachers and how much of it is driven by a real concern for student learning and those student life skills. I think that’s where we can really take this on effectively and make some progress when we are de-emphasizing that ego aspect and making sure that it really is about the students.

John: We should note there’s a really nice chapter in this book called Minds Online: Teaching Effectively with Technology that deals with these types of issues. It was one of the chapters that got our faculty particularly interested in these issues… on to what extent technology should be used in the classroom… and to what extent it serves as a distraction.

Michelle: I think that really speaks to another thing which I think is an enduring trend… which is the emphasis on really supporting the whole student in success and what we’ve come to call academic persistence… kind of a big umbrella term that has to do with, not just succeeding in a given class, but also being retained… coming back after the first year. As many leaders in higher education point out, this is as a financial issue. As someone pointed out, it does cost a lot less to hang on to the students you have instead of recruiting more students to replace ones who are lost. This is, of course, yet another really big shift in mindset of our own, because after all we did used to measure our success by “Hey, I flunked this many students out of this course” or” Look at how many people have to switch into different majors…our major is so challenging…”

So, we really have turned that thinking around and this does include faculty now. I think that we did used to see those silos. We had that very narrow view of “I’m here to convey content. I’m here to be an expert in this discipline, and that’s what I’m gonna do…” and sure, we want to think about things like do students have learning skills? Do they have metacognition? Are they happy and socially connected at the school? Are they likely to be retained so that we can have this robust university environment?

We had people for that, right? It used to be somebody else’s job… student services or upper administration. They were the ones who heard about that and now I think on both sides we really are changing our vision. More and more forward-thinking faculty are saying “You know what? Besides being a disciplinary expert, I want to become at least conversant with learning science. I want to become at least conversant with the science of academic persistence…” There is a robust early literature on this and that’s something that we’ve been working on at NAU over this past year as well… kind of an exciting newer project that I like very much. We’ve started to engage faculty in a new faculty development program called Persistence Scholars and this is there to really speak to people’s academic and evidence-based side, as well as get them to engage in some perspective-taking around things like the challenges that students face and what it is like to be a student at our institution. We do some really selected readings in the area we look at things like mindset… belongingness… these are really hot areas in that science of persistence… in that emerging field. But, we have to look at it in a really integrated way.

It’s easy for people to say just go to a workshop on mindset and that’s a nice concept, but we wanted to think about it in this bigger picture… really know what are some of the strengths of that and why? Where do these concepts come from? What’s the evidence? That’s something that I think is another real trend and I think as well we will see more academic leaders and people in staff and support roles all over universities needing to know more about learning science. There are still some misconceptions that persist, as we’ve talked about. We’re making progress in getting rid of some of these myths around learning, but I will say… I’m not gonna name any names… but, every now and again I will hear from somebody who says “Oh well, we need to match student learning styles” or “Digital natives think differently, don’t you know?” and I have to wonder whether that’s a great thing. I mean, these are oftentimes individuals that have the power to set the agenda for learning all over a campus. Faculty need to be in the retention arena and I think that leaders need to be in the learning science arena. The boundaries is breaking down and it’s about time.

Rebecca: One of the things that I thought was really exciting with the reading groups that we’ve been having on our campus… that we started with your book, but then we’ve read Make it Stick and Small Teaching since… is that a lot of administrators in a lot of different kinds of roles engaged with us in those reading groups, it wasn’t just faculty. There was a mix of faculty, staff, and some administrators, and I think that that was really exciting. For people who don’t have the luxury of being in your persistence scholar program, what would you recommend they read to get started to learn more about the science of persistence?

Michelle: I really, even after working with this for quite some time, I loved the core text that we have in that program, which is Completing College by Vincent Tinto. It’s just got a great combination of passionate and very direct writing style. So, there’s no ambiguity, there’s not a whole lot of “on the one hand this and on the other hand that.” It’s got an absolutely stellar research base, which faculty of course appreciate… and it has a great deal of concrete examples. So, in that book they talk about “okay, what does it mean to give really good support to first semester college students? What does that look like?” and they’ll go out and they’ll cite very specific “Here’s a school and here’s what they’re doing… here’s what their program looks like… here’s another example that looks very different but gets at the same thing.” So, that’s one of the things that really speak to our faculty… that they really appreciated and enjoyed.

I think that as well we tested good feedback about work that’s come out of the David Yeager and his research group on belongingness and lay theories, and lay theories is maybe a counterintuitive term for kind of a body of ideas about what students believe about academic success and why some people are successful and others are not and how those beliefs can be changed sometimes through relatively simple interventions and when it happens we see great effects such as the narrowing of achievement gaps among students who have more privilege or less privileged backgrounds… and that’s something that, philosophically, many faculty really really care about but they’ve never had the chance to really learn “Okay, how can I actually address something like that with what I’m doing in my classroom, and how can I really know that the things that I’m choosing do have that great evidence base…”

John: …and I think that whole issue is more important now and is very much a social justice issue because, with the rate of increase we’ve seen in college cost inflation, people who start college and don’t finish it are saddled with an awfully high burden of debt. The rate of return to a college degree is the highest that we’ve ever seen and college graduates end up not only getting paid a lot more but they end up with more comfortable jobs and so forth… and if we really want to move people out of poverty and try to reduce income inequality, getting more people into higher education and successfully completing higher education is a really important issue. I’m glad to see that your institution is doing this so heavily and I know a lot of SUNY schools have been hiring Student Success specialists. At our institution they’ve been very actively involved in the reading group, so that message is spreading and I think some of them started with your book and then moved to each of the others. So, they are working with students in trying to help the students who are struggling the most with evidence-based practices …and I think that’s becoming more and more common and it’s a wonderful thing.

Rebecca: So, I really liked Michelle that you were talking about faculty getting involved in retention and this idea of helping students develop persistence skills, and also administrators learning more about evidence-based practices. There’s these grassroots movements happening in both of these areas. Can you talk about some of the other grassroots movements that are working toward, or efforts that faculty are making to engage students and capture their attention and their excitement for education?

Michelle: Right, and here I think a neat thing to think about too is just it’s the big ambitious projects… the big textbook replacement projects or the artificial intelligence informed adaptive learning systems… those are the things that get a lot of the press and end up in The Chronicle of Higher Education that we read about… But, outside of that, there is this very vibrant community and grassroots led scene of developing different technologies and approaches. So, it really goes back for a while. I mean, the MERLOT database that I do talk about in Minds Online has been trove for years of well hidden gems that take on one thing in a discipline and come at it from a way that’s not just great from a subject-matter perspective but brings up the new creative approaches. In the MERLOT database, for example, there’s a great tutorial on statistical significance and the interrelationship between statistical significance and issues like simple sizes. You know, that’s a tough one for students, but it has a little animation involving a horse and a rider that really turns it into something that’s very visual… that’s very tangible… and it really actually tying into analogies, which is a well-known cognitive process that can support the advancement of learning something new. There is something on fluid pressures in the body that was treated for nursing students by nurses, and it’s got an analogy of a soaker hose that this is really fun and is actually interactive. So, those are the kinds of things. The PhET project, P-h-E-T which comes out of University of Colorado, that has been around for a while… again, faculty-led and a way to have these very useful interactive simulations for concepts in physics and chemistry. So, that’s one. CogLab, that’s an auxiliary product that I’ve used for some time in like hundred psychology courses that simulates very famous experimental paradigms which are notoriously difficult to describe on stage for cognitive psychology students. That started out many years ago as a project that very much has this flavor of “We have this need in our classroom. We need something interactive. There’s nothing out there. Let’s see what we can build.” It has since then picked up and turned into a commercial product, but that’s the type of thing that I’m seeing out there.

Another thing that you’ll definitely hear about if you’re circulating and hearing about the latest project is virtual reality for education. So, with this it seems like, unlike just a few years ago, almost everywhere you visit you’re going to hear that “Oh, we’ve just set up a facility. We’re trying out some new things.” This is something that I also heard about when I was talking to people when I was over in China. So, this is an international phenomenon. It’s going to pick up steam and definitely go some places.

What also strikes me about that is just how many different projects there are. Just when you’re worried that you’re going to be scooped because somebody else is going to get there first with their virtual reality project you realize you’re doing very very different things. So, I’ve seen, for example, it used in a medical application to increase empathy among medical students… and I took a six or seven minute demonstration that just was really heart-rending, simulating the patient experience with a particular set of sensory disorders… and at Northern Arizona University we have a lab that is just going full-steam in coming up with educational applications such as interactive organic chemistry tutorial that is is just fascinating. We actually completed a pilot project and are planning to gear up a much larger study next semester looking at the impacts of this. So, this is really taking off for sure.

But, I think there are some caveats here. We still really need some basic research on this… not just what should we be setting up and what the impacts are but how does this even work? In particular, what I would like to research in the future, or at least see some research on, is what kinds of students… what sort of student profile… really gets the most out of virtual reality for education. Because amidst all the very breathless press that’s going on about this now and all the excitement, we do have to remember this is a very, very labor intensive type of resource to set up. You’re not just going to go home and throw something together for the next week. It takes a team to build these things and to complete them as well. If you have, say, a 300 student chemistry course (which is not atypical at all… these large courses), you’re not going to just have all of them spend hours and hours and hours doing this even with a fairly large facility. It’s a very hands-on thing to guide them through this process, to provide the tech support, and everything else.

So, I think really knowing how we can best target our efforts in this area, so that we can build the absolute best, with the resources we have, and maybe even target and ask the students who are most likely to benefit. I think those are some of the things that we just need to know about this. So, it’s exciting for somebody like me who’s in the research area. I see this as a wonderful open opportunity… but those are some of the real crossroads we’re at with virtual reality right now.

Rebecca: I can imagine there’s a big weighing that would have to happen in terms of expense and time and resources needed to startup versus what that might be saving in the long run. I can imagine if it’s a safety thing that you want to do a virtual reality experience, like saving people’s lives and making sure that they’re not going to be in danger as they practice particular skills, could be a really good investment in these… spending the resources to make that investment… or if it’s a lot of travel that would just be way too expensive to bring a bunch of students to a particular location… but you could virtually… it seems like it would be worth the start-up costs and those are just two ideas off the top of my head where it would make sense to bend all of that resource and time.

John: …and equipment will get cheaper. Right now, it’s really expensive for computers that have sufficient speed and graphics processing capability and the headsets are expensive, but they will come down in price, but as you said, it’s still one person typically and one device… so it doesn’t scale quite as well as a lot of other tools or at least not at this stage.

Rebecca: From what I remember, Michelle, you wrote a blog post about [a] virtual reality experience that you had. Can you share that experience, and maybe what stuck with you from that experience?

Michelle: Right, so I had the opportunity, just as I was getting to collaborate with our incredible team at the immersive virtual reality lab at NAU… one of the things I was treated to was about an hour and a half in the virtual reality setup that they have to explore some of the things that they had… Giovanni Castillo, by the way, is creative director of the lab and he’s the one who was so patient with me through all this. We tried a couple of different things and of course there’s such a huge variety of different things that you can do.
There’s a few things out there like driving simulators that are kind of educational… they’re kind of an entertainment… but he was just trying to give me, first of all, just a view of those… and I had to reject a few of them… I will say, initially, because I am one of the individuals who tends to be prone to motion sickness. So, that limits what I can personally do in VR and that is yet another thing that we’re gonna have to figure out. At least informally, what we hear is that women in particular tend to experience more of this. So, I needed, first of all, to go to a very low motion VR. I wasn’t gonna be whizzing through these environments. That was not going to happen for me. So, we did something that probably sounds incredibly simplistic, but it just touched me to my core… which is getting to play with Google Earth. You can spin the globe and either just pick a place at random or what Giovanni told me is… “You know, I’ve observed that when people do this, when we have an opportunity to interact with Google Earth, they all either go to where they grew up or they’ll go to someplace that they have visited recently or they plan to visit. So, I went to a place that is very special to me and maybe it doesn’t fit into either one of those categories neatly, but it’s my daughter’s University… her school… and I should say that this is also a different thing for me because my daughter goes to school in Frankfort, Germany… an institute that is connected to a Museum. So, I had only been to part of the physical facility… the museum itself… and it was a long time ago… and part of it was closer to the holiday. So, this is my opportunity to go there and explore what it looks like all over… and so, that was an emotional experience for me. It was a sensory experience… it was a social one… because we were talking the whole time… and he’s asking me questions and what kinds of exhibits do they have here… and what’s this part of it. So, that was wonderful. it really did give me a feel for alright, what is it actually like to be in this sort of environment?

I’m not a gamer. I don’t have that same background that many of our students have. So, it got me up to speed on that… and it did show me how just exploring something that is relatively simple can really acquire a whole new dimension in this kind of immersive environment. Now the postscript that I talked about in that blog post was what happened when I actually visited there earlier in the year. So, I had this very strange experience that human beings have never had before… which is from this… I don’t know whether to call it deja vu or what… of going to the settings and walking around the same environment and seeing the same lighting and all that sort of stuff that was there in that virtual reality environment… but this time, of course, with real human beings in it and the changes… the little subtle changes that take place over time, and so forth.

So, how does it translate into learning? What’s it going to do for our students? I just think that time is going to tell. It won’t take too long, but I think that these are things we need to know. But, sometimes just getting in and being able to explore something like this can really put you back in touch with the things you love about educational technology.

Rebecca: I think one of the things that I’m hearing in your voice is the excitement of experimenting and trying something… and that’s, I think, encouragement for faculty in general… is to just put yourself out there and try something out even if you don’t have something specific in mind with what you might do with it. Experiencing it might give you some insight later on. it might take some time to have an idea of what you might do with it, but having that experience, you understand it better… it could be really useful.

John: …and that’s something that could be experienced on a fairly low budget with just your smartphone and a pair of Google cardboard or something similar. Basically, it’s a seven to twelve dollar addition to your phone and you can have that experience… because there’s a lot of 3D videos and 3D images out there on Google Earth as well as on YouTube. So, you can experience other parts of the world and cultures before visiting… and I could see that being useful in quite a few disciplines.

Rebecca: So, we always wrap up with asking what are you going to do next?

Michelle: I continue to be really excited about getting the word out about cognitive principles and how we can flow those in to teaching face-to-face with technology… everything else in between. So, that’s what I continue to be excited about… leveraging cognitive principles with technology and with just rethinking our teaching techniques. I’m going to be speaking at the Magna Teaching with Technology Conference in October, and so I’m continuing to develop some of these themes… and I’m very excited to be able to do that. I’m right now also… we’re in the early stages of another really exciting project that has to do with what we will call neuromyth… So, that may be a term that you’ve turn across in some of your reading. It’s something that we touched on a few times, I think, in our conversation today… the misconceptions that people have about teaching and learning and how those can potentially impact the choices we make in our teaching. So, I’ve had the opportunity to collaborate with this amazing international group of researchers who’s headed up by Dr. Kristen Betts of Drexel University… and I won’t say too much more about it other than we have a very robust crop of survey responses that have come in from, not just instructors, but also instructional designers and administrators from around the world. So, we’re going to be breaking those survey results down and coming up with some results to roll out probably early in the academic year and we’ll be speaking about that at the Accelerate conference, most likely in November. That’s put out by the Online Learning Consortium. So, we’re right in the midst of that project and it’s going to be so interesting to see what has the progress been? What neuromyths are still out there and how can they be addressed by different professional development experiences. We’re continuing to work on the Persistence Scholars Program on academic persistence. So, we’ll be recruiting another cohort of willing faculty to take that on in the fall at Northern Arizona University. I am going to be continuing to collaborate and really work with and hear from John and his research group with respect to the metacognitive material that they’re flowing into foundational coursework and ways to get students up to speed with a lot of critical metacognitive knowledge. So, we’re going to work on that too… and I like to keep up my blog and work on shall we say longer writing project but we’ll have to stay tuned for that.

Rebecca: Sounds like you need to plan some sleep in there too.

[LAUGHTER]

John: Well, it’s wonderful talking to you, and you’ve given us a lot of great things to reflect on and to share with people.

Rebecca: Yeah. Thank you for being so generous with your time.

John: Thank you.

Michelle: Oh, thank you. Thanks so much. It’s a pleasure, an absolute pleasure. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

36. Peer instruction

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

35. FLEXspace

Learning spaces greatly influence the teaching methods and learning activities we can employ. In this episode, Lisa Stephens and Rebecca Frazee join us to discuss the Flexible Learning Environments eXchange, an international platform for archiving, exploring and planning informal and formal learning spaces.

Lisa serves as the Assistant Dean for Digital Education in the UB School of Engineering and is a Senior Strategist for Academic Innovation in the Office of the SUNY Provost. Her work at SUNY includes serving as the Interim Director of FLEXspace. Rebecca Frazee is a member of the San Diego State University faculty in the Learning Design and Technology Program. She serves as the FLEXspace Manager.

Show Notes

Transcript

Rebecca M: Learning spaces greatly influence the teaching methods and learning activities we can employ. In this episode, we discuss the Flexible Learning Environments eXchange, an international platform for archiving, exploring and planning informal and formal learning spaces.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca M: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca M: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guests today are Lisa Stephens and Rebecca Frazee. Lisa serves as the Assistant Dean for Digital Education in the UB School of Engineering and is a Senior Strategist for Academic Innovation in the Office of the SUNY Provost. Her work at SUNY includes serving as the Interim Director of FLEXspace. Rebecca Frazee is a member of the San Diego State University faculty in the Learning Design and Technology Program. She serves as the FLEXspace Manager.

Rebecca M: Welcome.
Our teas for today are:

John: Are either of you drinking tea?

Lisa: No, but I really am jonesing for my gunpowder green at home. It’s highly caffeinated.

Rebecca F: Yes, my tea was a cinnamon vanilla yogi tea.

Rebecca M: That sounds tasty. How about you, John?

John: I have a blackberry green tea.

Rebecca M: …and I’m trying the berry blush black tea which I’m finding smells good not warm, but warm… I’m not sure about. It’s kind of weird.

[LAUGHTER]

John: We’ve invited you here to talk a bit about the FLEXspace project. Could you tell us a little bit about what FLEXspace does and how it’s used?

Lisa: FLEXspace is an open repository, free for use to all of higher education, and now K-12. The idea is to make exemplars of learning environments available for people to ideate from, to share best examples with each other, and to benchmark against peer institutions, and also to just use it as a general reference on how are we doing? How can we think about improving our learning spaces in general? Money isn’t falling off trees for higher education right now and we’re certainly all under a great deal of scrutiny in terms of how those precious resources are spent…. and if you go back 20 years or so it was not uncommon to go visit a peer institution by driving to the next state or perhaps flying a small team of people to another institution to really understand how they developed one of their spaces. One of the goals was to create, essentially, a virtual field trip to enable people to visit each other’s campuses and look at those spaces, not just by the visual cues, but also to dive into the details of those rooms in terms of: the type of equipment that’s used, the fit and finish of a particular space, how something may have been planned for. We don’t include detailed budgets but we include aggregate budgets if people are willing to share that information. So, for example you wouldn’t drill down to the level of the cost of a particular piece of equipment, but sometimes people share the aggregate cost of all of the A/V and information technology resources in a room… or the cost of the furniture… or the cost of the entire rehab… things of that nature. FLEXspace is actually an acronym, it stands for the Flexible Learning Environments eXchange, and we have found that people think of FLEXspace as flexible furniture… and interestingly enough we always thought of the emphasis being on the exchange, small e, large X,because it’s all about sharing spaces and best practices within an online community of practice. It was created by educators for educators.

Rebecca F: … and when I think of FLEXspace, I have come to realize, talking to different users over the the years now, that it really centers on the community. So, when I think of FLEXspace, I think of it as, from the users’ perspective, why does somebody come to FLEXspace? …and mainly it’s to get ideas… advice… to get assistance… to help make decisions about a learning space project. FLEXspace provides that community and provides that support through detailed examples of learning spaces, other resources, best practices, and so forth… and also that community network that help connect people who have questions with those who have experience and advice and answers.

Lisa: Yeah, FLEXspace is a large global community. Think of it as a big portal that serves as a one-stop shop for all those best practices. It’s considered an open resource. It was always developed as an OER in mind but we also wanted to make sure it was password-protected, so everybody within the community felt safe sharing within the community… especially if you’re going to share examples of rooms that might not be ideal rooms.

Rebecca M: You mentioned it’s a global community. Can you talk a little bit about who is a member and how one becomes a member?

Rebecca F: We’re super excited that we continue to watch our user membership grow week by week. Right now, we’re proud to say we have over 3,000 registered users and those individuals are from about 1400 different unique institutions from 45 countries. So, we’re all pretty proud of that.

Lisa: You become a member just by going to the website flexspace.org and once you receive a login you can go in and browse the collection… It’s a really neat tool. It really is designed to be the one-stop shop for people that are either designing… envisioning spaces.

Rebecca M: How did FLEXspace get started? It’s such a big community now.

Lisa: This is such a neat story. We were part of a task group… the Faculty Advisory Council on Teaching and Technology… when our Provost walked in and said: “For goodness sake, we are spending millions and millions of dollars each year at campuses all across the state.” We have 64 campuses… we’re the largest most comprehensive system in the United States. So, when you think of everything in the range from community colleges all the way through research universities, we’re all under the governance of SUNY… and Provost Lavallee was looking for more efficient ways for us to share ideas with each other… and he said “if we could even share pictures with one another, that would be helpful.” So, a small team of us got together and said: “Well, how the heck are we going to share information with each other? Nobody wants ads pushed to them… nobody wants to feel like their data is being collected…” and we were talking about that long before the Facebook challenges that have recently come to light. So, we ruled out the common platforms like Facebook or Flickr or any of the open tools that were available back when this was starting up. So, that was the impetus to get everything started. The first thing we thought through is what is it we need to share. Well, we knew we needed to share photographs of rooms, but we also figured out it would only have value if there was a lot of data and interesting details about the spaces… and then that only goes so far, because you really need a point of contact to call someone that was involved in the space… to ask those more detailed questions: “Would you use this particular piece of equipment again? Would you work with this particular group again? Tell us more about why you chose a particular path forward.”

We decided to just collect the details, figure out the attributes, and it happened that we stumbled into a partnership with Artstor… and at the time they were interested in that OER element as well. They were looking to empower campuses with the ability to create and curate their own collections. So that’s how we got started.

Rebecca M: It’s a neat story. As a designer and an educator I can see so many different ways that a system like this could be used, but I have some questions. As an instructor I know that spaces can impede certain kinds of learning activities and then spaces can also enable certain kinds of activities. So, I would imagine that instructors coming to the table with the kinds of things that they’d like to be able to do would be an important way that they’d be able to search images and then ask those same questions that you’re mentioning about how effective has it been for those kinds of learning opportunities. Can that be done?

Lisa: Yeah, thanks for setting that question up so nicely Rebecca and I’m sure my FLEXspace partner Rebecca will dive into more detail on this… but when we were talking with people throughout the community, we realized that the biggest challenge people have in spaces are looking at space development through the lens of their own expertise. Now, you just laid out the faculty expertise, but a facilities planner or an architect is probably going to view how to design and build a space differently and, of course, the A/V and IT integrators are certainly going to have a different set of concerns when they’re thinking about the integration of the technology tools in a particular classroom. For example, an architect may implement a more cost-effective air handling system, but the a/v integrator will be concerned that a particular space is going to be used for video conferencing and they don’t want any air handling noise to come through the microphones. So, you need to have that communication to say: “Alright, are we going to use ceiling mounted microphones? Are we going to use desktop table microphones? Are we going to use a different type of system? Does everybody in the classroom need to be heard?” So, there’s all sorts of opportunity to think through from the perspective of your expertise if you’re a teacher or a technology integrator or an architect.

Rebecca F: Yeah, and I’ll add to that. We’ve been hearing different user case scenarios when we go out and talk to folks. We show them FLEXspace and how we intended it, and then they tell us: “Oh, wouldn’t it be cool if we could use it like this?” …and so we’ve been really excited to get these examples. For example, there are some campuses that are saying they want to provide a resource for faculty. Let’s say you’re a faculty member and you are going to be teaching in an active learning classroom. You’ve never been in the room and from the comfort of your own laptop you can actually log into FLEXspace and look at detailed photographs, even a 360-degree view of the room… details about exactly what equipment is in the room… moveable tables and chairs… and whiteboards, and so forth… and so you can see what’s in the room. It kind of sets expectations, and then the campus can also upload instructor support materials. So maybe there’s tips for teaching in an active learning classroom, or a job aid, or instructional how-to quick sheet on how to use the interactive display or set up some of the equipment in the room. So, some campuses want to use FLEXspace in that way. One person suggested that they would want to use FLEXspace to help their own classroom support technicians. Let’s say you’ve got your technician and they need to go out into the field, because there’s some sort of piece of equipment that’s broken. They can log into FLEXspace… they can pull up all the specs that are in that room… they know exactly what model of projector is in there before they even walk across campus. They know what to take… what’s in the room… and so forth. So, we’re providing the resource and then we’re excited to see how people will use it. S

John: Some campuses are using it then for their own internal purposes as well as for sharing ideas… in terms of providing information about what’s available in each room.

Rebecca F: Exactly.

Lisa: We’ve learned a lot since 2011. When the task group launched in 2011, we were able to pilot in 2012 and then we actually had something into beta by 2013. and by that time we had received some attention from Herman Miller. We had talked with them at a couple of conferences and asked if they’d be interested in helping us to develop this tool. So, they gave us good ideas and as we went to the Consortium of College and University Media Centers, the CCUMC conference, they actually invested in the platform and helped us get it off the ground because they realized the benefit this would have for their members. Then ELI got involved from EDUCAUSE. …and SCUP, the Society for College and University Planning… and each one of those groups has their own tribe, if you will. So, SCUP tends to have a lot of high-level campus planners, people that are working on the financial resources side… and the architects. Of course, EDUCAUSE tends to have more teaching and enterprise level IT support people. All these different voices coming together really helped refine the taxonomies and we had over a hundred volunteers across the country pop in and help us to create and populate these room attributes that describe these rooms in great detail through the lens of each of those expertise areas… be it the faculty and the pedagogical perspective… or the architects and facility planning perspective… or again from the IT and A/V and academic technology perspective.

Rebecca M: I’m really familiar with the Artstor platform given my my background as an artist. I’m envisioning images and a database and all these different ways that you can search metadata…and you just mentioned taxonomy. Can you talk a little bit about some of the different ways you can search and find things? What are some of the attributes that are available to search?

Lisa: Historically, it was all focused on peer benchmarking. If you wanted to look up a classroom you would go naturally to one of your peers and see if they have any information uploaded and then you’d start to look at the type of teaching that was taking place in the room or the type of curriculum. Rebecca why don’t you take that one because you can speak more directly to the new portal under development.

Rebecca F: Before I even came on board the project there were a lot of smart people on this that came up with this taxonomy to figure out what were the different dimensions… aspects of the space so we have these main buckets if you will of descriptors that are looking at the layout of the space: the furniture, technology equipment, other kinds of furnishings in the space (like whiteboards)… different kinds of equipment in the space, even the facilities aspect. so for instance the ceiling… floor… wall… color… maybe it’s a certain kind of LED lighting in the space. We have all of those aspects and then, in addition to that, the types of learning activities that are supported in the space. That might be… if this space is used for active learning, a small group collaboration performance space, and so forth. As we evolved the design into this what we’re calling FLEXspace 2.0, after doing even more user experience research, we discovered that there were a handful of data descriptors that we could require and that way we provide some consistency for the data that’s in there… and then users can open text search the entire collection. They can also use filters and that’s for that required data came in so they can filter on the type of space. So, for instance, it might be an active learning classroom… general purpose classroom… media or computer labs… science lab… makerspace… even informal spaces like a Learning Commons… transitional area… performance space… that sort of thing… and then we can also search on seating capacity, because we found that was very important for users. If they wanted to search on examples, but they’re looking for a space that would accommodate a hundred students… or they might be looking for a space for 30 students… so we have filtering capabilities in that way. You can also filter on “what’s the project scope?” Is this a completely new construction project? Is it a renovation? Is it a room refresh? You can search on that and also, as we’re expanding beyond higher ed examples, we’re now building the community in k-12… you can filter on the institution type… and so we learned that this was important. Is it a private university? public university? is it a community college? a k-12 environment? and so forth. So, there are a lot of different ways you can filter and search the collection.

John: It might be helpful if you talk about perhaps some case studies of how some specific campuses have used this to help move forward… moving away from the old traditional classroom structures.

Lisa: Yeah, that really is the easiest way to explain the value of FLEXspace, John. Thank you. One of our early core team members was working at Iowa State at the time. They had a very large project underway, and in addition to FLEXspace being a very useful tool, another tool was under development at the time out of a constituent group out of EDUCAUSE – called the learning space rating system. So, their concept was to imagine the LEED environmental credit system and if you could apply that in a quantitative manner to the learning potential of different types of spaces you could actually measure the potential effectiveness of a space… and it’s nice because it’s quantitative… because then you can rank the condition of the space across a number of different measures. So, that’s exactly what they did at Iowa State, they went through a large audit of a number of classrooms with the LSRS and then gathered a very diverse group of people around the table representing faculty, and the facilities planners, A/V IT integrators, I think they even had a couple of students, and of course the financial people were represented. They said “Alright, how are we going to work together to have a conversation about the type of space that we need to build. We have an opportunity here.” So, they parsed out groups within the main group and said “everybody get a FLEXspace account” …and you know what’s going to happen is the faculty member knows of a very cool space at another campus, so they’re gonna go in and immediately search through FLEXspace to find a really cool classroom… and meanwhile the biology professor is going to do the same thing… and you might have a GIS teacher going after a similar type of search… They found, when they came back together, that a number of the rooms they were looking at may be different, but the room attributes were very similar. Now, you can start to have a real discussion. Why did you like that particular space? or better yet “Wow, we each picked the same space, why? …and they started to narrow the selection of the spaces they were looking at in order to put in front of their executive leadership. These are the types of exemplars. These are the types of spaces that we’d like to build here on our campus… and this is why we think they’re valuable.
Another good case study would be from SUNY Geneseo. SUNY Geneseo went through a very similar process where they were looking at recommendations from faculty. They started by saying “Gee, these 17 rooms on campus are in need of refresh…” and the more they thought about it they said “Well, let’s do something similar to what Iowa State had done. They took the same approach as Iowa State where they did an audit with the learning space rating system and then went through, ranked their classes, considered other metrics like enrollment, and decided to invest in three or four different classrooms to bring them up to beautiful new classrooms… active learning spaces… things of that nature… but they found a very similar process… where a group of people around the table using FLEXspace to serve as the glue of the discussion about why they like certain classrooms and what the attributes were that they felt were important to have on their campus.

Rebecca F: …and I can add a couple of short examples. When I started working on the FLEXspace project, I was focused also in my role… I teach at San Diego State… and so I was working with some of the Directors of Academic Technology at some of the different California State University’s campuses. I had one instance where the new Chief Learning Officer there was responsible for designing and building and planning some new active learning classrooms, and he found himself in a position of not being so familiar with what was the latest and greatest going on in this arena. So he reached to FLEXspace to get prepared… sort of get up to speed on what are other campuses doing in this area before he met with the architects, so that he didn’t go into that meeting with a blank slate. He could go into FLEXspace, look for some examples, and then bring those examples to the meeting, and say “Okay, we’d like to do something like this. How much would that cost?” or “Can we do that here? Is it feasible?” So he was able to get some examples and feel a little more informed going into those discussions with the architects and the facilities planners.

…and another example… There was a meeting of Library Directors that all met here in San Diego State on campus, and they were going to be discussing how to convert some of their library spaces into group study spaces… maybe maker spaces… and so forth. …and so the Director of Academic Technology there at San Diego State, he went to FLEXspace and started searching through examples of how different libraries across the country had converted their spaces. You can go in there and you can tag your favorites, group them into a collection, and then he took that collection of maybe a dozen examples and brought that to the meeting so that he could show those to this library directors’ conference and say “Okay, here are some examples…” and then used as a conversation starter to say “What are some things that you like that you see here? How does that spark other ideas for your campus? and so forth.

Lisa: We’ve been really lucky, because we’ve had experts all over the country volunteer time to help us refine this system. I think we’ve got a bit of a tiger by the tail because it’s clearly meeting a need. The combination of the learning space rating system with FLEXspace is invaluable in terms of having two free tools to assist people at a number of levels on campus to have an effective learning environment.

John: …and it’s won some awards, hasn’t it?

Lisa: Oh, thank you, yes. We did receive the Innovators Award in 2016 from Campus Technology, which was really cool, because it was a nice way to acknowledge all of the efforts, not just of the people who participated in developing the system, but at the end of the day if you want to keep a tool like this free to all of higher education or K through 12 or whoever wants to create an account to use it, you have to find a way to support it…. and we have some very generous sponsors that have been helping right from the get-go. Certainly Herman Miller played a huge role in getting us off the ground and there’s been a number of other sponsors that have stepped up and made it possible to keep this free… and we’re challenged to continue to keep it as a free service moving forward because this new portal development is pretty pricey… but we think we’re going to have features that are really really going to surprise people. Do you want to talk more about the sponsor relations, Rebecca?

Rebecca F: Yes, so we’re really grateful for all of our sponsors and we’re excited to be able to even recently announce that Herman Miller, in addition to being our founding sponsor, they are going to continue as our Platinum premium sponsor moving forward with the new FLEXspace portal. So, we’re really excited about that. What’s been great is that our community of academic users welcomes our industry partners. They see them as trusted partners. They want them to be involved in the community, so they don’t see it as some sort of direct advertising… and it’s not that at all. It’s really that this is community of experts from industry and from academia who are coming together dedicated to improving learning spaces and sharing resources that way. We feel like we couldn’t do it without our partners like Herman Miller. We also have our Gold Partner Computer Comforts who is another furniture manufacturer who’s come in this year and they’ve really taken a leap of faith because, as we build the new portal, they are supporting us and they see the value of FLEXspace. They see the value for the education community and they want to be a part of that. So, other sponsors we have including Shaw Contract, some AV companies (AVI-SPL, Crestron, Wolfvision, MediaSite, FSR, Sony) and another furniture manufacturer AvinEd, they’ve all stepped up and given their support to FLEXspace to make sure that we are continuing to thrive and grow and provide that free resource for the academic community.

John: You started on Artstor. Are you still using the Artstor platform or has it moved to a new site. You mentioned the transition.

Rebecca F: We started with Artstor and we were so grateful for their support. They had this wonderful platform that really focused on images and the descriptive data. We were able to start with Artstor and use it for FLEXspace and then over time we realized that it wasn’t necessarily built specifically for the needs of our users… and so we migrated away from Artstor because we did this user experience research. We found out that our users, first and foremost, needed the platform to be mobile. They needed some other collaborative features in there as well, and so now we’re in web development with a different platform. We’re really building it from the ground up at this point. We’ve got some new features in there that support collaborations. So, for instance, let’s say I’m at San Diego State and I want to upload an example active learning classroom at our campus. I might not have time or I might not have all the details to describe the space. so one of the new features in the platform is that you can add collaborators or co-editors when you’re uploading a space. I can start it. I could be out in the field with my mobile phone or tablet. I could snap some photos, upload it, and create a FLEXspace entry, and then I can tell somebody from my team “Hey, will you go in there and add the details because you’re really familiar with all the A/V that’s in the room…” and then I can add somebody else from my team as a co-editor and say “Hey, can you describe the furniture that’s in the space?” I can even ask a faculty member who’s teaching in the space and add them as a co-editor and and say “Hey, will you go into this record for the active learning classroom, room 101, and tell a little story about how you’re using the space? What’s working? What would you do differently? How are the students responding to the activities?” and so forth. So, you can really have more collaboration and, like Lisa was saying earlier, have many more perspectives to describe the space. it makes it even more useful when you’re reading the examples.

Rebecca M: I have to say as a web designer I got really excited because you’re using all the best practices for designing a platform like this so kudos for her user centered design it makes it better for everybody.

Rebecca F: Yeah, absolutely. We thought that was really important because it took on a life of its own and it really started gaining momentum and growing but it really hadn’t been designed with those user needs in mind. We started the deep dive into the user experience research with the higher ed community and, really, it was more focused on users in the United States. We also have an International Committee as part of our core team, and that’s one of our next steps. We want to do a little more research into what are the international users needs as well. Obviously, there’s a lot of overlap, but there are some differences in what their needs are… and then we also want to take a deeper dive into what are the needs for the K-12 community. Again, there’s overlap, but there are some subtle nuances there. We’ve been really fortunate in this dedicated community, like Lisa was saying, people who are contributing their time. We have some new representatives on our core team from the K-12 community as well… and we formed a partnership with ISTE.

Lisa:Now we have a custom portal and we’re so fortunate that Xennial Digital has really gotten behind this project with their expertise and their web portal development because they have experts and they’re able to turn this into a very dynamic community of practice in a way that, frankly, I don’t think we would have been able to afford any of this without their support. So, we hope that this is serving their purposes as well and showcasing what they’re able to do with a portal like this.

Rebecca F: FLEXspace started as a place to showcase exemplar spaces, but now we’re hearing more and more that in higher ed and K-12 they want to use FLEXspace to document and catalogue all of their spaces… so not only the shiny new innovative spaces but also the more typical spaces on their campuses so that they can have this classroom directory and so they don’t have to build their own. There are some campuses that have built their own directories and they have the support to do that, but we’ve heard time and time again… there are many smaller campuses or those that just don’t have the resources and they don’t want to build their own directories, so they can they can use FLEXspace…

When we’re presenting about FLEXspace, hands go up and they say: “When is the upload feature going to be ready? Because right now I have a Google spreadsheet with all of the data on our classrooms and I want to get it up into FLEXspace.” We’re really excited about that, and that takes it in a new direction as well.

So, you can search for exemplars, but you could also use it in the in this other way… to show these documented spaces to your faculty who are teaching in them, to your tech support folks… maybe you’re looking at this as sort of an audit of your classrooms – and then in conjunction with using the LSRS you might rate your different spaces… and some of them are in dire need of improvement… but you’ve documented those… you’ve got photos… you’ve got details… you can do the LSRS rating… and then you can go in and improve the space… and then upload the example of the renovated space… so, sort of the before and after photos as well…

Lisa: One of the really nice features of the learning space rating system is that it quantifies the value of what spaces need attention… and once you have that ranking it tends to take some of the politics out of the equation. We’ve heard from a number of people that once they go through that quantitative exercise, then you can dive into FLEXspace and start playing with the feature sets.

Rebecca M: You’ve been talking about a new system, and I was wondering whether or not that was actually implemented or not. Is that what’s up and live now or is that something that’s in progress?

Rebecca F: We have a phased rollout of the development of the new portal. Right now, it is indeed live and you can request an account. Go to FLEXSpace.org, request an account, you can log in and you can see you can filter and see all of the examples of learning spaces that we migrated over from the old system… and then, even as we speak, we are in development of the next feature set… which is the case study template feature that allows you to upload examples. Right now, you can look at examples and in June of this year you will be able to upload your own examples. and we’re also launching the membership directory and community forum. Prioritizing all the different features that we have on our roadmap, we felt like the community aspect and then obviously being able to upload spaces were the biggest priorities and so that’s what is coming soon.

Lisa: A lot of people are familiar with the website Houzz – H-O-U-Z-Z….

Rebecca F: Houzz or Pinterest… those kinds of tools.

John: I’m one of those people who is not familiar with Houzz. Pinterest, I’ve seen.

Rebecca M: Yeah.

Lisa: Well, we think that this portal that’s under development, FLEXspace 2.0, is going to rival the features that people are already used to seeing in applications like Pinterest…. and Rebecca can speak to the idea boards and the tool kits. It’s going to be a really nice way to keep track of your own ideas… pull together a collection… and then perhaps invite others to come in and ideate in that space with you.

Rebecca F: Yeah, so you were asking what are some of the things that you might find in FLEXspace. Originally, the collection consisted of examples of spaces and so that meant a lot of photographs, a lot of detailed tags and descriptions, and then also users can upload, let’s say, a floor plan, a detail spec sheet, maybe details about the wiring layout of the room, or the wall colors, and so forth… so, lots of details and lots of images. But, now we are expanding those kinds of resources. We are including more kinds of spaces, not just formal learning spaces but also informal learning spaces and different types of settings… so, more k-12 examples, museums, libraries, more kinds of details. You can upload video, 360 images, more details about the case study of the space (including evidence of impact and efficacy). We’re also encouraging the upload of more kinds of support resources… research papers, white papers, things that don’t necessarily have to be tied to one particular space but more broadly useful resources.

We’re also building in more ways to connect. You can add collaborators or co-editors to your space. We’re offering the member profile and membership directory, so you can go in there and see who else at another campus is working on learning spaces. You can connect with them. We are starting the community discussion forum. If you have questions about a particular challenge that you’re having you can participate in that discussion forum. We’re also starting to use this new tool called an idea board. It’s sort of like that Pinterest board. You can tag your favorites, but you can also create folders or collections. Let’s say you’re thinking about creating a new makerspace or a STEM lab. You can start gathering inspiration and create this idea board called “ideas for makerspaces” and you can keep your idea board private… you can add your team members to it so that we could all be adding to the same idea board. Let’s say we’re working at San Diego State and we want faculty to be giving input… we want the technology folks… the facilities folks… maybe it’s housed in the library… we want somebody from the library staff. We can all be added as collaborators and all be adding ideas to this idea board. so that when we talk about planning our makerspace we can see what everybody else is finding is inspiration.

Lisa: One of the guide posts with our sponsors and they both brought this up and agreed to it is we know that if anyone were to use the portal to pull data or use this in some fashion to solicit to the members that we’d all be dead in the water. This is a safe community forum place where you can interact with people who understand the details of new products or new innovations that they’re offering, but you aren’t going to be hounded inside the portal in any way. We know it’s a big concern these days about the amount of email everyone’s getting. Our sponsors get that people don’t want to be hounded. We’re trying to find that fine line between “Look folks, if you want it to be free we have to provide something of value back to the sponsors…” and I think everybody understands the trade-off in that environment. So, the sponsors are going to have access to the tracking and the analytics and the things that are valuable to them, but we want to keep everything upbeat and positive around that community.

John: So, both groups benefit… the sponsors as well as the participants because the people who are looking at these spaces are going to be designing things and getting access to information about the sponsors and information from other users, I think, could be quite useful.

Lisa: The sponsors actually, depending on the level of sponsorship that they participate in have a space within the portal to talk about their new products and to be able to respond and interact with people on the platform within the portal. We think it’s a useful communication tool and the vendors are welcome to be a part of that community of practice. That’s what’s made CCUMC so successful over the years, is having that relationship with the vendors in a safe place where people can exchange ideas freely without feeling pressured.

John:okay so this started off as an initiative from a small committee and it’s grown into this huge collaboration space. How have you been able to maintain momentum of keeping this going?

Lisa: Adrenalin?
[Laughter]
The momentum has been carried on by the community. I think the most exciting thing that both Rebecca and I experienced when we were at EDUCAUSE this past year was walking into a room and hearing other people talk about FLEXspace and talk about the value of FLEXspace and how they want to be a part of it without us initiating the conversation. That was a groundbreaking time for us. It was very energizing.

Rebecca F: Yeah, I agree. We’ve been on the road for the last couple of years. You mentioned we did get the Campus Technology Innovators Award… I believe that was in 2016. We’ve had a lot of support from affiliate organizations like UBTech this year. We’re a program sponsor at that conference and we were a program sponsor last year. We’ve gotten a lot of support and encouragement from Infocomm, Campus Technology, the folks at EDUCAUSE. We’ve been doing a lot of outreach by giving conference presentations, workshops, webinars, and really it also comes from I’d say a very dedicated network of our colleagues who are all committed to improving learning spaces.

We regularly meet with architects, A/V integrators, faculty members, researchers, leaders in academic technology and information technology. We have our core team meetings. They are out there in the field being FLEXspace champions. They spread the word, they encourage others to take advantage of FLEXspace and so forth… and to echo what Lisa said even this year just recently we were excited that somebody from the ISTE organization (that’s the International Society for Technology and Education, they’re a big professional organization for the K-12 community) they reached out to FLEXspace and said “Hey, we want to partner with FLEXspace. We see this as a valuable resource. We don’t want to reinvent the wheel for the K-12 community. Can we partner?” …and so we’re planning to do some sort of train-the-trainer activities with some of the ISTE Learning Spaces Professional Learning Network Committee. They will learn more about FLEXspace and how the K-12 community could benefit from it… and then they are going to go out at their annual conference, and through their local chapter meetings, and really be proponents of FLEXspace. So, it really has garnered the support from the community and that’s why we want to keep listening to the community’s needs to make sure that we’re providing a very valuable resource that they want to keep going back to again and again.

Lisa: …and a shout out to the vision of the people at the systems as well because SUNY has been very generous with allowing this to grow… enabling us to put time into it… which is of course the most valuable resource and to the Cal State University system which has provided generous support to enable this partnership between SUNY and Cal State and Rebecca and myself.

Rebecca M: Well, this has been really great. I find it really informative and I look forward to jumping in and trying to use this FLEXspace platform for some of our upcoming renovations on our campus.

John: I know a number of people here do have accounts and have been using it.

One thing we always ask is: “What are you going to do next? In this case, where is FLEXspace going next?

Lisa: We think that the continuous quality improvement cycle is going to be at play in a big way here, because once people get in to the new feature sets and see all that they can do, we’ll probably spend about a year spreading the word about the new features… and then I have no doubt that people are going to have fresh ideas about how to improve it. So, as long as our sponsor community continues to support us, we will continue to improve it in any way that the imagination seeks.

Rebecca F: Yes, and I will leave you with this one statement here. Our vision is that we want FLEXspace to become this one-stop-shop for best practices, detailed examples, and a community dedicated to improving learning spaces around the world. So, that is where FLEXspace is going in my mind.

Rebecca M: Well, thank you both for joining us and taking some time out to talk to us about this great endeavor and all the time you spend working on it.

John: Yes, thank you.

Rebecca F: Thank you
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca M: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer. Audio editing assistance provided by Nicky Radford.

34. Flex courses

Working towards a degree for some students can be a struggle as they balance full-time work, families and coursework. In this episode, Marela Fiacco, a Healthcare Management Instructor and Curriculum Coordinator at SUNY Canton joins us to explore options that give students greater access to courses and co-curricular activities. Dr. Fiacco is the first instructor at her institution to teach a flex course, a modality in which students may participate either in person or remotely.

Show Notes

Transcript

Rebecca: Working towards a degree for some students can be a struggle as they balance full-time work, families and coursework. In this episode we’ll explore options that give students greater access to courses and co-curricular activities.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today our guest is Marela Fiacco. Marela is a Healthcare Management Instructor and Curriculum Coordinator for the Healthcare Management Program at SUNY Canton. She is also the first instructor in this program to teach a flex course. Welcome Marela.

Marela: Thank you, thank you for having me.

John: We’re happy to have you here. Are you drinking tea?

Marela: I’m drinking water.

Rebecca: It’s a nice, healthy choice.

John: Our teas today are…

Rebecca: I’m drinking English afternoon tea… again.

John: And I have Harry and David’s Bing Cherry Black Tea.

Marela: Oh, that sounds yummy.

John: It really is. It’s hard to find– you have to go to a Harry and David store or order from them online, but it’s a Republic of Tea tea that’s custom made for them.

You were the first instructor at your institution and one of the first in SUNY, I believe, to teach a flex course. Could you tell us a little bit about what a flex course is?

Marela: Certainly, at SUNY Canton we received a grant from SUNY system, I believe, and this is by our Dean of Instructional Technologies and the idea was born to create what we called at the time, probably about this time last year, we called it a converged modality classroom delivery. We can call it a flex class or a converged modality. To us at the time, it was a mix of face-to-face and an online class, so we have students in a face-to-face class and students in an online class. Students in an online class can watch the video live or they can watch a recording later. Face-to-face students, if they are not in attendance, could watch a recording later. It’s really a mix, and when you say a flex course that’s what it is. It provides flexibility.

John: A flex course, then, is a combination of a face-to-face and an online class. Are students free to choose the modality, or are they enrolled in one section or the other?

Marela: Since this was a pilot, it was really difficult for students to understand and determine when they were first registering for a class. We have since gotten a little bit better and created two different sections and they understand what it is and they understand what they’re signing up for. At the time, we just provided a face-to-face and an online section. Those students who were purely online and couldn’t come to class physically, they chose an online version. Little did they know that they were going to be in a converged class, or a flex course, but at the very beginning I had a video done where I was explaining what it is and should they choose not to participate they could have picked a different section… but, this was just something added that they could benefit from. It wasn’t anything that we were taking away from their online experience.

John: So, if they were in the online course, could they attend face-to-face classes?

Marela: They absolutely had an opportunity to participate in a face-to-face class if they chose to do that. But, I think, for the health care management program, we advertised it as a hundred percent online program, and so most of our online students choose online because of the convenience. But we also do have students on campus and commuters who come to class face-to-face.

Rebecca: What was it like teaching a flex class?

Marela: Quite frankly, it was a lot of prep work at the very beginning. Well, first I thought: “How am I gonna do this? I’ve never done anything like this. It’s going to be a lot of work and then I thought “Well I have to start from somewhere.” I taught this class both online and face-to-face in the past, so that helped. We teach in Blackboard. I took the online class and just really took a hard look at it, and thought to myself “How do I make this class more user-friendly for both groups. I modified my online class. I of course added both on converged modality. We have videos for each lecture capture, so each time I’m in class and I’m lecturing I am using those to upload into those weekly modules for those online students to watch and for livestream. The thing I really wanted to achieve is I didn’t want to have it separate. The face-to-face students, I think they had more work than anybody else. They were asked to participate in discussion posts online together with online students. So, for them, it was almost a hybrid. They were supposed to upload assignments in Blackboard and also discussed whatever topic it is… whatever questions that we had for those weeks in Blackboard.

I also created a group assignment that I think was a bit of a challenge for all of us, because I intentionally picked the groups and they didn’t have any say in that and I picked online students and face-to-face students, and provided him with links in Blackboard Collaborate to get together and work on their assignment. That was a challenge for all of us.

John: From your perspective, are you compensated for teaching two courses or is it treated as if it’s one course, in terms of your workload.

Marela: I did get an extra that an adjunct would be paid or that a faculty member would be paid for teaching an extra course, because of the workload. Because this was a pilot because we capped both of the sections at 15 students normally our caps are at 30. So, I did get an additional pay last semester for teaching this converged modality or flex class. Moving forward, I don’t think that they’re actually doing that with other faculty.

John: To get things started, it often helps to give a stipend to encourage people to experiment. In the future, would the combined sections be capped at the same level as a single section would have been?

Marela: They are capped at 30 and it’s just one section.

John: …and then students are free to either attend in person or online. That’s what I was thinking..

Marela: Yeah.

John: …because that’s what I’ve generally heard about flex courses. I just haven’t seen many examples of them in practice yet.

Rebecca: What was one of the biggest challenges you had as an instructor? and what might you do differently next time you teach a class like this?

Marela: One of the biggest challenges was technology, to be honest. Just working out the kinks… because this was new for all of us, including the online support staff. You come in and the camera is not working… or the sound’s not working… videos not working. We might be missing a livestream or we might be missing a recording. It was the technical difficulties that were really the hardest. I struggled with attendance in the face-to-face class, because now students are thinking: “This is super flexible, I don’t have to show up.” So, I struggled with the balance between allowing them to have flexibility and the fact that you signed up for this class. So, you want to provide students that flexibility if there is a snowstorm and we lose power and whatever it might be and if they’re traveling a distance. So that was kind of the fine line, but I think each instructor determines their own attendance policy, so every one of us will approach it differently, I suppose.

John: Did you use any tools such as polling or quizzing in class where students had to participate either virtually or physically?

Marela: No, not this time. Just because the technology was new and I think next time I do it I definitely would want to do that. But at the same time, that is another hard one, because the online students, the reason that they take online classes is for convenience. Most of them are working professionals. In my program, 85% of the students are working professionals. The classes… let’s say nine o’clock in the morning… well, they can’t exactly participate live. They do appreciate the recording later at night working on their homework or whatever it might be.

John: So, for online students synchronous attendance isn’t required? It’s an option but not required?

Marela: Correct.

John: What would you say would be the major advantages that students get from this sort of offering?

Marela: I think the greatest advantage for online students specifically is the lecture capture. If there are any misunderstandings about the assignments, whatever it might be they, they actually get a lecture instead of being self taught if, you think about a purely online class. Another thing too, is it provided greater connectivity with students. I really can’t stress that enough, that it really gave me an opportunity to connect with the online students… one that I wouldn’t have otherwise… because it often feels disconnected from the campus. With this grant, it wasn’t just the converged modality, it was also to connect with online students. We invested money in live streaming our Excellence in Leadership lectures and speakers so that we can bring online students and have them participate in different things on campus. We really wanted to create that connection and that’s one thing that they really appreciated the most… that was their feedback. They want to see us, that we exist… we’re here… and that we care enough that we want to do this for them. So, I think that was one thing that they really appreciated the most.

John: It created more sense of instructor presence and more of a connection to the institution.

MAREA: Yes, and they feel a sense of community… they feel a sense of belonging.

Rebecca: Did you find that a lot of the students took advantage of some of the extra things that the college invested in, so they could take advantage of those extracurricular opportunities?

Marela: They did. They really did… and we were really pleased with that… and actually the Dean of Students, myself, the Assistant to the Provost, and the Dean of Instructional Technologies we are presenting at the CIT conference and we are reporting our findings, not just on academic side but also the extracurricular and non-academic piece of it… and what is it that students took part in, what they enjoyed the most, what is it that we should continue doing for our online students. We have a large population, and let’s face it, we are all looking to online to look outside of local and geographic area because our enrollments here are really declining because of the graduation rates in high schools. We are all looking for ways to connect with online students on different levels to make them feel part of our campus and our community.

Rebecca: The extracurricular piece seems like it’s one of the most powerful additions to this particular opportunity because I think you’re right that the students, when they’re taking a class online and they’re not coming to campus, they miss out on a lot of that intellectual development from these other points of view that we don’t always offer just in the class… having the opportunity to get involved just seems like it would be really exciting for some students.

Marela: It is. It is. It’s very exciting for them. Actually we have had, for instance, one time we had a CEO of one of the local hospitals come and speak… particularly to the networking and career goals and things like that… and the students were very interested… and actually emailing and asking when is the live stream going on? Usually, these are in the evenings… part of our Excellence in Leadership series. They really wanted to take part in that… listen and understand their career options. or whatever it might be. So, those are some of the things of value to them. Actually, I was just talking to the President the other day… we had our scholarly activities, where students come and faculty come… present their poster presentations… and present their research and such… and I was just talking to him and I said: “You know, it would be great to involve our online students in these scholarly activities a little bit more, not only our engineering students and nursing students who are here on campus, but because our online students are getting involved a lot and some of them are lobbying. They’re involved in so many different activities… in their own communities and some of them do research, so it would be really great for them to present… to have their posters there and have them on Skype or somehow live streaming, where they can be present or invest in bringing them here or whatever it might be.

John: For the synchronous sessions what are you using as a platform for live streaming the classes? are you using interactive video?

Marela: We were using Panopto. This was purchased by our online programs. So they were using Panopto, which is embedded in Blackboard and then that’s how we’re doing that.

John: We use that here too. We have it in pretty much all of the classrooms and we have a site license for it. It works really well. The only limitation I could see for it in this context, and I’ve experienced the same thing, is it doesn’t work as well for two-way communication. The remote students who are viewing synchronously only have the option of typing in little text messages. They’re not able to interact in real time other than with text.

Marela: That is correct, and we are looking at different ways to fix that. I don’t know if they were able to accomplish that this semester, but you’re absolutely right. Students would have to type in their question. I would have to come back to the computer to check every now and then to see if there are messages there.

John: Because when it pops up, it’s only there for a few seconds, so it’s really hard to see unless you check on the screen itself.

Marela: That’s correct.

John: Just a thought… it might work better if you use something like Google Hangouts or Zoom or something similar for real-time sessions where students who are viewing in real time could actually communicate with voice without having that barrier. We used Panopto for many years here for our workshops with remote participants, and it wasn’t quite the same experience. We switched over a few of the workshops about a year and a half ago, and we switched all of them in the past year to using Zoom and it’s been a much better experience for people who are participating from other cities, countries or just from their offices even.

Rebecca: It’s a better experience for the person doing the presentation or teaching as well, because then you can see what the students are doing. You can see who those other participants are, when they might have questions, if they’re bored, or whatever, just like you can students in the class.

John: Just having those recordings can be useful. We have a lot of ESL students, students who are foreign who sometimes struggle with English, and having the ability to go back and replay parts of the course or look things up while they’re watching it and pausing, or slowing it down to half speed sometimes, is something they find really helpful, and in Panopto you can go back and look to see what portions of the video people were watching, and it’s also could be useful to see “well, maybe I need to explain this a little bit better, next time” if you see that there’s areas where students were going back more often.

Marela: Exactly.

Rebecca: If someone wanted to pursue a flex class what advice would you give them?

Marela: Be flexible.

[LAUGHTER]

John: I think that’s good advice with any form of teaching.

Marela: Really… be flexible. Actually, at the beginning of this semester, the Dean asked me to do a bit of a SWOT analysis for the instructors who are teaching it this semester… and I said: “Be ready to have technical difficulties. Be ready to laugh it off and not get caught up in it. Be ready for students to not show up. Be ready for multiple things.” So really, be flexible and allow for it to take its own course, I suppose… and just try to be as accommodating with students as possible. They don’t always understand it. They like it and I just keep going back to this connection with students. Just keep that in mind. It’s worth it. It does take a little bit more work outside of class and you have to be ready to be there a little bit early to get set up and stay after the class is over to make sure it’s all uploaded. Make sure that you give a lot of instructions. Revisit your attendance policy, those types of things, but really be flexible for a flex class.

John: You mentioned that you started with your online course and then you thought carefully about how to restructure it to work in this environment. What were some of the things that you did more of? Some things that you trimmed back? How would you characterize the main changes you made so that things would work better in this environment?

Marela: To be honest with you, I did not alter the online course as much as I thought I would have to. The biggest change was capturing the lectures. The one thing that was challenging was the discussions. The online students were not participating, so they are participating afterwards when the discussion post is due. Face-to-face students may be having a discussion in class. My biggest worry was how do I now capture that and make it fair because they are participating, there is a classroom participation. So, I then additionally was asking my face-to-face students to almost hold on to those discussions. Go back after class… go into the blackboard… and post those, and write about it. So, I had to think about the grading policy actually the most… and make sure that they receive credit for their thoughts and discussions in addition to what they were providing in class. The assignments I altered a little bit more. I wanted him to know that I valued their patience with the group assignment and so I gave more weight to that assignment… and I know some of them really saw a benefit to it. There was a student or two who complained about it, but I said: “Well, think of it this way…. If you are in the real world and you have to work with a facility that’s hundred miles away and you still have to connect and you’re working on this project. There is logic behind my madness, and why I’m asking you to do this.” That was one of the things. But, in terms of the material, the content what was being taught wasn’t anything different. I would say probably the assessment piece was a bit different and then capturing the lectures for students.

Rebecca: I’ve been thinking this whole time as you’ve been talking about what the classroom experience is like versus an online experience and so I was thinking about my own classes…. thinking about: “Well, how would I change a hands-on activity that’s in class so that people could participate in a different time and space, and then make sure that everyone can come back together and see what the results were and share out.” One of the kinds of activities that I do a lot in my web design classes is little code examples where they’re practicing putting code into play… and I guess what I’ve discovered is, over the course of this semester, I started doing things in a more flexible way because I realized that my mix of students was a lot more diverse than I had been in the past. I had students from different majors that I hadn’t had in the past before and to accommodate that I started giving out little exercises… giving some time in class… giving some tips out in a lecture… that I could easily have shared out to online students and then having students finish the exercise for homework, taking the tips into place, and then coming back and going over the example or going over the results the following class period. I think that over time I ended up having to implement something that was flex-like just because my students were a lot more different from one another than they had been in the past… and so, although it wasn’t because they were in different places, I think this strategy might work in other scenarios.

John: …and actually I’ve done a couple of things over the last five or six years too, partly because of the availability of things like Panopto. I teach a class in the fall with generally 360 to 420 students in it, and it’s always offered on Tuesday-Thursday and we have this Thursday Thanksgiving holiday, and a lot of the buses leave late afternoon on Tuesday and often there aren’t a lot of students in class… and a lot of classes on campus end up being canceled effectively or not covering anything substantive on that day… and I never wanted to miss that class. So, what I’ve been doing is in my class we use clicker questions, but now that over the last several years over half of the students use mobile apps for it, they don’t have to physically be there. So, each Tuesday before Thanksgiving when I have class I may have half or two-thirds of my students sometimes even three-quarters of them not physically present but I’ve had up to 150 students who’ve been watching the video stream on Panopto and participating in the clicker quizzes all through the class. They don’t have to miss the class… and I’ve had students who are on vacation… I’ve had students on cruises… I had students participating on that Tuesday class while they were on a family vacation in Florida, for example.

Rebecca: Or on the bus somewhere…

John: …and it’s worked pretty well… and actually, more generally, I have students who are at various sports events in my other classes, where they’re going to be away traveling on a bus or they’re going to be out of town and if they tell me in advance, if it’s not a class that I regularly livestream, I’ll just click the little button in Panopto to set up the live stream, especially in classes we they have clicker options… and they can participate and even though some of them use a physical radio frequency clicker, they have the option of getting two weeks of free use of the app version of it. So, that allows them to participate from wherever they are, and it’s essentially a mini version of the Flex course. Going back to Rebecca’s comment, the other case where I did something very similar, in some ways to yours, is I taught a COIL course which was jointly offered with an instructor in Mexico. My class was an online class, her class was a face-to-face class, but we had some common components where they were working in groups. Most of their work was done asynchronously. But they worked in small groups synchronously with each other and there was a lot of benefits from that… and the students really enjoyed that community, especially the cross-cultural community where they were working with students from another country. There’s a lot of advantage of this modality that makes our offerings more available to a wider range of students who wouldn’t otherwise be as much a part of the college communities. I think it’s great.

Rebecca: I’m appreciating your advice to be more flexible. As you were talking I was thinking like “What else could I do that would be more flexible in general.”

John: That’s really good advice.

Marela: Thank you.

John: How many classes are now being offered? Your’s was the first class… that was in the fall wasn’t it?

Marela: That was in the fall, yes. Right now we have, I believe, two classes from the School of Business and Liberal Arts. One is a finance course, the other one is economics… and we also have classes in the criminal justice and law enforcement leadership from the School of Health and Sciences and they are using the classroom… and interestingly enough I think some instructors are using the technology in the classroom similar to some of the things that you described… whether it’s clicker, whether it’s using it for different group projects and things like that. My class was just Intro to Healthcare Management, it wasn’t anything super exciting.

John: …as opposed to economics, yes. [LAUGHTER] My students might disagree with that.

Marela: As I was going through the semester, I thought to myself that perhaps greater value in this type of delivery lies in courses such as finance and math and economics where students may struggle with the concept. They really need to pay attention and they really need to be tuned in to the videos and watching it and rewinding it and whatever they have to do to get it and to understand the concept. I think, for them, it may be a little bit more of value versus teaching yourself some concepts that may not be as abstract or as hard to understand. I think even toward the end of the semester it was thinking there are so many things that I would do differently… provided that we might alter the technology and have it where students can be interacting and asking questions. I really wanted that classroom interaction to be better, or to foster more of it, rather than just online students watching it later and listening to other students having a discussion… and then I was thinking “well, how can they participate?” So, there are a lot of ideas. Of course, everything in hindsight is different… some of the things that you might do differently and have them build more of a connection, I suppose, between the face-to-face and online students.

John: But, having your group projects, I think, is a good way of doing that because then it does provide those connections for both groups of students.

Marela: Yeah, they really enjoyed that and I think they learned a lot from each other. Online students are professionals, non-traditional students. About 80-85% of them are already working in the field. A majority of the students on campus are first-time freshmen. I think it was a great way for them to bridge, not only the age gap, but also the knowledge and skills type of gap so students here could learn from the online students.

John: …and we’re moving into a world where people will often be working with people locally but also be working with people remotely, and this type of experience is good preparation for those future work skills.

Marela: Absolutely.

Rebecca: I think the more we emphasize that for students the more adaptable they are and the more likely that they are to appreciate the platform or the methods.

Marela: Yes, you almost need their buy-in and because there are other sections of the same course being offered, you have to sell it to them: “What’s in it for you in this class?” and so, if they really appreciate what they can get out of it, they might be more willing to participate and be vested in that class.

John: How are your colleagues responding? Do they generally enjoy this format or is there opposition to it? What’s the general reaction to this modality?

Marela: I think you’ll hear a mix of both.

[LAUGHTER]

Some of us are more technologically challenged than others and some of the facultyu members when they hear the technology and when they hear what it’s like and all of the little things that can go wrong, they shy away from it and are probably unwilling to try it. Some are embracing it and saying “This is awesome, I can do all these different things with my students now.” You will have the early adopters and then…

John: Yeah, even those people who are reluctant to try new technology often drive their vehicles. They’re not riding a horse. So people come around eventually.

Marela: Yes.

Rebecca: So, we usually wrap up our interviews by asking: “What’s next?” What’s next for you?

Marela: I would like to do a converged modality or a flex class model in some of my upper-level courses and try to get our other faculty in the program to use this modality… especially, for instance, healthcare finance courses where we use simulation. I think those would be some of the things that I would like to experiment with and try that and see how students respond to it and whether we have a good response.

Rebecca: I can imagine a flex class at a lower level being quite different from a flex class at an upper level so it’ll be interesting to see how your experiments go and how that experience for you and for the students might be a bit different.

Marela: I believe that it would be different, and yes, I just don’t know how until I do it.

[LAUGHTER]

Rebecca: Well, thank you so much for joining us today. It was really interesting and definitely got both of our heads buzzing about ideas for our own classes I think.

John: Yes.

Marela: Thank you so much for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

33. The Marmots of Finance

In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, Alex Butler, a Professor of Finance at Rice University, joins us to discuss how rich imagery can be used to help students make connections and deepen their understanding.

Alex received the Rice University Presidential Mentoring Award and the George R. Brown Award for Superior Teaching in 2018. He is also the recipient of the Jones School’s Award for Scholarship Excellence in 2011 and 2012.

Show Notes

  • Duke Talent Identification Program (TIP)
  • Medina, J. (2011). Brain rules: 12 principles for surviving and thriving at work, home, and school. ReadHowYouWant.com.
  • Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
  • Alex’s web site at Rice

Transcript

Rebecca: In our ongoing coverage of wildlife in the classroom, we can’t help but ask: How does a marmot become a mascot for a finance class? In this episode, we’ll discuss how rich imagery can be used to help students make connections and deepen their understanding.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Alex Butler, a professor of finance at Rice University. Alex received the Jones School’s Award for Scholarship Excellence in 2011 and 2012. Alex teaches financial markets and corporate finance in the undergraduate MBA, MBA, and PhD programs. Sometime in the latter part of the last century, Alex and I spent three summers teaching introductory economics to highly gifted middle school and high school students at the Talent Identification Program at Duke University.
Welcome, Alex.

Alex: Oh, thank you. Thank you for having me, John.

Rebecca: Today’s teas are:

Alex: I am not drinking tea…. just water at this point.

Rebecca: Another one… epidemic.

John: My tea is ginger peach white tea.

Rebecca: I’ve Prince of Wales today.

John: Could you tell us a little bit about the courses that you normally teach?

Alex: Sure. I have, over the years, taught almost every course that we have… either here or at other schools. Right now what I teach (and I’m glad to finally have settled into a group of classes that are my classes year in and year out) and those classes are the undergraduate business finance class and PhD courses – one in corporate finance, and one that’s a topics class on causal inference. I really enjoy teaching the undergrad business finance class in particular because I was an undergraduate student here at Rice many, many years ago, and so it’s fun to be back and be on the other side of the podium teaching the students. One of the things that I really like about teaching the undergraduates here is that they are able to appreciate my dorky sense of humor.

Rebecca: …which is indeed a very important thing.

Alex: Agreed. For years, I tried to suppress my dorky sense of humor in the classroom… ad tried to teach the course sort of straight up and dry, and it made everybody miserable… the students…. me… and so, eventually over time, as I grew more and more confident teaching, I started incorporating more and more jokes here and there… and then I allowed them to become more elaborate, and then before you know it, I’ve developed a full-blown dad sense of humor and full-on dork mode.

John: That’s actually why we invited you here. We read a little bit about that in a teaching award you just received. What do students expect the course to be about when they take an introductory finance course?

Alex: At Rice, the students have a good grapevine of information about what courses are about and so, at this point, the students come in with pretty solid expectations that line up with what the class actually is… and whereas some students are hoping that I’m going to teach them how to become millionaires in the stock market, what the course mostly is about is that’s very hard to become a millionaire investing in the stock market and how to make decisions in a corporate setting that will maximize firm value.

Rebecca: So you mentioned that some students come in with this misperception of becoming a millionaire. What are some of the strategies that you use to dispel that myth?

Alex: I should say this class is one that is a fairly standard course that’s taught in lots of different business schools all over the country, all over the world. One way that I do it differently is in the order of material that I cover… and so the very first week of class, I talk about market efficiency. That’s the notion that it’s very difficult to earn abnormal returns in the stock market. In other words, you can’t beat the stock market very easily, unless it just happens to be by luck… and so I come in the first week talking about the reasons why it’s very difficult to beat the stock market… and the reason why is because there are literally tens of thousands of people who have more money and more resources and who are faster and are doing this as a full-time job who are also trying to find the stocks that are mispriced… and so unless you are the investor who is faster, and smarter, and has more money to throw at the trading strategy that you think of, it’s very very difficult to beat the other 10,000 people… and so I spend the first week of class introducing this concept and then providing copious amounts of evidence… research that highlights how difficult it is for lay people to earn abnormal returns in the stock market. That sets the setting for the rest of the course, which is this notion of how competition affects prices and how that feeds through to other applications and the corporate domain as opposed to the financial markets domain.

Rebecca: What made you switch the order?

Alex: What I realized after teaching this class for a long time is that most of corporate finance, most of business finance, is about discounting cash flows back to the present at some appropriate discount rate, and I found I was having a hard time getting the students to understand the notion of what interest rate, what discount rate, should be used to make these cash flows that are spread through time to get a present value equivalent. I figured if I started with some aspect of that, where that rate comes from, where those prices come from. but that would make the rest of the course easier for them to understand as we go through. So, that’s why. Most people wait until after they’ve introduced things like: “oh, portfolio theory” and “capital asset pricing model” and other asset pricing concepts before they talk about market efficiency… and so I just sort of turned that around backwards… and I open the course with that. So, point number one is pedagogical, and point number two is that the lectures that I do on market efficiency are really fun, and so I really like starting the course off with something that’s really fun. So, we can talk about stories of insider trading, and we can talk about stories of surprise announcements and how that affects stock returns, and we can talk about “oh, so you think you’re going to beat the market, well let me explain to you how hard it is and the reasons why …” also were very fun. So, we can spend the first week talking about fun stuff. It’s a giant bait and switch.

[LAUGHTER]

I lure them in with fun stuff and then beat them to death for the next fourteen weeks with discounted cash flow analysis.

Rebecca: It also sounds like it’s a good way to motivate students. Not only is it fun, but it gets students motivated and interested and they buy into the class, which I wouldn’t discount that. I think that’s an important task.

Alex: Oh, absolutely.

John: …discounting in a different sense, but…

[LAUGHTER]

Rebecca: Yeah, sorry… I’m a designer

John: …but it also starts a class by dispelling that myth that they’re going to learn tools that will allow them to become really wealthy in financial investments and so forth. So, you’re setting it up by getting rid of that myth and they’re ready to start actually learning without having that at the background.

Alex: Correct.

Rebecca: I really like the idea of just meeting students where they’re at. If this is where some students are coming with, and maybe they’re super motivated in some ways but not in others, that you just tackle both of those in one week.

Alex: That’s right.

Rebecca: Cool.
You recently won a 2018 George R. Brown Award for superior teaching, and the Georgia R. Brown Awards are based on an interesting selection process. Can you tell us a little bit about that?

Alex: Yeah, so what the university does is they solicit feedback from recent alumni, people who graduated, I think, it’s two and five years ago. Now, they have graduated and they’re looking back what teachers would they want to see when these awards… and so, it’s really a neat honor, because once the students are two, three, four. five years out, they’re not responding to the short-term incentives that some professors used to gain teaching evaluations and things like that, they’re looking back and they’re actually remembering what the course was, what the professor was like, and whether it was meaningful to them. It’s really been just a phenomenal honor, and one that I honestly thought that I would never win. So, it’s been really special to have gotten that award.

John: It’s a really interesting idea to have an award given in that way, because it focuses on that long-term learning, that you’ve taught them something that’s going to benefit them later rather than, as you said, something that people do short term. I know some faculty, when we’re doing course evaluations, will give out cookies or other things just to boost their short-term course evaluations. But, that’s not going to pay off very much two to five years after graduation.

Alex: That’s correct. I went through the cookies and brownies phase myself years ago and decided that that just wasn’t who I wanted to be, so…. Now, we’re on to just just the pedagogy at this point.

Rebecca: I think there’s a couple of interesting things that I see about those awards, too, and one is that is more meaningful probably to win it because you can feel the impact but two it meshes really well with what we know about evidence-based practices in that students generally don’t like them while they’re happening, but they have longer-term effects… and that students tend to learn that material and transfer that material later on.

Alex: That’s absolutely correct, and I’m a big subscriber to that basic view that students often don’t like things that make them uncomfortable, and learning new and difficult things is uncomfortable. I think one could make students happier in the short run by giving them lots of little assignments where they feel like they’re making progress every day but they’re not actually being challenged, they’re not actually being pushed, and so instead holding them to very high standards for demonstrating their mastery of material while that is very uncomfortable for some of them, it is that that makes them better students and better scholars of that topic.

John: It reminds me of Bjork and Bjork, in their writings on “desirable difficulties.” that the most learning occurs when students are faced with feasible challenges… that if things are too easy, they get bored, and they may be happy with the course if they don’t have to struggle much… but they learn the most when they’re struggling but they see it’s possible.

We’ve heard that this award, though, based on some of the write-ups we’ve seen at your institution, may somehow be tied to marmots, wolves, and The Princess Bride. What do they all have to do with finance?

Alex: Several years ago, I was reading some books I thought would be…. some I thought it’d be helpful for my teaching, some that I just thought would be helpful for me, and one of the books was a book called Brain Rules by John Medina, and it’s basically a book that tries to take cognitive science, brain science, down to a level that lay people can understand and gives several rules of thumb of how the brain works and why the brain works the way it does… and as I’m reading this, reading it mostly for my own consumption so that I can be a better researcher and more thoughtful person, smarter, that sort of thing, I realized “Gosh, a lot of the rules here applied very directly to teaching, at least in the lecture format that I use in most of my courses…” and so one of the things that really stood out to me is how people learn better, remember better, I guess I should say, when they see images images that relate to whatever the topic at hand is. So, text maybe a PowerPoint slide with text, and you remember X percent but if you see an image, you remember much more of that material later on… and so this gave me just a license to, all of a sudden, start having fun on a completely new dimension. Reading this book and sort of embracing the notion that I could maybe help students remember the material better simply by infusing my lecture slides with some relevant images, was just eye opening for me… because now I could take my completely dry, boring slides with words and numbers and equations and now I can have fun with them…. and have this entire new dimension, a degree of freedom, to play around with what the slides are gonna look like and how the students are going to experience them. So, that’s sort of the extensive margin. The first part is: “Hey, I need images…” so the intrinsic margin is “what kind of images do I need?” “what will work best?” and Brain Rules comes to the rescue there again… and it says people respond to images that are faces, that are things that are scary, things that they can eat, things that might want to eat them, and things that they might want to mate with. Well, that last one’s kind of out for most of my lecture slides… [LAUGHTER] I can’t really incorporate that very directly. But it got me thinking “okay, what’s scary?” So, I started looking around for images of things that are scary to people… and I found this great image of this really, just terrifying, snarling wolf. I’m like “Ah, I’m gonna use that to get people’s attention…” But, as I started thinking about it, I wanted to lead into the wolf a little bit… and so the main prey of wolves in North America are marmots… yellow-bellied marmots… and so I found this great image of a little cute yellow-bellied marmot sitting on a rock somewhere in the mountain somewhere… and now when I come in to teach what could be the absolute driest lecture of the entire course which is time value of money and understanding how to discount cash flows (it’s the tool that everything else builds on, so it’s incredibly important to get it right, but it’s also potentially incredibly technical and boring)… and so I start that lecture not with an equation not with numbers but with a giant image of a marmot filling the entire computer’s projection screen…. and I just leave it up there… and the class is all looking at it… and I look back at them… and I pick someone at random. I cold call… and I say: “So, Charlotte, what do you think?”

“What do you mean what do I think?”

“What is it?”

… and we go through a series of guesses, and the guesses range from just ludicrous things: “it’s a gopher.”

“No, no, clearly it’s not a gopher. Gophers are a lowland creature. This is obviously in the mountains.”

“It’s an otter.”

“No, no, no. Otters prefer marine habitats, and this is clearly not there.”

… and go through this for a while until usually somebody recognizes it as a marmot. I say “Yes, very good. Alright..”

…and so then on to the next slide and the next slide is this picture of George Soros with no caption, no explanation, but again filling the entire screen, here’s this giant picture of George Soros… and so I go back to the first person: “Charlotte what do you think about this one? You didn’t get the marmot, how about this one?” …and so invariably somebody will eventually guess it’s a hedge fund manager.

I say: “Yes, very good. That’s George Soros, a famous hedge fund manager.” then the next slide is the wolf, the snarling wolf, and so at this point the captions on the slides read “This is a yellow-bellied marmot,” “This is a hedge fund manager” … and then it’s obvious what the wolf is. People get that right away. So, Charlotte gets to redeem herself at that point.

[LAUGHTER]

“This is a wolf, one of the main predators of the yellow-bellied marmot,” and then the next slide is another picture of a marmot but this one looking somewhat quizzical and the caption here is: “Why do wolves eat marmots, but not hedge fund managers?” …and the answer that I propose is because hedge fund managers understand the time value of money, but marmots do not. Now, that obviously doesn’t actually follow, but the correlation is there… that part is true… and so I just sort of lean into that and the ridiculousness behind that statement and from there on the marmot becomes our time value of money mascot… and so every time throughout the course I introduced a new application of time value of money then I bring in another image of a marmot to sort of tie all that together. iIt’s hard to see that the tie that binds everything together is time value of money and so the marmot, the mascot, is the visual cue, that “oh, this isn’t special, this isn’t different, this is just another application of the same technique we’ve been doing over and over again.

John: That notion of using visual cues goes back to the Greeks who use it to remember long stories before there was much printed word, and one of the arguments is that it’s because visual imagery developed much earlier than the use of language and the things you described in terms of things that might eat you may be tied back to our evolutionary adaptation, and so we’re tying into things that evolve fairly early in the evolution of human beings.

Rebecca: …and it also is just that emotional response.

John: Right, when you trigger an emotional response and fear is a major one…

Rebecca: Yeah. mm-hmm

John: Do the wolf’s chase the Princess Bride? I don’t remember that…

Alex: Well no…. So, the Princess Bride lecture is where I talk about decision rules using discounted cash flow techniques…. and so this sort of standard playbook for any finance professor anywhere as you talk about net present value and internal rate of return and payback period and maybe a few other rules as well, you just sort of go through the… whatever they have in the book in the order in the book, and you talk about each one in turn. But, it’s a lot more fun if you can personify those.. and so that’s what I use the characters from Princess Bride to do… and so the main character, which is net present value, is personified as Wesley / (there’s a spoiler alert here – so you haven’t seen the movie Wesley is the Dread Pirate Roberts). I always show my kids my lecture slides and when I showed them that one my older daughter was mortified that I would ruin the Princess Bride movie for students by revealing that Wesley and the Dread Pirate Roberts are one and the same… and so from there, I personify the different rules based on the different characters in Princess Bride… and so internal rate of return which is sort of flashy and useful but has some flaws as Inigo Montoya; the payback period which is sort of a blunt instrument is Fezzak the Giant, and so forth… modified internal rate of return is the Cliffs of Insanity.

[LAUGHTER]

Rebecca: So, the important thing to ask then is do you have the spoiler alert at the beginning of your lecture.
AEX: I do now.

[LAUGHTER]

John: How old was your daughter at the time when she reacted to that?

Alex: About ten years old.

John: Okay. So, by the time students are in college, they’ve probably either seen it or they may be less likely to, so it probably doesn’t do quite as much damage.

Alex: I hope so, and in fact I even now encourage them to watch the movie before that lecture without really telling them why. So some of them do. Some of them ask around and figure out why, but that makes it more fun.

Rebecca: That was gonna be my follow-up… is that homework?

Alex: It’s the best homework they’ll ever have.

John: So, how have students responded to this? Do they remember this later?

Alex: Yes and no. for the Princess Bride lecture I believe that the students sort of uniformly enjoy that…. but I don’t have a good sense of how much that actually impacts the depth of their learning. For the marmots, it’s a completely mixed bag… some of the students really love it, and I really do lean hard on the fact that this is the mascot… this is the thing that ties it all together… and this is the visual cue so you will now recognize that when we do bond pricing “oh, here’s a marmot, that means it’s just time value of money…” “oh, when we do net present value, here’s a marmot showing up alongside our Princess Bride character it’s just time value of money” and so forth. Many of the students really enjoy that and grab on to it and some students, they’re not having any of it. They want boring… they want dry… and I’m afraid they’ve come to the wrong place. Been there and done that. I don’t do that anymore.

John: But that use of imagery is really common. People who work on developing memory… the memory palace type things where you tie specific concepts and bundles of concepts with chunks of item to key images, helps people remember things long term. I can see how it would be really effective.

Rebecca: I also could see that one of the things that students often struggle with when information is new to them is making those connections. So providing that visual cue like “here’s time to make a connection…” it’s actually really helpful, because those are the kinds of things that might seem really abstract and very separate if you’re not making an explicit. So, I like that you don’t even have to say explicitly like “this is the thing” you have by putting the image up there, and you’re prompting the students to predict what that connection is or challenging them to think of it on their own before you reveal what that connection is.

Alex: That’s right, and so what started off as just a fun way to get attention: “Hey, class is starting let’s all key in… Here’s a scary wolf.” So, now we’re all paying attention to the scary wolf because that’s how our brains are hard-wired. It’s now grown into this entire thing where throughout the entire class is a continuing callback.

Rebecca: It sounds to me like you must have some pressure every semester to have to have something new that you introduce into some sort of lecture so that there’s some anticipation.

Alex: I’ll be the first one to say that I shamelessly recycle all of my jokes…

[LAUGHTER]

… and so I’m always sort of terrified when I have a student who started the class last year but had to drop at the 6 week mark or the 8 week mark because whatever was going on their life, and then they’re back the next year. Like “oh man, you’re gonna get the exact same jokes with the exact same timing, the exact same patter.” It mostly seems to work fine, and I do continually try to incorporate new things. One of the nice things about teaching finance, although some of it is very static… the basic concept of time value of money is going to be there for basically forever and I won’t need to change the actual examples in those slides really ever… but a lot of the other material changes very rapidly. So, the notion of what does market efficiency mean? who can beat the market and win? what is the evidence? But for topics like market efficiency, for topics like financial markets, for topics like investment banking, those areas transform rapidly… and so I’m continually changing those lectures year after year because one of the big topics that I cover in my course that is not really traditional for an undergrad business finance class is how firms raise external capital. The reason why is because it’s one of my main research interests, and so I have lots of ideas of what I want the students to know, and lots of research that I can tie into the lectures that I give. For the past five or ten years financial markets, the regulation of financial markets, firms’ ability to raise external capital, has changed tremendously as regulation has changed and so I’m continually revamping that portion of the course basically every time that I’m breaking it out.

Rebecca: One thing that I can’t help but think is that you and John have some similar backgrounds in terms of content, so the time value of money that you keep referencing probably makes sense to you but I feel like our listeners might not know exactly what that is so maybe we should just take a minute and give a quick cap of that so people know.

Alex: Yeah, absolutely. So, if you have $100 and you invest it earning an interest rate of 5% per year, in a year how much money will you have?

Rebecca: One hundred and five?

Alex: One hundred and five dollars.

[LAUGHTER]

Rebecca: It’s not a trick question, right?

Alex: No, No, it’s not…

[LAUGHTER]
… and so, as you invest money it earns some rate of return and so then money in the future you have more, because it earns some positive rate of return, and conversely, would you rather have $100 today or $100 in a year? Well, the answer is $100 today, because you could invest that for that year and have the hundred dollars plus some additional return. So, you’d have a hundred and five dollars one year from now instead of the: “oh, I’m going to give you a hundred dollars a year from now” and that’s all the time value of money is.. and then it’s application of that over multiple cash flows and multiple periods where people start to lose track that it’s just math.

John: That’s the fun part.

Rebecca: Yeah. Yeah.

John: I also ask my students have they ever burned their mouth on a slice of pizza, and why don’t they just wait? ..and it’s one way of introducing the notion that we prefer things now to later. It’s a fairly important concept in economics and finance and it’s at the basis of finance.

Alex: Oh, absolutely… and I like the way of framing it as how patient you are. Are you willing to wait or are you impatient? …and so the way that I sometimes describe that in class is if you have a low discount rate that means that the future is worth about the same as the present and so you are patient. If you have a very high discount rate well then the future and the present are very different, so high discount rates → impatient, low discount rates → patient.

Rebecca: You realize that the irony of this whole thing is that I have these conversations with my husband all the time because he’s in finance, right? Yeah….

[LAUGHTER]

John: Ok, well, does that help explain it?

Rebecca: Yeah, yeah. I’ve had it explained to me many times.

[LAUGHTER]

I figured our listeners would need it.

Alex: Well, since we’re talking about the things that our students do, I’ll just share one example that I use in class that I continue to use even though it completely does not work with undergraduate students. It’s when I’m trying to introduce the concept of sunk costs… and so a sunk cost is some amount that you have paid… a cash flow that has happened in the past perhaps… and so once that is paid, you can’t get it back …and so it’s like the notion of “should he throw good money after bad” is another way of phrasing it… and so, what I used to say is, “Well, you go to a movie, you pay your fee to go in, and you decide it’s a terrible movie, should you keep watching to get your money’s worth or should you leave? and then a student pointed out to me one year that well, actually, if you go and you complain you can get your money back from the management.

[LAUGHTER]

Okay, different example. This is the one that never works on the undergrads. You go to an all-you-can-eat buffet and you have a choice of do you buy one plate for this amount or do you pay a little more to get the all-you-can-eat buffet and you decide to go for the all-you-can-eat. You pay the $10 for the all-you-can-eat instead of the $7 for the single plate… and you go and you fill up your plate and you eat… but you decide you’re full after just your first plate… but you’ve already paid that all-you-can-eat price. Should you go back for more? …and the intellectual scholarly answer is “Well, no, because the marginal benefit of eating more food is negative at this point because you’re full. Even though the marginal cost is zero.” But, for the students, the undergraduates, the marginal benefit of eating more food is always positive.

[LAUGHTER]

They view it as a sport. Dinner’s not over until they kick me out.

Rebecca: I think that’s really important to consider your audience and what works. So, I think that’s a really good demonstration of knowing your audience and why something might not work for a particular audience.

Alex: Yes, nonetheless, I still use that example… but I tell them ahead of time “I know this is not going to make sense to you. You’re going to push back on me. Nonetheless, I want to talk it through with you so that we can think about who has marginal benefits and marginal costs of what amount.”

John: Just last semester I used exactly the same thing of asking them how many of them would leave a movie theater if it was clear that they weren’t enjoying the movie and this time I had some people argue “I do that all the time.” So, yeah…

Rebecca: Movie theaters? Don’t you just have Netflix?

[LAUGHTER]

John: Okay, that’s actually a relevant point. In terms of the Princess Bride, we’re moving into a generation where many of your students might not have seen that. Do you have any other movie references that you might substitute in the future?

Alex: You know, I haven’t found the right one yet and, honestly, even if I found a good substitute, I’d really want to keep the Princess Bride, just to give them the incentive to watch that movie.

[LAUGHTER]

It is such a classic and such a gem. I really would like for them to all see that movie.

Rebecca: It’s an investment you’ve made.

[LAUGHTER]

Alex: That’s right.

Rebecca: How have faculty responded to the methods that you’ve been using?

Alex: Faculty are not surprised that I do the things that I do.

[LAUGHTER]

John: They’ve known you for a while.

Alex: They’ve known me for a while and I really lean into the corny pun bad jokes big time in class…. and I really I’m quite shameless about it.

John: That’s not new… I seem to remember that back at Duke when you were still in college.

Alex: …and it’s just a question of do I try to suppress that innate desire or not?

[LAUGHTER]

…and I’m at the stage in my career where “nope, not anymore.” You’re just gonna let it all out. My colleagues are not really surprised, but interestingly, one of my colleagues has adapted her lectures quite a bit to embracing this notion of adding images into the slide decks… and that’s my wife. She teaches business law and regulation of business. She was an attorney by training and I described her as a born-again economist because when she started teaching regulation of business she had to teach herself basically all of public choice economics. So, she got a couple of high-level textbooks and worked through them all but she and I regularly discuss teaching techniques and so she now has gone down the rabbit hole of finding that perfect image to highlight the point that you want to make… to have that really stretched metaphor that you can then call back throughout your course. So, it’s been a lot of fun to have her as a sounding board to go back and forth with.

Rebecca: I appreciate that you’ve brought design into the process.

Alex: Yeah.

John: Excellent. It’s something we all should probably do more of and think about more and certainly much more effective than those PowerPoint bulleted lists that are so common.

Alex: Yeah, a little bit of both actually goes a long way.

Rebecca: You’ve won some awards for your research. How do you maintain a balance between teaching and research?

Alex: It’s difficult. There are some ways in which research can feed directly into the teaching… and so my research that touches on market efficiency, some of that can come into the classroom. My research that touches on how firms raise external capital, some of that can come into the classroom. Now when I teach my PhD courses, those are heavily flavored by my research interests and preferences, but when I teach the undergraduate core finance class there’s not a whole lot that can flow back from that teaching into the research. So, that’s one of the downsides of teaching that class as opposed to perhaps a class that is more specialized or an elective or something that’s a little bit further downstream from the core class.

Rebecca: But at the same time, if it’s something that you teach routinely then the prep isn’t as difficult.

Alex: Absolutely. That’s true. It is hard to overstate how useful it is to teach a class a second time, or a third, or a fourth, or twentieth. Nonetheless, every single year, even though it’s the same class and mostly the same topics, I go through every slide, every lecture from the beginning every year.

Rebecca: But, I remember teaching a bigger selection of classes and one of the things that I’ve liked about my position at Oswego is that that suite of classes has gotten smaller… and then there’s a little less I’ll keep on top of to make sure that you have all that fresh information and what-have-you for classes.

Alex: Absolutely, and I’ve done the same thing. I’ve taught a variety of courses over the years and it’s been nice… as you describe it that suite of classes narrows, so that you have the same core group of classes that you’re teaching over and over again and you can start to specialize. You can really invest the time to get over the fixed costs of finding all those right images for the slides… to going through and taking time to invest in the design aspects of the lectures. That if you had four different courses every term, it would be incredibly difficult to find the time to do that effectively.

John: While you’re working with PhD students, you must do quite a bit of mentoring of them. How do you see the role of a faculty member as a mentor for graduate and undergraduate students?

Alex: Faculty vary widely on their views of how much mentoring PhD students should have… and so you have one model where it’s sink or swim… the PhD students are some of the smartest people in the world… they’re good students… they’ll figure stuff out… just point them in the right direction and let them go, and they’ll get there. That’s not the view that I subscribe to. Because I think we frequently overestimate just how much the doctoral students know, particularly about how the profession works. There’s no book for that. They can’t just go down to the library and find a textbook on how to be a good assistant professor or… there are books on how to write a dissertation but that only gets you so far, because it really needs to be very field specific. So, I tend to go very much the other way, which is a lot of sort of high-touch mentoring. I write co-authored papers with many PhD students, one of whom is now your colleague there at Oswego in the Finance Department in the School of Business, and it’s enjoyable for me. It is a good learning experience for the students and I think it helps them to learn how the profession works much more efficiently, because when it comes time to write a paper and they might put together some tables and say I want to structure the introduction this way. Oh, no, no, no, no, we can’t do that… because if you do that then it’ll make people be concerned about this issue here. So, instead, we need to twist it around this way and start with this… start with the big picture, not what your paper does but what your paper’s about. That sort of thing. That’s hard to learn on your own.
PhD students are PhD students because they are extraordinarily good students and they’re really good at learning. Though, that’s not the job for academics. The job is not the learning, the job is creating knowledge… and the transition from being a consumer to a producer of knowledge is scary, and it is the road that has very few signs or roadmaps to help them get down. It’s a transition of going from a consumer to a producer of knowledge… is very profound for a lot of people.

Rebecca: What’s interesting about what you’re hearing is you know my field the terminal degrees in an MFA a Master of Fine Arts, and the undergraduate degrees are really professional degrees. But, it’s the creators of cultural content ,and so that struggle happens at the undergraduate level too, of going from being that consumer of culture to a producer of culture. It’s really not that much different… just what they’re creating is a bit different.

Alex: That’s right.

John: I remember when I was working on my PhD, one time, where up until that point I had been meeting with my advisor every month to talk about my research, and at first I was just asking him questions… and then he was asking me questions and I realized suddenly that I knew more about the topic than he did… and that’s I think that sort of transition that’s sometimes difficult… because when you’re working on your research you’re mostly going out and finding all these earlier studies and so forth but you get to some point where suddenly you become the expert in the field and that’s a tough transition to make. It’s scary, as you said.

Alex: It is. It’s quite the watershed moment when you realize… when you are presenting your research to a room of 30 presumably learned scholars that maybe collectively they know more about the topic than you, but you know more than any individual person in that room… and becoming that expert and then owning it, so that you can write confidently is, I agree, a very tough transition

Rebecca: I like the emphasis on the owning it part. I think that’s key.

Alex: Yeah, imposter syndrome is real.
[LAUGHTER]

John: Everywhere.

Rebecca: So, we usually wrap up by asking “what are you gonna do next?”

Alex: Well, that’s a good question. I really wanna keep doing what I’m doing, making my class better year on year, teaching PhD students and training them year on year and working on research, hopefully research that people will actually find interesting and useful.
I have a sabbatical coming up.

Rebecca: When’s your sabbatical? What’s the countdown?

Alex: Well, the sabbatical is a year from now and in between then, one of my colleagues with whom I co-teach a doctoral seminar, he’s on his sabbatical. So that PhD course, that normally I teach half of, I’m now teaching the entirety of next fall. So, I think the first thing I’m gonna do is prep the rest of that class.

John: Excellent.

Rebecca: Deadlines make a difference.

Alex: Yes, indeed.

John: That procrastination thing… and that time preference…

Well, thank you!

Rebecca: Thank you so much.

Alex: Thanks.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.