42. Flipping the classroom

Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode, Dr. Dominick Casadonte, a Chemistry Professor at Texas Tech University, joins us to discuss research and best practices related to flipped classrooms.

Show Notes

  • Camtasia
  • Mediasite
  • Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement. International Society for Technology in Education.
  • Sams, A., & Bergmann, J. (2013). Flip your students’ learning. Educational leadership, 70(6), 16-20.
  • Bowen, J. A. (2011). Rethinking technology outside the classroom. Journal of Music History Pedagogy, 2(1), 43-59.
  • Bowen, J. A. (2014). The Teaching Naked Cycle: Technology Is a Tool, but Psychology Is the New Pedagogy. Liberal Education, 100(2), n2.
  • Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve student learning (ed.). Jossey-Bass
  • Belford, R. E., Stoltzfus, M., & Houseknecht, J. B. (2015). ConfChem Conference on Flipped Classroom: Spring 2014 ConfChem Virtual Poster Session. Journal of Chemical Education, 92(9), 1582-1583.
  • Stoltzfus, J. R., & Libarkin, J. (2016). Does the room matter? Active learning in traditional and enhanced lecture spaces. CBE—Life Sciences Education, 15(4), ar68.
  • Stoltzfus, Matthew (2016). Engaging Students in the Flipped Classroom (video)
  • Coats, H. J. (2016). A study on the effect of lecture length in the flipped classroom (Doctoral dissertation).
  • Casadonte, D. J. (2016). “The Effectiveness of Course Flipping in General Chemistry – Does It Work?” ACS Symposium Series, “The Flipped Classroom”, December 2016 (Book Chapter) The Flipped Classroom Volume 2: Results from Practice, Chapter 2, pp 19–37, Chapter DOI: 10.1021/bk-2016-1228.ch002, ACS Symposium Series, Vol. 1228, ISBN13:9780841231627eISBN:9780841231610,
  • POGIL.org

Transcript

Rebecca: Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode we discuss research and best practices.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is dr. Dominick Casadonte, a chemistry professor at Texas Tech University. Dr. Casadonte is recognized as a global leader in flipped learning by the Flipped Learning Global Initiative.
Welcome.

Dominick: Thank you very much.

Rebecca: Today our teas are:

Dominick: Well, my usual afternoon beverage is an iced green tea with three pumps of raspberry, but since I’m getting over a cold, today I’m drinking a hot green tea with vanilla, lavender, and honey.

Rebecca: That sounds really nice. Finally, a tea drinker.

[LAUGHTER]

John: I’m drinking Irish Breakfast tea.

Rebecca: …and I’m drinking chai today.

John: You’re recognized as a an expert on flipping the classroom, and you’ve been doing it for a while. Could you tell us a little bit about what a flipped classroom is?

Dominick: Sure. The usual paradigm, if you think about it with regard to teaching, is lecture, homework, lecture, homework, lecture, homework, tests, lecture, homework, lecture, homework, and so on… and the problem with that format is that it’s not conducive to deep learning… with the exception of maybe very bright students or very bright asynchronous learners. Trying to keep up with a lecturer is often difficult. A student, for example, may not have gotten down everything that they wanted to write down… they may not have gotten it down correctly… or it might be fragmented… or perhaps they just didn’t understand well enough to write it down in a meaningful way, or a way that’s meaningful for them. Then they try to go home and they try to do homework, but since they didn’t really understand the lecture in the first place, the homework is again difficult and they might only be able to work part of it. So then, the very next day, there’s a new lecture and a new topic. The normal sort of didactic approach to education often leads to what I’ll call fragmented learning, in the sense that there isn’t enough time in class to practice many of the active learning strategies that work so well because faculty are so concerned with getting the lecture done. In the flipped model, on the other hand, we flip the homework-lecture-homework paradigm so that the students’ homework is to watch a lecture online. Now it can either be PowerPoint or a video and different people use different techniques. It can be done on Camtasia, it can be done through Mediasite, it can be done through a variety of different platforms. It could be done on YouTube for example. So, they’ll watch usually a video or a lecture online before they come to class, and then they may or may not do online homework prior to coming to class as a part of the pre-class experience. Then, once in class, time can be spent, for example, checking the students knowledge… clearing up muddy points or any misconceptions they might have… working advanced problems. But now, in this particular case, class becomes more of a discussion and you can use a lot of the active learning strategies that really engage the students in learning the material. By flipping the classroom, we freed up time to really engage and interact with our students and it helps with their learning in significant ways which I can talk about later.

John: So, in a traditional class, you tell students what the content is in class, and they come in with very different backgrounds and some of them are able to pick it up, others get lost along the way, and then they’re sent out to do homework… where students seem to have the most difficulty on homework or tests… and we’re pretty much leaving them alone to do that. But, here in the classroom, you’re able to work with them and help them through some of the issues.

Dominick: Right… and so, for example, in the classroom with homework, now they have a mentor who can help them work through the more difficult problems to help guide them along. There are models of the flipped classroom where there are peer mentors in the class who can help and walk around and mill around as well. So, there are opportunities for group work. There are opportunities for group discussion. You can engage the topics in a much deeper way than you can by just simply lecturing at students. So, it’s a very very non-passive way of teaching.

Rebecca: What prompted you to get into a flipped classroom model?

Dominick: Desperation, I would say. [LAUGHTER] I first started teaching an online class in the fall of 2007 as part of a multidisciplinary science master’s program of which I was a part here at Texas Tech… and getting everyone on the same platform at the same time, my thinking was that lecturing to them live online was a waste of time, basically. The question that motivated me was the same one that motivated John Bergmann and Aaron Sams around the same time at the high school level up in Colorado. Namely, what could I do to use my classroom time more effectively? What could I do to use my online time more effectively? …and I thought, “Well, let’s take the lecture out of the classroom.” So, I pre-recorded my lectures. I had the students watch the lectures before we got together in the online environment, and then we spent the majority of our time having discussion and working problems…. and these were teachers at the time and they were very very enthusiastic about the ability to actually discuss the material that they were trying to learn… and so then I thought “Well, hmm, if that works really well on an online format why don’t I try it face to face?” …and so then, starting in 2008, I adopted this to my general chemistry class. The following summer, a high school teacher that I had as part of a workshop said “Oh, I see you’re flipping your class” and I said “Flipping, what’s that?” ‘Cause at the time, there were a lot of different terms for flipping. It was called: time-shifted instruction, reverse instruction, blended instruction, all sorts of things. Flipping is a term that’s really sort of stuck.

So, I started out of a sense again of “How can I engage my students more effectively in the classroom?” and once I realized that it worked swimmingly in the online environment and then I said “Well, okay, will this work in the face-to-face?” …and it worked even better there. By that point, there are pockets of people around the country who are doing this, and they used all sorts of interesting terms. Jose Bowen at SMU, who’s an art professor, used the term “naked teaching” because it can be unsettling when you walk into a classroom and not have the comfort of your lecture notes to be able to project or read to the students. So, very often I will walk into the classroom and say “What are we going to talk about today? or “What would you like to talk about today?” You can’t do that unless you feel comfortable about the subject material and you have some expertise, but it’s a great way, in the long run, to really, really impact students.

Rebecca: You mentioned both doing this in an online environment and also in the classroom. Is there a difference between your experience in both?

Dominick: Yes, it’s so much easier face-to-face. To be able to walk around to gauge what’s going on in the classroom, their level of understanding, and how learning is happening. The power of direct peer-to-peer contact should never be underestimated, I think. Now, in the online format, my focus was more on the development of a learning community working on a particular topic, rather than in peer-to-peer mentoring and things like that. So, they’re very different approaches, but you do develop a sense of community in both, I think.

John: Now, with your online classes are they synchronous or asynchronous?

Dominick: My online classes are synchronous, in the sense that everybody meets in one place at one time. We have various software programs that allow us all to be in the same place. Yeah, so they’re all together at one point, and that’s kind of interesting because I have people from all over the country who are taking these classes. They’ve never actually physically met, at least for the first class that I teach. It’s interesting trying to, in the discussions, get a sense of the personalities of the people who were providing discussion.

John: Are they participating in video formats or is it just audio or text?

Dominick: I’ve done both. I’ve done video and audio and depending on the bandwidth and the number of students in the class both can work well.

John: How large are your classes?

Dominick: Well, my online class had 24 students in it. So, a fairly large, I think, from an online perspective class. It wasn’t one of these very very large classes like you might see at MIT, for example, but 24 is a good class in terms of bandwidth… trying to get everybody in the same room at the same time… and my face-to-face classes… they’ve ranged anywhere from a low of 25 to a high of 150.

Rebecca: You mentioned two different techniques that you use both in online versus in person. So, online you mentioned community formation and then in person you mentioned peer to peer. Can you expand upon each of those?

Dominick: Yeah, my first adage for teaching is know your audience, and in the face-to-face environment I was working largely with in-service teachers who were trying to develop a more significant content knowledge of chemistry, and so there the idea was the development of a supportive community where these teachers could bring their ideas to the table in terms of not only the content that I was teaching them, but also how they could then apply that content in their own classroom settings… and in that regard they were able to help teach each other techniques that they could use in the classrooms based on what they were learning in terms of the content. There really was a community focus, sharing knowledge as opposed to just gaining knowledge. In the face-to-face classroom, it’s more a sense of the students trying to understand the content at a deep level. There I found that it is true that when you teach something you really hopefully really understand it and so peer-to-peer mentoring is much more effective there. So, we tend to work in groups… we tend to work with dyads… two people working next to each other… and then sometimes I will just have people go to the board… but once again it was in that context of community because I think it’s very important, if you’re going to do flipping, and do it well, that it’s an active, encouraging, engaging classroom experience. If my students go to the board, whether they get the problem right or wrong, the class gets into a habit very very quickly of applauding the student for their attempt… whether or not it’s right or wrong… and then we debrief. We talk about what works… what doesn’t work… So, it really is both peer to peer and community building there as well. But, the emphasis is more on the individual in the face-to-face classroom, I think.

Rebecca: It sounds to me a little bit like the choices that you’re making online and in-person aren’t necessarily because of the medium, but rather who’s in those particular classes. Am I hearing that correctly?

Dominick: That’s right, and one of the nice things about flipping is that it is such a rich environment in which to work. As I mentioned earlier, the pre-class videos can be video lectures, they can be audio lectures (if that’s appropriate), they can be PowerPoint presentations, they can be any number of things. In the in-class experience, it’s an active learning environment. so you tailor it to the people in the classroom. For example, if you’re trying to teach in a flipped environment of the class of 24, there you can do all sorts of things that promote individual learning in ways that it’s a little bit more difficult to do in a class of 300, for example. But, I have a good friend, Matt Stoltzfus at Ohio State, for example, who routinely flips like general chemistry class of 600 students, and he’s able to give them as close to a personal learning experience as one can, I think. The point that I’m trying to make is that the flipped environment is a very rich one and it allows you to tailor the learning experience to your class specifically.

John: Do you create your own videos or do you use ones created by other people?

Dominick: I actually create my own videos. I have my own recording studio in my office… I have a Mediasite setup…. I have a video camera… I have a document camera… and I have a wonderful microphone…. and so every couple of years or so I re-record all of my videos. Now, I do use a mixture of other formats. So, for example I do have post-video homework that the students have to do online before they show up to class… and that’s done using an online learning platform that we have here at Texas Tech through a national distributor. The advanced problems that we work in class come both from the textbook that we use and also from problems that I develop. So, it’s sort of a hybrid. But, the pre-lecture videos I actually produce…… and one of the things that studies have shown is that students develop a certain sense of identity with regard to the person teaching the class. Some people, when they’re starting to flip, might want to just use Khan Academy videos, for example, or things like that. But the studies show that the class wants to see folks on the videos (or your voiceovers if you’re using PowerPoint with voiceover)… they want to see the professor who’s teaching the class, because that’s their professor… and so they develop ownership, if you will. Plus if you’re using other media, for example a Khan Academy, they may not be teaching it exactly the same way that you want to teach it, and so then you have to either reteach or undo. There’s a quality control issue there. So, it’s just easier, if you’re gonna do flipping, to make your own videos… and there are so many different ways of doing that now that there’s really no excuse.

Rebecca: That had been my experience as well when I’ve done videos in my classes. The students really liked the quirkiness or knowing that it’s the same person that they had in their classroom and if you try to slip in something else occasionally they really didn’t like it.

Dominick: …and a lot of people have a perfectionist tendency and really want their videos to be really super perfect. Well, once again, studies have shown that that’s not really what students want. They want to see the foibles. They want to see you as you are. If they know that you’re going to say “um” or “uh” in the classroom then if you don’t say “um” or “uh” in your videos, then they’re gonna say “Is that a robot teaching the class that looks like my professor?” So, it’s okay to be human when you’re doing the pre-lecture videos. But, I think one of the things that often hangs up people when they’re starting to do flipping is this notion that it has to be perfect, and it really doesn’t.

Rebecca: I think they really appreciate when you make mistakes and things, too. I know that my students did when I do like a coding mistake, I’m like “Whoops, I made a mistake..” and go back and fix it and explain what I did and why it was wrong. We all make those kinds of slips and errors and things and we would do it live. So, it’s kind of nice to do it in videos, too.

John: It makes you seem more human by doing that.

Dominick: One of the nice things about them seeing you make mistakes is that it gives them permission to make mistakes. They don’t have to be perfect. I’ve had experience with students in classrooms where they come in and they’re intimidated… they’re shy… they’re just afraid that they can’t master the materials. So, seeing somebody make a mistake who is an expert gives them permission to make mistakes… and one of the things that that really does is it empowers them to learn, because at the end of the day when somebody’s trying to learn they’re going to make mistakes and I give my class permission to make mistakes. In fact, I tell them you have permission to make mistakes while you’re learning. After you’ve learned something, that’s a little different. If you’re an engineer, for example, and you’re building a bridge, I don’t want you making mistakes. But while you’re learning… absolutely, make mistakes. Part of education is this movement from novice to expert and in that process one makes mistakes. So, I told my class “You have the right to make mistakes” and I use an example. So, let’s say you’re a five-year-old and you’re trying to learn how to ice-skate you fall down, what does a five-year-old do? They laugh, they brush themselves off, they might giggle a little bit, they get up and then they just skate some more. Now, imagine you’re an eighteen year old and you’re learning to ice skate and you fall down. Most of the time, people stand up and say “Did anybody see me?” and they worry about what people are gonna think, instead of just getting up, laughing, and moving on. So, what often happens is the 18-year old never learns to ice skate, whereas the five year old, who’s willing to make mistakes, learns. So, I tell them it’s okay to make mistakes while they’re trying to learn… and also it’s about empowering students to be able to have confidence in themselves… and we talk about this a lot. We did a study of what motivates students in the flipped environment, and part of it is the confidence… the autonomy… that they develop in the flipped environment. When they really think they’ve got and they really understand, there’s a certain level of “Geez, I understand this. I can understand the next thing…” and so on and so forth. That sense of confidence really improves their educational experience.

John: How long are your videos? Do you tend to have very long ones or do you chunk them up into smaller chunks? and what would you recommend in terms of video length?

Dominick: We did a study a few years ago on the optimum length for flip videos, and it came about because our book dealers: McGraw-Hill, Prentice-Hall, Cengage… those are the top three that we have here at Texas Tech… were telling us that they were creating videos and they’re creating these five to seven-minute videos… and I said why are you making five to seven-minute videos?” and they said “Well, everybody knows that the Millennial generation has a five- to seven-minute attention span and so they want five to seven minute videos.” I said: “But, what is best for learning and improving learning outcomes? I don’t care what their preference is. What’s best for them to learn?” He said “Well, we don’t know.” I said “Well, do you have any data or evidence that shows the five- to seven-minute videos are really great for learning?” and they said “No, but we would be interested if someone would do a study and tell us.” So, I had a graduate student who embarked on a study of lecture video length. We set up essentially short video people… those are people to watch five to seven-minute videos… and we did this with master videos, so each video was probably 40 minutes to an hour long. We put stop signs in the video and when they hit a stop sign they would stop. They would work some online homework and then they could pick up again… or they could go off and do something else. but the short videos were five to seven minutes and then we had a long view group, as it were. We allowed them to decide which videos they wanted to do and not surprisingly, 62% decided to do short videos. So, I have no problem with the book dealers’ notion that Millennials prefer shorter videos. But, then we let the semester go on and we didn’t force them to stay into one group or the other. We let them move and we watched their video habits. We weren’t video stalking them, but we could watch their lecture habits using the Mediasite analytics down to the millisecond.

What we found was that 60% of that short video group switched to the long video length, which is very surprising to us, and we did also a variety of assessments. We looked at online homework grades. We looked at quiz grades. We looked at exam grades. We looked at final exam grades. We looked at the American Chemical Society standardized tests that we give as a pre-post and what we found was that there was a subgroup of the long-view group that watched the video as a long video, but then they stopped at specific points to either have a snack with a friend, go to the bathroom, whatever they needed to do… and we called them the long pause people. It turned out that in every assessment that required global understanding… so final exams, ACS exam individual exams… the long-pause viewers actually scored one standard deviation higher than the short viewers. In fact, the short viewers had the worst learning outcomes of all three groups. Then we gave them Likert scale questionnaires, and we also gave them open-ended questions, and we said “Why did you make the switch? What do you find?” and they said “Well, it just got too fragmented to look at these short videos and we couldn’t take what we saw in video A and connect it to video B and so on and so forth, especially if several hours had gone by, because we then had to go back and watch video A again to remember what we forgot. But, if we did the long videos we were able to just put it all together.” Also in that we found that the best optimum video time. So, what constitutes a short video versus a long video for Millennials is 20 minutes or less is short, 30 minutes or more is long. So, 20 to 30 minutes is the sweet spot for video length. That’s what the study showed us and we’ve just submitted that for publication.

Rebecca: Have you adjusted how you’re teaching based on that information?

Dominick: Yes, my videos are roughly in the range of 30 to 40 minutes, but I tell my students “Take the time you need. Take breaks. It’s going to help your learning outcomes.” I also share with them the results of our studies, because I think if you’re going to, in my case if you’re gonna be a scientist, you should be data-driven. So we want our students to know that we’re not just telling them this because it’s something anecdotal, but rather it comes from data that we’ve collected.

John: Have you thought about controlling for self selection and randomly putting students in groups? Because one concern I’d have with that is that it could be the case that those students who select the short videos might have done less well no matter which group they were in or vice versa… although you do have switchers in there.

Dominick: Yeah, I’ve actually been very lucky in my studies that I teach our honors general chemistry sections… and so I look at the SAT scores… I look at their previous class performance… and it’s a very very, as much one can have, a homogeneous group. So there’s really not much of a selection bias, I think, as far as the study goes.

Rebecca: You’ve also done some research on the flipped classroom approach in general, not just the video length. Can you share some of your findings?

Dominick: Sure. We found out a lot about the flipped environment over the past 10 years or so. As I mentioned a minute ago, I’ve been very lucky in that I tend to teach very bright students. I did a five-year longitudinal study on the effect of flipping which has been published in American Chemical Society monograph and we found that the average exam grade increased by 9.2 percent over that five-year period, and that the largest gains in learning came during summative assessments (for example during final exams and externally developed independently normed exams like our ACS exam). We also have done work on what motivates students to do well in the flipped classroom and we just recently presented at the biennial conference on chemistry education regarding the effectiveness of peer mentoring during the flipped classroom. The results there were very astounding to me. It shouldn’t have been because peer-led team learning has been around for more than 15 years, but nonetheless, in trying to tweak the classroom, the addition of the peer mentor took what was already much better than had been before and improved it dramatically. I’ll give you an example. On the American Chemical Society end-of-term exam, without the peer mentor present… So, I give my students incentives if they score above the 95th percentile on that exam, and it’s a challenging exam, they get an automatic “A” in my class. They don’t have to take the final because this exam is normed against thousands of students around the country in a variety of different university settings. Pre-flipping, I had zero to one student scoring above the 95th percentile; post-flipping the average was roughly nine. So, it increased tremendously. With the presence of the peer mentor, the number went from 9 to 34 and so almost a six-fold increase in the number of students scoring above the 95th percentile. The only difference being, since I used to run the discussion sections that I subsequently allowed the peer mentor to do, that was really the only difference in the environment as far as I could see in terms of controlling for the different factors, except for the obvious one of different students. But she did this in the fall of 2016 and the fall of 2017 and there were 34 in one in the fall of 2017, 33 in the fall of 2016. The percentages of the class actually increased. This is kind of reproducible, if you will. So, we’ve looked at a variety of different things, and with regard to motivation we looked at that from self-determinacy theory and found essentially three things that really sort of motivate people: one is autonomy, second is pace, and the third is responsibility.

In the flipped environment, this sense of autonomy… the sense that “Oh yeah, I am really learning something” is very important to the students. The pace, the fact that they could watch the videos in their pajamas, for example, was very important to the students as a motivating factor… and responsibility, the fact that they had to take responsibility for their own education as opposed to being sort of spoon fed in a lecture format was something that motivated them as well. That student just graduated in December, so we’re now preparing that for publication. So, we looked at a variety of different aspects of flipping.

Rebecca: Can you clarify about your peer mentor model? The students are watching videos outside of class, and then they’re coming to a class with you, and then also recitation session with another student?

Dominick: They watch a pre-lecture video. They do online homework. We have a class discussion… work problems… and then there’s a separate recitation section… and historically I have done all of that. I had a very bright student who really wanted some teaching experience and she said “Would you let me run your recitation?” and I, like many faculty perhaps, don’t really want to give up control of my classroom environment. So, it took a lot of cajoling on her part to get me to do that…. and I said “Well, let’s look at it and see what happens.” She took over the recitations and… talk about a motivated young lady… she would provide review sheets for them… she would do all sorts of things that I would do, but the way she would do it, I think, spoke to the students so much more effectively than I was able to do. I think that’s one of the real reasons why their scores went up.

Rebecca: She knew how to meet them where they were at in a way that, as we become more of an expert in our field, we lose touch with that.

Dominick: Yeah, that’s exactly right. So, one of the things that I’ll be interested in following is that there was a recent study that Prentice-Hall published about the Millennials versus the Gen Z students… and one of the things they noticed is that, while there was about a sixty or so percent “like” amongst the Millennials for using computers and computerization and video and things like that, that number was almost half for the gen Z population. It’s almost as though it’s such a normal part of their life, the gen Z population, that there’s nothing special… there’s nothing unique about it… there’s no value-added to doing videos… it’s the sort of normal expectation. So, I’ll be very curious to see how this sort of flipped environment works over the next 10 to 15 years when the expectation is that they’ll be seeing videos at some point either prior or post classroom experience.

Rebecca: …definitely an interesting question.

John: You’ve organized several symposium on flipped classrooms. What are some of the biggest takeaways from those symposia when you bring people from many different disciplines together? Are the results pretty similar across disciplines or do they vary substantially?

Dominick: Well, I think the first thing that struck me with regard to these symposia is how diverse the flipped environment really can be. Since active learning occurs during the classroom time, there’s almost as many different active learning strategies as there are teachers, and so no two flipped classrooms are the same… and that’s the first thing that I learned. The other thing that I learned is that you have to be committed to flip. You can’t do it half-baked. If you try to do flipping, there was one example of a professor who said “Well, all this is is what we’ve always done. You tell the students to read the material before they come to class and then we have a discussion.” So, he said “Okay, I’m going to ‘flip my classroom’ and just have them read the textbook before they come to class and then we’ll have a discussion.” He found very quickly that the students weren’t reading the textbook, so there was very little discussion going on, which frustrated the professor and frustrated the classroom and set up more of an adversarial relationship.. and it was the worst teaching experience he ever had and he said “I’m never going to flip my classroom again.” So, one of the takeaway messages, and this was reported at one of the symposia, is that if you’re really going to flip your classroom, you’re in for a dime, you’re in for a dollar. You do it as well as you can and be very concerned about what you’re putting in and what you’re expecting to get out or it can be a very very bad experience. Now, I will say this, that almost everybody when they get up on board the flipped bandwagon, especially if you’re using technology prior to classroom, it’s hard at first. That’s the other real take-home message. It takes a lot of time to flip your class the first time. But, once you do it, it actually is much more enjoyable. It’s actually easier, I think, than the regular didactic approach. Those are some of the take-home messages.

Rebecca: Can you talk a little bit more about some of the other challenges faculty might face if they’re doing it for the first time?

Dominick: Sure, yeah. The first and biggest challenge that people have is time. Don’t decide you’re gonna flip your classroom two weeks before the start of a semester. It can be the most horrible experience you’ve ever had as a teacher, because students have an expectation that each lecture is going to be there for them. As we all have various things come up or university requirements… meetings… things like that… and you may find yourself at 3 o’clock in the morning trying to record a video for dispersal at 8 o’clock in the morning. So, don’t wait till the last minute. That’s the biggest sort of thing. That’s one of the reasons why people are sometimes a little bit risk-averse with regard to flipping their classroom.

Other things that have come up: saying “Oh, I can just use Khan Academy videos or videos that are on the web” …and they haven’t previewed them, for example, and then they find that the students have a very different concept of the material than the professor has and then you end up spending a lot of time: “Well, which one do we believe? Do we believe the video that you showed us? or do we believe what you’re telling us?” and so it can create an environment which doesn’t propagate trust in the learning experience. Other issues are: online homework versus homework that you put together for the students… what are you gonna do during class time? how are you gonna fill that time. If you’re not familiar with active learning strategies, that can be very daunting. For example, you walk into a classroom and the students have already watched a video… they’ve done some homework… So, what’s your value added? If you’re not used to active learning strategies, that can be very difficult the first time that you’re doing it…. and then do I need to change my assessment strategy based on the fact that now I’m using a different kind of pedagogy? So, there are a lot of different moving parts and I think putting all those moving parts together can somewhat be inhibitory for people who are trying to flip for the first time.

John: What types of active learning strategies do you use? You’ve mentioned some group work on problems, but could you give a few examples of types of activities you use during the class sections?

Dominick: Sure, there’s group work first of all. I send students to the board. I’ll pass out file cards, for example, and ask the students to put down one thing that was unclear in the video or something that they really would like to learn about in addition to the video. So, it’s not just muddy points, but it’s also how do we expand and extend. Because in my case it’s an honors class and I want to give them a little bit more than the normal amount of material and experience, and so those are some of the things that we do. We have to make molecules and structures and do all sorts of things, and so I use human atoms. I have volunteers come up to the front of the class and they have to then make molecules. They have to develop particular structures. They have to show how they bond, how they vibrate, how they move. What they do. So, they have to actually sort of insert themselves into the molecular dynamic, if you will. They’re trying to understand that, as an atom won’t understand its environment. The first couple of times, students aren’t used to that level of kinesthetic learning, but once they get it, then I usually have a fair number of students who are willing to volunteer and come up. Because, while they’re doing that, we’re also discussing, and I’m asking the class: “Okay, so why is it that this particular atom won’t bond here? What’s wrong with this?” and so it’s a real discussion… but now using human beings as the models rather than just making stick figures or things like that.

So, I try to move students through a variety of different learning environments that engage them not only visually, auditorily, but also tactilely. and kinesthetically, and that’s somewhat unusual, I think, in chemistry classes. Because, once again, most people don’t think of chemistry as a visceral activity in many cases.

Rebecca: Actually, it sounds like a lot of fun.

Dominick: Well, and it is, and I’ll tell you I walk into my class the very first day and they’re all very respectful, because Texas students typically are very respectful and they’re honors students. So, they really want to make a good impression. I usually start my class laughing, telling them this is the last time this class will be quiet. A noisy class is a learning class, a quiet class can be a sleeping class, I don’t know. So, yeah, my class is always very, very engaging, I hope.

Rebecca: Have you ever had the moment when you’ve asked your students “What do you want to talk about today?” and then not have anything they want to talk about?

Dominick: Yes, I think every faculty member has that “Oh, my gosh” moment. I always come prepared with questions, so if they don’t have anything to talk about, then I’ll ask them “Well, what did you think about this particular part of the video?” or “Did you really understand this?” or “Let’s take this concept and move it farther” because one of the things I never do in my classroom is just do a rehash of my video. I figure they’ve watched the video. So, I might say “Well, okay, you saw this but how could we apply this in this other setting?” …and then if they really didn’t understand it, then I’ll be able to tell in a heartbeat whether or not there are real questions because they really don’t have a good sense of understanding… and then I can go back and say “Okay, at what point is this breaking down for you? How did this not work?” Silence kind of tells me that there is usually a breakdown somewhere… and so I try to address what that breakdown is and try to correct it. So, that’s what I meant earlier when I say one of the things I do during my class is try to clear up misconceptions… try to address muddy points… and just make sure that they really understand the lecture part as well. That can take anywhere from five minutes to 45 minutes. My classes are an hour and a half long. Depending on how difficult a lecture was… on what their level of understanding is… So, I want them to have deep understanding of the content and so if they’re coming in silent then I worry that that depth is not there.

John: When they see the videos, you have them take tests. Are the tests the same for each student? or do you vary the questions? Is there some randomization there?

Dominick: Well, no. So, we give departmental exams, so the exams are the same for every student, basically. I know many universities have a format and they can vary the numbers that are put in, for example, but no, my students really… they’re fairly separated when it comes to the exam, so I don’t really feel the need to give each one a separate numerical set of calculations.

John: …and it sounds like many of your students are honor students where that might be less of an issue.

Dominick: Right, and we have an honors code through the Honors College, and I tell them the first lecture if you’re cheating you’re out of here. If you’re cheating you’re out of the University… and I’ve had the unfortunate occasion to have students suspended from universities… not at Texas Tech, but at other places I’ve been… and so they understand that I’m very serious about that.

John: On those tests, do they have multiple attempts or just a single take on the test?

Dominick: Well, my tests are usually set up so that they’re half multiple choice, half free response. Because at the end of the day, I tell my students “Qe don’t live in a multiple-choice world… A. agree, B disagree… you know, we don’t… and so I need to know how you’re thinking. While multiple choice exams are expedient in terms of grading, they don’t let me know what you know. So, I give them both because some students like multiple choice exams. They think they’re really good at them. Some students really want the free response. So, it’s a mixture of both. I try to give them a rich assessment environment as much as possible.

Rebecca: So, to follow up on what John was asking, are your homework assignments kind of a multiple attempt to help learning? or is it a one attempt kind of thing?

Dominick: It’s a one attempt. Yeah. Now, what I do allow them to do after they’ve provided their answer, is that I’ll allow the question to be open so they can go back and review it, especially if the answer is wrong…. and I allow them to do that to help them review for the exams as well.

John: So, they can go back and retake it, but only the first attempt counts towards their grade?

Dominick: Right. So, they have to think about what they’re putting in there before they put it in. Because in some cases, I’ve heard stories of students who put a wrong answer in purposefully and then the online learning environment gives them hints or tells them how to work a problem just like that… and then they go back and they have another problem with just different numbers but they’ve already been coached essentially in terms of how to answer the problem. Once again, perhaps it’s because I think you have to do things right eventually at the end of the day, I really want them to get it right… and these are… once again… they’re relatively straightforward pre-class questions. Their designed as just-in-time or warm-up questions. They’re not multifaceted. The questions we’re getting in the class are really challenging problems. They’re challenging problems using an honors book. So, hopefully they differentiate between those.

John: Are they graded on the problems they do in class as well?

Dominick: They’re not. Because part of that environment, once again, is to have groups… have community… have a mentoring process… and so, the ultimate goal in that whole process is the solution of the problem. So, I’ve already tested them in the online learning environment. I’ll test them on the exams. I’ll test them on quizzes. But, in the class I want the process of how to solve the problem come forth and not the grade be the most important thing.

Rebecca: So, we always end or wrap up our podcast with the question: what are you gonna work on next?

Dominick: Well, I have a pretty active and diverse chemical education research group and with regard to flipping, specifically, I mentioned that I had a recent PhD who looked at motivation in the classroom and what we found there were there were basically three reasons that motivates students to want to do well: autonomy, pace, and responsibility. Through the flipped environment they learn how to develop confidence in their ability to learn, and secondly they liked that the class is largely self-paced and they get to watch the videos in their pajamas if they want to, for example… and finally they really appreciate the fact that they are responsible for their own learning. So, we’re going to be looking at the role of metacognitive intervention as an autonomy motivator in the flipped classroom. That is to say, if we help them think about how they’re thinking during the early parts of the flipped classroom, do they proceed to confidence in their ability to learn that much faster… and we’ll also be looking at how the flipped classroom, especially with community building activities and community building learning strategies can improve the learning outcomes among historically underrepresented communities in the sciences… though especially communities where the notion of family and community is so important in their lives… that are not necessarily in the classroom. So, those are the two areas that we’re going to be moving into with regard to flipping… and I have a number of other projects that are not related to flipping as well. So, it’s a very diverse group of questions that we’re trying to answer, but once again I think that the flipped environment is a very value-added environment for both the students and the faculty… and so I think it’s a mature pedagogy in the sense that we talk about process oriented guided inquiry learning (POGIL) being mature and peer led team learning activities (PLTL) as mature pedagogy. Service learning is another mature pedagogy that has matured over the last 20 years or so… and I think it’s now safe to say that flipping is a mature pedagogy. In fact, at the biennial conference on chemistry education, there was a wonderful paper doing a meta-analysis on flipping and the presenter showed that in terms of looking.. I think he looked at 18 or 19 different papers on the flipped environment… and he found that, in general, there’s about a 30% improvement in student learning outcomes and it’s even better in organic chemistry than general chemistry (which was surprising to me). But, nonetheless it really does improve learning for students and that, in the final analysis, is what we’re trying to do.

Rebecca: It sounds like some really interesting projects. We’ll be looking forward to finding out what you find out.

Dominick: Thank you.

John: Well, thank you. this has been fascinating.

Rebecca: Yeah, thanks so much for spending some time with us today.

Dominick: Thanks, I’m gonna have another sip of tea. Thank you.

Rebecca: Thank you
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

41. Instructional Communication

There is often a misperception that being a well-liked, kind and caring faculty member comes at the cost of rigor or high expectations. In this episode, Dr. Jennifer Knapp, an expert in the field of instructional communication, joins us to discuss strategies we can employ to make the classroom a positive and productive learning environment.

Show Notes

  • National Communication Association instructional resources
  • Mottet, T.P., Richmond, V.P., & McCroskey, J.C. (2006). Handbook of instructional communication: Rhetorical and relational perspectives. London: Routledge.
  • Chesebro, J.L., & McCroskey, J.C. (2002). Communication for teachers. Boston: Allyn and Bacon.
  • The journal Communication Education also contains many useful articles.

Transcript

Rebecca: There is often a misperception that being a well-liked, kind and caring faculty member comes at the cost of rigor or high expectations. In this episode, we turn to an expert in the field of instructional communication to provide us with strategies we can employ to make the classroom a positive and productive learning environment.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Dr. Jen Knapp, an associate professor of communication studies and an associate dean in the School of Communication, Media and Arts at SUNY Oswego. Welcome, Jen.

Jen: Thanks, John. Thanks, Rebecca.

Rebecca: Thanks for coming. Today, our teas are:

Jen: Black raspberry green tea.

John: Tea Forte black currant tea.

Rebecca: I’m having Prince of Wales tea.

John: We’ve invited you to join us today to discuss your primary research area, instructional communication. What does research in instructional communication tell us about creating a productive classroom environment?

Jen: So, I’ll start by telling you exactly what instructional communication is… and what we do. Essentially we’re talking about communication between instructors and students that enhances learning or perhaps in some way affects the learning process negatively. We’re more interested in how messages are delivered than the actual content of the course. So, we’re talking purely about communication behaviors by instructors and students and how that affects what goes on in the classroom, which should be learning.

Rebecca: Is your area of research focus only on in-classroom communication or does it expand beyond the classroom?

Jen: One of the things I research is out-of-class communication and I think maybe at some point we will talk a little bit about that, but primarily I focus on what is going on in the classroom – specifically what instructors are doing in terms of communication and how that affects students.

John: What can instructors do to create a better classroom environment?

Jen: There are a lot of communication variables related to instructional comm. The primary instructional comm bread-and-butter concept is this idea of immediacy – and immediacy has to do with increasing physical or psychological closeness between instructors and students… and the bottom line is, if you, as an instructor, engaged in these verbally and non-verbally immediate behaviors, there’s going to be more positive outcomes in the classroom for your students… specifically learning… but ultimately, what I think is really interesting, is that even on a nonverbal level, you can influence what’s going on with your students and how they are perceiving your messages… but also how they’re wrestling with the content. So, it comes in two flavors: verbal and nonverbal immediacy. We were talking about nonverbal communication… we’re talking about everything but the words. People will commonly refer to it as body language, but it’s also your tone of voice and how you use space and touch and things of that nature. E ven something as simple as eye contact can make a difference in terms of what’s going on between instructors and students in the classroom… engaging in vocal variety… but also using humor… calling students by name… all of these things can help increase the connection between students and instructors. Most people believe that the instructor-student relationship is an interpersonal relationship, or a type of an interpersonal relationship, which means you’re connected to each other in some sort of meaningful way. All the things that you value in terms of how you communicate with your friends and your family… a lot of that plays into what goes on in the classroom as well. People want you to make eye contact. People want to be around people who are funny. So, there’s a lot of research that suggests instructors that use humor in the classroom tend to get more positive evaluations, but also there’s more learning that occurs in the classroom if an instructor is using humor effectively.

Rebecca: Does that shift with culture?

Jen: Yes. All communication occurs within a context. Culture is our biggest context. Immediacy, in particular, is very culturally based. It is something that you need to be careful of. Most of the research that I do and that I’m familiar with has been conducted here in the United States with traditional college-age populations, but certainly if you were to travel abroad and perhaps you were to teach a semester away then these rules may not apply.

John: …and it might not also apply if we have foreign students here who have not adjusted to U.S. classroom climates.

Jen: Of course. Yes.

Rebecca: So, what are your biggest secret secrets? [LAUGHTER]

John: …related to teaching.

Rebecca: …related to teaching.

Jen: Oh… no one warned me that I had to divulge my… my biggest secrets today.

Let me go back to immediacy for a little bit and talk a little bit more about that and why that essentially is a positive thing. I don’t think I listed the outcomes. You’re perceived as more approachable… you are perceived as more student-centered… more responsive… you’re friendly… you’re open… and you are essentially inviting communication. So, if you engage in these types of behaviors you are going to invite communication. If you are an introvert, I don’t recommend that you try to be overly immediate because students are going to pick up on that and then they’re going to think: “Oh, well this person is friendly. This person is a good listener, so I want to spend time with them. I’m gonna visit with them. I want to get to know them.” So, you are inviting communication when you engage in these behaviors. But something you should also keep in mind, in terms of immediacy, and this is probably more of a personal choice for me… and other people may not agree… is that it decreases the status differential between you and your students. You are trying to give the perception (hopefully it’s not just a perception and it’s reality) that you care for your students… you are engaged… you are enthusiastic… they see that you’re passionate about your content… you’re moving around the room… you kind of work the room when you engage in these physical behaviors… and so it decreases the status differential between you and them. For me, I like that in my classroom. I don’t want to give the air of being the professor who has all the knowledge and the expertise and I’m looking down on everyone and being condescending. For me, I like to have… not an equal partnership… but I want my students to feel like they are a partner in what is going on in the classroom and anyone can share an idea. I can share an idea. It’s open. It’s friendly… and that’s important when you’re teaching something like interpersonal communication. You’re talking about relationships. Sometimes that class turns into a self-help class and everyone’s talking about their problems with their partner or their family. Everyone’s telling personal stories. You can’t not tell personal stories when you’re in that class. You don’t want anyone to feel like you’re being judged or that you are judging other people. So, I like to have low status differential… low power distance between me and my students… and I can get to that point by engaging in these types of behaviors. I don’t know if that’s a secret, necessarily.

Rebecca: …maybe a secret if you don’t know about it.

Jen: …it could be…

Rebecca: …not a secret anymore,

Jen: …it could be… but I think a misconception… and if you think of it in terms of power differential or having low power distance between you and your students… and some instructors might be uncomfortable with that setup…

Rebecca: Is there a difference in gender, related to this low power difference perception?

Jen: I don’t know if there’s a difference in perception but female instructors and feminine communicators… so those are two different things… are more likely to engage in immediate behaviors than more masculine communicators.

John: You talked a little bit about how instructors can create more of a sense of immediacy by walking around the classroom, by maintaining more eye contact, and by using humor. What else can faculty do to help create the sense of immediacy?

Jen: So, remember that it’s psychological closeness or also physical closeness… if you ever had a student approach you after class and they want to talk to you, and the desk is between you and that student… or the teacher station… or something like that. Something you can do in order to create that perception of closeness is to come out from behind objects. You don’t want to stand in front of the classroom. You don’t want to stand behind the little desk. If you’re in Lanigan 101 and you’ve got that teacher station, but you also have a couple of tables in the front… the student approaches, you don’t stand behind the table. You can move out from behind the table… trying to make eye contact with people in the room… smiling goes a long way in terms of just coming across as approachable and friendly… and the idea is, if people find you to be approachable and friendly, they’re going to engage in something like out-of-class communication. You’re not going to go to your instructor’s office hours if you feel like they’re an evil troll, but you will go to their office hours if it feels like “You know what? I got this thing that’s going on in my life. I need some extra time on an assignment. I feel like if I were to go see Rebecca, she seems like the type of person who would understand or who would at least listen to me” and you can do all of that just by modifying your behavior in the classroom.

Rebecca: What happens when that openness gets to a point where those conversations move beyond class-related conversations like you just mentioned?

Jen: Yeah.

Rebecca: So, that particular example is “There’s something in my life but it’s related to the class.” What happens when it goes past that?

Jen: Sure. That is definitely a risk. If you are engaging in this behavior and you are giving the impression that you are approachable and friendly and someone that listens, as I mentioned earlier, that invites communication. So, you will have students show up at your door for reasons completely unrelated to the class… and maybe it is to seek help or advice about the relationships because they’re in your interpersonal communication class… or it just might be they think you’re a friendly person to talk to. That has happened to me and I’ve sat through very awkward conversations or heard things from students that I felt like I had no business hearing. But, you know what? Maybe if you can be a force of good… or if they are disclosing something to you… if it’s something like a sexual assault or something like that, then obviously it’s much better… you don’t ever want to hear that type of message… but it’s better for them to feel as if that’s someone they can talk to you and they can confide in and then you could help them get connected to resources, or something like that. But, then there are also, on a much less serious note, students who are just looking for a friend and they’re hangers on… and they don’t understand leave-taking cues. So, you might be packing up your things to teach your next class and trying to give the signal that it’s time to go, and they might not realize that. Sometimes you have to have very direct conversations at that point: “I have to go. I’m sorry, I can’t talk to you about this any longer.”

Rebecca: You had mentioned a physical closeness, but you also said that there was verbal immediacy as well?

Jen: Right… psychological closeness… the verbal messages would be: using students’ names, using humor, telling personal stories, engaging in self disclosure. Those would be all examples of verbal immediacy… and then the nonverbal immediacy would be: moving around the room, using vocal variety, decreasing space between you and the students, using eye contact. That would all be examples of nonverbal immediacy… and ultimately this leads to affective learning… and my goal as an instructor is always to create more communication nerds. So, I did not start as a communication major, but once I fell into it, I absolutely fell in love with it and thought I cannot live my life without this… and everything I was learning in the classroom I could immediately apply outside of the classroom. Every day in the classroom that is my goal with my students: to get them to know something… be able to do something… to better their lives… better their relationships… find an internship… whatever it might be… and I just love helping to produce comm nerds… people who are quoting comm theories to me… who are analyzing their conversations or the relationships and then telling me about it… or having them explain how they taught their father about cognitive dissonance theory and then how they used it in a work situation or something like that. That’s something that I love… and ultimately affective learning, I feel, is really one of the best outcomes of immediacy and something that’s important to instructional communication: getting students to learn because they like what they’re doing… they see the value in it… they develop a positive attitude to what’s going on in the classroom and the content that you’re teaching them… and also a lot of these behaviors… instructor behaviors… Frankly, if you like your instructor, there’s a good chance you’re going to work harder for that instructor and that you’re going to do well in the class. You might get to a point where you don’t want to disappoint your instructor… but I’d actually like to ask you a question: if you could talk about some of your favorite professors and the types of behaviors that they engaged in that you really liked?

Rebecca: That’s a good question. I need a minute to think. It’s funny, but the first thing I can come up with are all the behaviors I don’t like… [LAUGHTER]

John: Yeah… a strong emotional reaction, either way…

Jen: Sure. Absolutely.

John: I think, thinking back to my college career, which was a while ago… sometime last century… many of the professors that had the most impact on me did exhibit these behaviors. They interacted with you outside of class a bit and they demonstrated some sort of passion for the subject.

Jen: …and I think students want you to care about them… for sure. I start all my classes by asking them how they’re doing? What’s going on? So, many are in clubs and organizations, so I say “What are you promoting right now? What is your organization doing? What’s important to you?” and then finally “Does anyone have any good news?” I just like to hear good news and students appreciate that… and they sometimes, maybe once a month, remember to ask me how I’m doing, which is a win I think… to get that at least once a month? [LAUGHTER]

Rebecca: If you model it and eventually eventually it’s reflected back, right? [LAUGHTER]

Jen: Yeah, Eventually. I guess that’s the theory behind it.

Rebecca: The faculty that I remember the most, or that I had really good experiences with, are the ones that I had, probably, interactions with outside of class. Those are the faculty that I felt like I could go talk to. Who maybe pushed me harder because they got to know me a little bit, to know how to push me in a way that was positive rather than pushing in a way that would have a negative impact on me. They always got more out of me. So, I think everything you’re saying was completely true for me.

Jen: Yeah. That out-of-class communication piece is really important, and before we were studying it in communication and calling it out-of-class communication, people in education were calling them out-of-class experiences. There’s a whole program of research in education devoted to this… and they studied more the outcome of those events. In comm, we study what leads to out-of-class communication more than anything else. In education, they were saying “But here’s the good news… here’s all the good stuff that happens if students are communicating with you outside of the classroom.” So, whether it’s during office hours or whether they run into you at Price Chopper, the first time you see an instructor outside of the classroom can be a bit daunting or jolting. Students think that we just get put away in a closet overnight and brought back out the next day to teach. The first time they see you it might be a little bit weird, but ultimately if they see you, they see you as human and you stop and you say “Hi Rebecca. Hi John. What’s going on? I know you’ve been playing your bass lately. What are you working on? What are you excited about?” In those little things, like you mentioned, Rebecca, they add up and they definitely make students feel better about themselves. It really helps with their development of sense of self and can also help with motivation in the classroom.

John: How would this work in a larger class setting? Can these behaviors scale very nicely? Certainly walking around can, but what else can you do if you have a class of 400 students or so?

Jen: Sure. All of this can certainly be scaled up. Now I don’t recommend if these types of behaviors or being immediate does not come natural to you, that you launch right into trying to do all these things, because students will sniff out that…

Rebecca: inauthentic…

Jen: Yeah …lack of authenticity. They will definitely sense that. The same with verbal immediacy; using humor is an example of verbal immediacy, but if you’re not funny do not try to be funny. It will not go well. But, certainly you can scale this to larger classes. Whether you’re teaching Micro at 400 or I used to teach Comm 100 to over 200 students and I want to say (I’m sure it’s not true…)… I want to be able to say that my teaching style was not that different, whether I was in front of 20 students for a capstone or 200 students for a large introductory course, because ultimately I’m still teaching the way that I think students should be taught. I’m still engaging in these behaviors. I’m still aware of other instructional variables like clarity… like credibility. All of those things are still important. It doesn’t matter necessarily the size of your audience. We typically say “the bigger the audience, the more formal your communication needs to be.” But, I think there are exceptions to that as long as you are still being authentic in some sort of way. Any of our instructional variables that you might learn about can certainly be applied in a large lecture room. There’s no set of categories that “here’s what you do in a large lecture versus here’s what you do in a smaller studio level class.”

John: I know when I teach the large class I generally get in somewhere between 30 and 50 flights of steps every class and usually two or three miles of walking, because it’s a big ways around.

Jen: Oh my gosh. Yeah, Lanigan 101 is a big room. It’s a hard room to work too, because there’s a whole sea of people in the middle that you can’t get to. That’s where eye contact really makes a difference. You just try to make eye contact with them because you can’t physically get that close to them, but you still want them to feel as if you are speaking directly to them, and you’re not trying to be everything to 200 people in the room.

Rebecca: Other than immediacy, are there other theories or principles that we should be aware of as instructors?

Jen: There are a lot of instructional variables, and I think I’ll share some resources that maybe your listeners would be interested in taking a look at later on. Something else that is important to me is credibility. Credibility is essentially believability, and if you are a professor you should be in the business of being believable. It’s important to remember that communication is about messages, but at the end of the day meaning is in the mind of the receiver, and so you can do your absolute best to craft what you think is the perfect message. However, whoever is getting or receiving that message in decoding that message… it’s going through their personal filter. It might be a very benign message, but maybe they’re having a bad day… maybe they’re really hungry, so they’re not quite paying attention. You don’t have complete control over how people decode your messages. You have to remember that meaning is in the mind of the receiver. What you might find credible is going to be different than what John feels as credible. Credibility is a perception. Whether or not I am truly credible doesn’t matter. As long as you think I’m credible, I win. I might be a complete moron, but if you think I’m credible then it doesn’t matter because then everything I say is going through that credibility filter.

We usually talk about credibility as the three C’s: competence, character, and caring. …and for some people different elements are more important. Some people (who perhaps are more logically based) competence or that perception of expertise or knowledge rules the day, always. For some people, they just want to feel like you have some level of goodwill, and you have their best intentions in mind, and that’s the caring aspect of it… and for some people it’s character or it’s honesty and trust that you are being honest and your being truthful with them, and nothing else matters other than that character piece or that trust piece. For different people, different things are important, or they’re gonna pay attention to different aspects of the message based on what they value more… whether it’s the competence the, character or the caring. So, credibility is an instructional variable and it’s not just instructional it goes across different communication contexts. But, that’s something that I think would be interesting for people to know about and to learn about power… how you influence what’s going on in the classroom… also something that can be studied across communication contexts. But how ultimately are you influencing your students? Are you getting them to do what you want them to do because you are rewarding them? …’cause you’re punishing them? or are they doing it because they feel like it’s the right thing to do and they are internalizing your message and they believe in the value of the work? …and there’s some other types of power as well… and then just plain clarity. Clarity is another instructional variable that’s important, in terms of how you structure your messages for your students in the classroom.

John: The next thing we should probably talk about is: what might go wrong or what should faculty avoid doing that might create a negative environment?

Jen: There’s a program of research in the 90s that investigated teacher misbehaviors. So, I thought it’d be fun to ask you what some of those categories are. I bet you can come up with a lot of teacher misbehaviors. So, what are things that instructors do that students don’t like? Just rattle them off at the top of your head.

Rebecca: I’m thinking. I’m a thinker.

Jen: Don’t overthink it.

Rebecca: I know, but I have to still think. They don’t like it when when you’re condescending or like a know-it-all.

Jen: Sure.

John: …especially if you’re not only condescending but wrong. So, that competence is kind of important as a factor there.

Jen: Yeah. I do want to add a fun fact… yet, also our cross to bear as people who study communication. I love producing communication nerds. I love people who are analyzing their conversations. They are putting into practice positive conflict management strategies. However, you can often get accused of applying your communication knowledge in a less than savory way. So, some people get really upset because they feel like you’re Jedi mind tricking them with your communication skills. …something to keep in mind… that as comm majors, we often get yelled at for actually using what we’re learning in the classroom… because people don’t wanna fight fair. They want to get below the belt and say mean things when you’re like “Let’s be constructive. We don’t want to be verbally aggressive. Let’s try to just be argumentative… we’ll stick to the arguments.” That doesn’t go over very well when you’re having a fight with your girlfriend. [LAUGHTER]

Rebecca: I think being late…

Jen: Yup, that’s a big one.

John: …or not being prepared at the start of class is another thing.

Rebecca: I hate when the technology doesn’t work or there’s serious user error.

Jen: For sure. Anything else on your mind?

Rebecca: They don’t like it when you don’t know their name or… that extends to… it’s not just name but gender pronoun… pronunciation. There’s a whole slew of things that probably snowball onto that.

Jen: Absolutely. You got some good ones. I thought I would touch on a couple others that maybe you hadn’t been thinking about. You did mention being condescending… but sometimes being sarcastic and using put-downs is a problem for students, naturally. Unreasonable or arbitrary rules… If you think about your syllabus and what’s in there. Your syllabus sends a message on day one. You want to think about ultimately what you’re sharing with students based on your syllabus. Inaccessibility… Students want to be able to see you out of the classroom. They want to visit you during office hours. Being late… definitely. But one I think that’s interesting, that we probably don’t often think about, is information underload. Students want to be challenged. Most students want to be challenged, and this ties into something that we’ve been talking about previously. There’s this misconception that if you have a classroom that seems to be open and friendly and you are approachable as an instructor, that that means you are the easy instructor… and I have a major problem with that. I think it’s absolutely possible for you to do all of those things to be liked as an instructor, but to also have high standards… and frankly, if you set a bar for your students and they exceed it then you should continue to raise that bar. …and ultimately having or doing tasks that the students don’t feel like are getting them to the end goal of the course is actually considered by them a misbehavior. That’s something that you would want to avoid.

Rebecca: It was a good one that it’s most definitely overlooked… and you definitely hear those conversations: “Oh, take this class because so-and-so is easy. All we do is talk.”

Jen: Yeah, there’s certainly that misconception too… in comm studies, in particular, like “What do you do in that major? …and I come from what we call “communication and social interaction” or “communication,” “communication studies.” We’ve had different names over the years. We thought CSI would be super cool and hip and turns out people are like “I don’t get it. I don’t know that is.” [LAUGHTER] We’re changing it back to “communication,” but if I tell someone “Oh, I’m a journalism professor or public relations professor or a broadcasting professor” like everyone has an idea of what that means… and if I’m the communication professor they’re like “So, you just talk all the time?” I’m like like “No, there’s actually more to it than that.”

John: Well, you do talk all the time, but it’s about something. [LAUGHTER]

Jen: We’re communicating about communication. So, it’s all very meta. Yes. [LAUGHTER] It’s a good time.

Rebecca: It’s very deep.

Jen: Yeah, it is. Of course it is, all the time.

John: Where can faculty go to find more information about instructional communication?

Jen: Penfield [Library at SUNY-Oswego] does own the handbook of instructional communication. We asked them (we being the Comm Studies department) a few years ago to purchase that so people can check that out of the library. The National Communication Association has some great links in terms of instructional communication and what to do in the classroom and how to enact certain behaviors. That is a great resource. There’s another book that I like a lot called Communication for Teachers which summarizes a lot of instructional communication literature and also talks about how to apply that to a classroom… whether it’s K through 12 or in a college classroom.

John: We’ll share links to some of these materials in the show notes.

Rebecca: So, we mentioned earlier on about talking about communication that happens outside of the classroom and we’ve hinted at a couple things here and there, but could you talk a little bit more about those out-of-class experiences and that impact on learning?

Jen: It impacts student motivation, positively. So, they have those moments…and it can just be passing in the hallway or walking through the breezeway in Marano and it’s just a simple “Hello” to a student. That’s something that they can take with them, put it in a little pocket and store that. “Oh, Professor Kane remembers my name” or whatever it might be that makes a difference. But, ultimately it gives a student an opportunity to connect with you on a different level… in a different sort of time-space continuum, if you will. Everything is crazy before class… after class… lots of people want a piece of you… If they take the time to come visit you during office hours and that’s more that’s one-on-one time that they get to spend with you to develop those relationships and certainly that can help them. Students who engage in more out of class communication tend to do better in their classes than students who do not engage in out of class communication. But, it also has… besides classroom outcomes… has better outcomes for them personally. Networking, which you were alluding to earlier… as you met with your professors, you got to know them… they got to know you… now, when they get a call that someone needs an intern or needs someone who can do graphic design work, well you and I were just talking an hour ago in my office and I know that you have this skill set, so now I’m gonna pass this opportunity on to you… because I know that you’re interested and I know that you can do the work. So, that’s a tremendous outcome for students if they take the time to get to know their professors and their professors know them, when those opportunities come past, they can give those to the students that they’ve met and they’ve spent time with… and it just gives students another way to practice their interpersonal communication skills.

John: We always end with the question: What are you going to do next?

Jen: Something that is important to me, as someone who studies communication, 1. is to always correct people who say “Communications” instead of “Communication.” No “s” just “Communication” but also to show people the value of what we study, in what we know as communication scholars. One of the committees I sit on is the Title IX committee, and I’m also a Title IX investigator. One day, Lisa Evaneski was describing some of the cases that she was seeing as Title IX investigator and she said “These aren’t necessarily Title IX cases. We’re not talking about instances of interpersonal violence or sexual assault or anything like that. They’re just, I don’t know, messy breakups…” and I’m like “Ah, we can help with that.” So, in communication, and those of us that study interpersonal communication, we talked a lot about how to treat people positively… how to breakup constructively… how to just be a good human during those difficult times… and so there’s been a group of us that are working in comm studies to create a workshop that Lisa can potentially direct people to that maybe need a little bit of coaching about how to treat people or how to be in a relationship or how to break up… but also we would open it to the campus in general. So, anyone who’s going through a nasty breakup or thinking about “maybe it’s time for me to dump this person and move on. How can I do that in a healthy positive productive way?” …how to use social media or not use social media during during those those times… So, we’re working on building a workshop on messy breakups… which will maybe eventually have a different title, but so far we’re just stuck on messy breakups.

Rebecca: I think it works.

Jen: Yeah, and our goal would also then be to turn that into some type of research as well. Something that we could could share with our discipline, in terms of how we are applying and using our knowledge as communication scholars to help solve a problem on campus… something of that nature… A dream that I’ve always had, and that I know John knows about, is to develop some sort of instructor boot camp. It would go nicely with your badging program if we could have something where people would learn ultimately how to teach… or how to best employ some of these instructional communication variables, in order to get the best out of their students. We can also talk about how to build a syllabus… how to write a syllabus… how to structure assignments… how to ensure that your messages are clear to your students… those types of things. So, one real thing that I’m working on and one thing that I would like to at some point…
JOHN… an aspirational goal…

Jen: Yeah… actually launch…

John: oI think we’d like to see something along those lines to here.

Jen: …and I do think it’s important to say I’m not the only person that knows about this stuff and that studies it so I’ve got colleagues in Comm Studies Katherine Thweatt and Mary Toale, all three of us graduated from the same doctoral program in instructional communication, so there are a handful of us that are interested in this and that are dedicated to it, along with some other great interpersonal scholars in Comm Studies.

Rebecca: I think that what’s really exciting about your workshop idea… that hopefully is not just an idea real soon… is that students will see a discipline in action… and the more ways that we can do those sorts of things on campus, the more real it is for students about how these things that seem like they’re not applicable or they’re not applied somehow…

Jen: Right.

Rebecca: …in action. Some fields are maybe more obvious than others and so the more we can be visible as scholars in the community and sharing that knowledge with the community, I think, is always really nice.

Jen: Yeah, instructional communication is a great example of an applied field.

John: Very good. Well, thank you.

Jen: My pleasure. Thank you both very much.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

36. Peer instruction

Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know (and don’t know). Did you picture a lecture class, students taking a test, or students writing? In this episode, John discusses three ways in which he has been using peer-instruction in his classes: classroom polling, calibrated peer review writing assignments, and two-stage exams.

Show Notes

Transcript

Rebecca: Imagine a scenario where students retain knowledge effectively and are active and engaged participants who are self-aware of what they know. Did you picture a lecture class, students taking a test, or students writing? If not, stay tuned, this episode explores ways to use peer-instruction to transform the learning experience.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Today’s guest is my co-host John Kane. John is the director of the Center for Excellence in Teaching—that’s not even right…

[LAUGHTER]

John: …Center for Excellence in Learning and Teaching at SUNY Oswego.

Rebecca: Yeah, woops! Welcome to your own show, John!

John: Thanks, Rebecca.

Rebecca: Today our teas are…

John: Prince of Wales.

Rebecca: Oh, that’s a good one.

Rebecca: I have Golden Tipped English Breakfast today.

John: Excellent.

Rebecca: One of the areas you’ve been teaching experimenting in, and that I’m fascinated in, is peer instruction. Can you tell us a little bit about what peer instruction is and why you’re drawn to using this methodology in your courses?

John: Peer instruction involves using peers to assist with instruction, where students explain….

Rebecca: Thanks John.

[LAUGHTER]

John: …where students explain things to each other. One of the issues that we have is that, once we become experts in the field, it’s very hard for us to express things in terms that are easily understood by students. There’s a “curse of knowledge;” once you become adept at something, it’s really hard to explain things at a level that’s appropriate to the level of understanding that students may have. There was a classic study done in which a researcher gave people a list of songs, very well-known popular songs, and asked them to tap out the beats from that song.

Rebecca: Oh, I would fail…

John: …and then before actually seeing if people would recognize it (who had the same list), she asked them to make a prediction of what proportion of people would understand it based on their tapping… and they overestimated that by a factor of I believe, somewhere around 20 times. Basically, it was purely random if people happen to guess it. But the issue is, once you hear something in your own mind, it’s clear to you, but it may not always be clear to the people who don’t have the same rich net of connections. When students are explaining things to each other, they benefit from taking a position, arguing that position, trying to filling gaps and they’re also explaining in terms that are appropriate for people at their level of cognitive development for people who have a similar background in terms of what they know and their prior knowledge.

Rebecca: Sounds like a really good way to expand and refine mental models and also just develop better metacognition. Because, as soon as you go to explain it, you realize what you don’t understand.

John: …and if you don’t understand it yourself, your peers will often help you understand. they’ll say: “Well, you haven’t considered this…” and that sort of interaction is one that doesn’t work as well when it’s instructor to a large group of students. But, it does work very well one-on-one.

Rebecca: You’re known on our campus for teaching really large lecture sections. Implementing peer instruction in a large setting can seem pretty daunting, especially to someone who teaches smaller classes like I do. What strategies do you use?

John: The most commonly used one is to use clicker quizzes… and I use a methodology that Eric Mazur developed slightly over 20 years ago, where you ask the students a challenging question… you try to find questions that about half of them will get wrong… and over time you can develop that, you can come up with a pool of questions that fit somewhere in that range… and you let students first vote on the response themselves after they’ve had a little bit of time to process it, and then you look at the results. If you see that 90% or more of them got it correct or even 80% or more, you can just go over it and move on to the next topic, because most students understand it. But, if you see that somewhere around half of them get it right and somewhere around half of them get it wrong (plus or minus 20% or so), then the next stage is to let them explain it to each other, and that’s where the peer instruction comes in. When you have students argue it and take a stand and a position on it, we get a very significant gain and improvement when we then let them vote on it a second time… and the usual practice is not to reveal the poll results or the answer until after they’ve had that opportunity to engage in that discussion.

Rebecca: Just make sure, to make sure I understand correctly: you do the poll, you see the results as students don’t see the results…

John: Right.

Rebecca: …based on their answers or their responses when you decide whether or not they do the peer instruction piece. How long do they usually talk to each other about the topic?

John: It depends on the problem and normally I will have some undergraduate TAs and I’ll wander around the class and see what they’re talking about, listen in, answer some questions from them and the TAs will be doing the same thing…. and it’s usually pretty clear when they’re coming to a consensus. You can see them reaching for their clickers or their phones and getting ready to vote, so generally it may only be a minute or two, it could be longer… it depends on the complexity of the problems. Some of the problems require a bit of effort and require some calculations, but normally they’ve already done that… so, the second stage, where there’s a discussion, you can hear the volume build-up and then as they’re approaching solutions and consensus, it tends to drop back down again. It’s fairly easy to get a pretty good read on where they are and when they’re ready to vote again.

Rebecca: I imagine that you would really need to keep your ear to the ground, otherwise chaos could ensue. Because now, if they’re finished talking about the problem and there’s still time, then they could easily derail if you’re not quick to get back to the clicker question.

John: Right, and normally the time is generally held fairly tight. I suspect sometimes it’s only 30 seconds to a minute, other times it may go up to a couple minutes, but if I see them getting distracted and doing other things, the polling starts immediately.

Rebecca: Obviously technology is your friend in this particular situation. Can you talk a little bit about the technology you’re using to manage this many students all at once?

John: Here, we’ve adopted iClicker as a campus standard, so we use that in pretty much all of the classes where we’re doing polling and there’s both a physical radio frequency clicker that students may buy or they can buy an app and pay by the semester or over four years for the use of the app.

Rebecca: How do you make sure that the cost doesn’t get prohibitive to students?

John: That’s an issue, and it’s been a major source of concern…

Rebecca: They’re not very expensive, right?

John: Well, they can be expensive. A new clicker costs somewhere around $40. A used one can often be purchased for $15 to $20, sometimes less… and the apps I think, are somewhere around $12 to $15 for a semester and I think about $35 for four years.

Rebecca: …and you can use the clickers in all of the classes, right? So if multiple faculty member(s) are using all the same system, then the investment is a good one for students.

John: …and that’s why we have a campus adoption because in places where you don’t have that, students might have to buy two or three or four different clicker systems in different classes. So, once they buy the clicker for one as long as they hold on to it, they can use it in classes for the rest of their career. Almost everyone in the economics department, for example, now uses clickers, so if they’re economics majors or business majors, it’s very likely they’ll use them in multiple courses. The cost is much more tolerable when it’s spread out over multiple classes.

Rebecca: The other area where you do some peer instruction in these large classes is in writing. Which seems kind of crazy. You have all these students in this big classroom and somehow you manage to do writing assignments.

John: Yeah, my large class generally is somewhere between 350 and 420 students. At one time, for actually about a decade or so, I was giving weekly online discussion forums. But grading that or evaluating that and providing feedback was taking an awful lot of time…probably 30 to 40 hours a week. So, I pretty much…

Rebecca: A full-time job in and of itself…

John: I stopped that a few years ago and, a few years back, I replaced that with calibrated peer review assignments. The calibrated peer review system is something that Eric Mazur talked about while he was here… a visit in 2014… and when he mentioned it, a lot of people got excited. The way the system works is that you create an assignment, you store it on a central server at UCLA, and then it’s something that other people can adapt and use and modify—it’s released under a license, which is similar to a creative common license within the system… and you create the assignment… you create an evaluation rubric for the assignment… and you have to be really careful in designing that to make sure it’s one that students will be able to apply, because other ones that do that… and then you create three sample assignments yourself: a low-quality one, a medium quality one, and a high quality assignment… and you have students submit their own assignments first (according to the rubric and guidelines you provide to them)… then they go in and they evaluate the three that you’ve done. They’re given in random order, and they’re assessed in terms of how closely their evaluations match yours. That’s the calibration part. Students receive a calibration score based on how similar their evaluations are to the ones that you assigned to the sample responses. Then after they complete that stage, they evaluate each other, using the same rubric, and a weighted average of those scores is assigned as a component of the grade. They’re graded in a number of dimensions. One is based on the weighted average of the peers, where students who had a high calibration score will have evaluations that rate more highly in evaluating other students. They’re also rated in terms of how closely their evaluations match the others during that stage. So, if their evaluation is an outlier… much higher or much lower than other students… they lose some points on that… and then after they evaluate the other three students, they rate their own work… and one of the goals of that is so that they have improved metacognition. That, by the time they go back and look at their work again, they’ve rated three works by the instructor and three assignments done by their peers and then they’re asked to evaluate their own using the same criteria. What’s really interesting about the calibrated peer review process is their grade on this is tied not to whether they give themselves a high or low score on this, but it’s how close their self evaluation comes to the weighted average of their peer evaluations. So, they have an incentive to try to look at their work more objectively, and not try to game the system… because if they score their work too high or too low, they could end up with no points on the self-evaluation stage. So, the closer they get to the weighted average of their peer evaluations, the higher the score will be on that component.

Rebecca: I think that’s an area that we often see students struggling, is being able to effectively evaluate their work or other work. So, really training them to use a rubric and understand and think about what’s important or what’s not important about particular kinds of assignments or particular kinds of work could be really valuable to students in a way that we don’t really have other systems to do that.

John: The nice thing about this is it scales really easily. There’s a lot of upfront work in creating the assignments, creating the rubric, and a really good practice is to test them thoroughly before you give them out the first time. What I normally had done is asked some of my peers to look at that, some of my colleagues to look at it, and sometimes I’d have some upper-level students were…. and this does give students a little bit more reflective practice, where they get to look at their own work a bit more critically, perhaps, and reflect on it and see how they’re doing compared to how other students are doing in the course… and I think that’s helpful.

Rebecca: I think that the rubric would probably be a challenge to make but I think what would be more challenging is putting up those different assignments that are scored at different levels at the very beginning as your calibration tool. What strategies have you developed to make those in a way that it doesn’t take forever?

John: Well, I only do this three times in a semester, and once you’ve done it once, if you design it in a way so that it won’t go stale… and I generally have students, for example, find some articles in the news in the last six months that relate to a topic that we’ve talked about, or I ask students to find some examples in their own life to illustrate behavioral economics concepts in one of the assignments, for example…. where it’s not something that they could easily copy and paste from other people’s work. Because, there is always a concern with academic dishonesty and so forth. You don’t want these things showing up on Chegg or any of those other systems, where it would be easy to copy and paste good responses. So, I’ve tried to design assignments where once they’re done, they can be used for multiple years in one form or another. I modify them each year based on how they work. But perhaps a more serious problem is what happens when students really don’t like the evaluations. One of the things I’ve done when I’ve used this is to have three of these assignments, but I drop the lowest score… because, sometimes people will get some scores back that they didn’t expect or they may have neglected to look at the rubric I sent them and they may have omitted a major part of the assignment and ended up losing quite a bit of points all the way through that. But, as long as one of the scores is dropped, they have the opportunity to learn from their mistakes and do a little bit better. But, there are procedures built-in that make it easier to catch any outliers when you have someone who is just rating everyone extremely highly or rating everyone really poorly—inappropriately highly or poorly. There are tools in it which will give you a list of all the cases where there’s a high variance across reviewers or where someone happened to be evaluated by people who had very low calibration scores… so, if you end up with two out of the three peer reviews with low scores, that’s something that’s flagged by the system. I check all the cases where it’s flagged and I tell the students if they’re unhappy with their score or if they have any questions about it, to contact me, explain why they’re dissatisfied with their score, and then I’ll go in and look at it. In nearly all cases, it’s been an issue with the students submission and not with the peer reviews. Because, while some people tend to overrate things and some tend to underestimate some of it, compared to where I would evaluate the wok… on average, it’s been very close, typically, to what I would have scored or what I would have assigned as a score. But I do make, in rare cases, some adjustments when I see that something went wrong in the process.

Rebecca: Do you prevent students from seeing the score then, until you’ve reviewed all of the scores to make sure that you’re okay with what has happened before they have access or…?

John: In this system, that really can’t be done easily…

Rebecca: ok.

John:… because what happens is they get the results as soon as the last stage is completed. I’ll send a note out saying, “Now that the stage is completed, you can review your scores, you can read all the comments that your peers have provided, and you can see what your grade is at each component…” and we have gone over that in class so they know what they’ll be seeing.

Rebecca: What kind of workload do you end up with, dealing with problems?

John: In general, when I’ve used this in the class of 360 to 420 students, there’s usually 3 to 5 students who find their grade unreasonable, and sometimes, I found the grades perfectly fine. Occasionally one or two of those, I’ll make some minor adjustments to—if something went wrong where one of their peer reviewers didn’t show up, for example, one or two of them didn’t complete that stage of the assignment, and someone was overly harsh or perhaps overly harsh in their grading, but it’s rare.

Rebecca: Can that system be used for things other than writing? Like other kinds of documents?

John: It could be used for any type of document because basically students will either write something up or they’ll submit something and it could be an image, it could be used for peer review, or calibrated peer review, on pretty much anything as long as it can be disseminated in digital format. It could be used for websites, for example.

Rebecca: Well, that’s what I was getting at when I was asking.
You also teach some upper-level seminar courses with 30 or so students. This semester, you tried a two-stage exam after talking with Doug McKee when he was on campus about it. What is a two-stage exam and how did it work?

John: Backing up a bit, I was considering it even before Doug came here because I heard the episode of the Teach Better podcast where they discussed a two-stage exam and then when we were talking here and he was in one of our earlier podcasts and we discussed this very issue, I became more interested after we talked with Doug. A two-stage exam is one where in the first stage of the process, students take the exam by themselves and then in the second stage, they do some group work– either on a subset of the questions or on some very closely related questions. It’s being used quite a bit in the sciences and there’s a growing amount of research indicating that it has been successful. Some studies have found weak results, others are finding stronger results, but it’s still fairly early in the exploration of this. The Carl Wieman Science Education Initiative has quite a few resources associated with two-stage exams. This leverages peer instruction in the second stage.

The usual process, or the most common practice, is to take the exam period and have students work on this for the first two-thirds or so of the exam time slot and then they work in a group in the last third. I did it a little bit differently than this. In my case, I gave the exam on a Wednesday and I graded the exam but didn’t get them back to the students and then I selected a subset of the questions and I had them work on them in groups on that Friday… and that worked pretty well too, they had a little chance to review in between, they didn’t get to keep the exams, but there were only seven questions on it. They could go back and review things. I didn’t tell them which questions would be on the second stage in large part because I didn’t know. I told them that two of the questions would definitely be on it, but it would depend on how they did on the other part. So, I was able to look at the exam, find the parts where they had the most trouble, and assigned those as ones for the second stage… and in general, it was a remarkable experience. It was really nice to be giving an exam and to see students working in groups of three or four, actively discussing the issues, arguing over them, trying to explain things to each other and it was a really fun experience. It was very energizing to see that much effort being devoted to try to understand concepts that students had some difficulty with.

Rebecca: I remember seeing an image of your class being really actively engaged, really talking about the core class material that you shared during your test and I think the caption was: “This is during a test!”
[LAUGHTER]

John: Yes, I took a picture of it from my phone and I think I sent that to you during the exam because it was just so exciting to see that… and it was also a reminder for myself just how well this was working. I wandered around the room and listened in on the discussions and they were all very focused and coming up with much better explanations of these things then they would have likely been able to see if it was a whole class discussion… because they were very focused, they were arguing over what was the best approach to deal with some of these problems. I could see people making connections and suddenly understanding how things they had done before fit in and pulling together a lot of concepts that they might not have done as effectively if it had not been for those small group discussions.

Rebecca: Were you tempted to join in on those conversations because they were so lively?

John: I was, but I mostly just listened in and let them work it out themselves… and in general, they did quite a bit better… and what I should have mentioned before is that the overall grade for the exam is a weighted average of the first part and the second with most of the weight being on the individual part. One of the things that really appealed to me is that typically, when we give an exam and then grade it and return it, the students who did well generally just put it away and are happy with the results and they may glance at some of the things they got wrong (if they got many things wrong), but they’re not going to spend a lot of time actively processing it. The students who did poorly tend to get discouraged, some of them may give up a bit, but rarely are they likely to go back and try to put in the effort to correct their mistakes and to see where they went wrong. It was really nice to see that processing taking place by both groups. The students who did really well the first time deepen their understanding by explaining it to others and I suspect that should increase their long-term recall of this. The act of explaining it to others in some studies seems to be really helpful in encouraging transfer, where you can take concepts and apply them to other circumstances and when you’re in a course like econometrics, you have to be able to apply the same concepts in a wide variety of topics and areas. I think it was a very useful experience.

Rebecca: I think it’s a great method to allow some time and space for a reflective practice, because students tend not to do that on their own unless they’re asked to do it and if you do it as a homework assignment, I suspect that students don’t really spend that much time doing it, but this time they spent the whole class period doing the reflection. So, that seems really valuable.

John: Because I know a lot of people will do that. They’ll have an exam, they’ll give it back to students, and they’ll tell them they can make up part of the grade if they turn it in with corrections… and many students would do that, but I don’t think that would be as effective as having the group discussion on this. Some of them were able to make very clear what they didn’t understand and then they were able to get explanations from others and sometimes the explanations were right, sometimes they were wrong, but they had to process it much more actively and that’s always helpful, I think.

Rebecca: The grade weights is what seems most compelling to me in this situation because I’ve offered quizzes in my classes, more low-stakes assignments where I let students work on it for a while and I don’t tell them that they’re gonna get to do some peer instruction as part of it, but then they’re struggling with what they’re doing and then I say, “Oh, well, you have five minutes to work with your peers to revise anything you want to do before you turn it in.” And those generally result in some pretty active conversations as well, but there still are those few students who just copy down the answer and don’t engage in the conversation… but I think if there was that wait between before and after, that would really change that dynamic. So, I think that that’s a really compelling opportunity.

John: I thought it was useful and another reason why I didn’t do it all at one stage in one day is because I’m teaching on a Monday-Wednesday-Friday schedule and we only have 55 minutes and I have quite a few students in the class who are not native English speakers and they always take more time or they need more time to process and write information in a second language. So, I didn’t want to constrain the time and make both parts of it much shorter.

Rebecca: If you encourage people to practice and retrieve that information in extra time outside of class, there’s nothing wrong with that either.

John: Exactly.

Rebecca: I’d rather the students learn the material rather than just panic about a test. What do you recommend to our listeners to read to learn more about this evidence-based practice?

John: In terms of peer instruction, Derek Bruff has a really good book on using clickers. Eric Mazur’s original book on this, which is now slightly over 20 years old, is still very good… where he describes a process of developing this peer instruction technique. Eric Mazur also gave a talk here a few years ago and we have a recording of his presentation on this. There’s a really great example in there where he used peer instruction and what was most compelling about it, and Rebecca’s heard this before, but…

Rebecca: I was there!

John: …and Rebecca was there, was he used this example where he gave a really short presentation on what happens to the hole in a plate of metal if you heat it up… and people were asked to vote on that and then they had a chance to discuss it.

Rebecca: …and he never told us the answer!

John: …and then he noted how energized people were and he said, “You were so actively discussing these things…” When he tried to go on after making a point about how they suddenly were interested in something they normally wouldn’t have been interested in… he started to go on to the next topic. People were really upset, because they wanted the answer and he finally gave the answer, but he did that deliberately to show that this sort of thing… where the students don’t know the answer but they committed to a position and they want to know if they’re right… builds a sort of interest in learning that might not intrinsically be there otherwise.

…and that’s exactly what I saw, by the way, in my exam. They were so actively discussing things that normally they’d be bored out of their minds with. So, that environment can be very supportive of learning.

Rebecca: Yeah, it really gets people curious. I remember being in that room… dying to know what version was right? People had such compelling arguments.

[LAUGHTER]

John: Exactly, and that’s why it’s really good to pick questions, with any of these things, where it’s not going to be clearly obvious, where they have to process it, and they have to make connections, and you could build a case, correctly or wrongly, for different answers, and people want to know what the answers are.

Rebecca: I mean it was key that he finally gave the answer, right? So there was some corrective feedback there, so that people didn’t continue to mislearn the information.

John: And that was nearly four years ago, and we remember that very vividly. If that was just a point in a class that was given… say, four years ago, we probably wouldn’t be talking about that now.

Rebecca: I can’t believe it was that long ago.

John: I think it was.

Rebecca: It was a while ago.

John: Yeah.

Rebecca: …and so I’m dying to know, what are you gonna do next?

John: One of the next things I’m going to do is a follow-up to something we talked about in an earlier episode, when we talked to Judie Littlejohn about the metacognitive cafe. One of the things I’ve been observing is that the use of this process by having students work to improve their metacognition about how they learn and what they’re learning… Students, at least, perceive there is being some significant learning gains from that. That’s convinced me that I’d like to do something similar in a large class, but an online discussion forum for 400 students again doesn’t scale quite as well. So, I’m going to be doing some weekly activities and I’m working with Liz Dunne Schmitt who teaches our large macro class in the spring semester, and a couple of other people: Kris Munger, and Michelle Miller, who also who’s the author of Minds Online: Teaching Effectively with Technology (and was a guest here a while back). We’re going to try to put together an experiment where we use some evidence-based methods as weekly assignments, say for ten weeks in a semester…. that’s our current plan at least)… and students will be exposed to this… and they’ll engage in some sort of reflection or some practice with one of these activities… and then in terms of evidence-based methods of learning, such as retrieval practice, spaced practice, and interleaved practice, and similar things… and then we’re going to see how that exposure along with some reasonably easily assessed activity, which could be just some short responses in a forum or it could be perhaps some online quizzes, evaluating whether that impacts their actual behavior in the class, and their actual performance in the class. One-half of the group will be exposed to those types of interventions, and the other half will be exposed to some form of standard study skills module, because most of the students in this class of freshmen and basically what we’re looking at is, if we present students with evidence on what really increases our ability to learn, whether that will result in significant change in either their behavior, or in their performance. So, we’re going to try, at least the plan, is to try to see whether that affects the number of times they take quizzes that can be taken repeatedly, whether it affects the number of times they log in and view other materials, and whether it changes a perception of how we learn. so right now we’re at the…

Rebecca: And performance too, right?

John: …and their performance.

Rebecca: And is the plan to start collecting that data in the fall?

John: The plan is to put all this together the spring, I’m hoping and then to submit a proposal to the IRB, and then to conduct the study and the fall and the spring, at least for a first stage and then we’re hoping to be able to follow these students up, to see if this has a significant effect later in terms of their grades or their persistence.

Rebecca: Sounds pretty exciting. I’m looking forward to hearing how that goes.

John: It is. I’m looking forward to it being all together and actually being implemented. I think it’s an interesting study.

Rebecca: We’ll have to have you back, John.

[LAUGHTER]

John: I think we can manage that.

Rebecca: Well, thanks so much for sharing all this information about peer instruction. I know it’s something that I’m always kind of asking you about and like to hear about, and I’m sure others will too.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

16. Student attention span

Have you ever been told that to keep students engaged you should chunk lectures into ten minute segments? Neil Bradbury, a Professor of Physiology and Biophysics at the School of Graduate and Postdoctoral studies at the Rosalind Franklin University of Science and Medicine, investigated the origins of this myth. In this episode, Neil joins us to discuss his review of the research on student attention spans.

Show Notes

  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • Bradbury, N. A. (2016). Attention span during lectures: 8 seconds, 10 minutes, or more?. Adv Physiol Educ, 40, 509-513.

Transcript

John: Have you ever been told that to keep students engaged you should chunk lectures into ten minute segments? In this episode, we examine the origins of this myth about student attention.

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist,

John: and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Our guest today is Neil Bradbury. Neil is a Professor of Physiology and Biophysics at the School of Graduate and Postdoctoral studies at the Rosalind Franklin University of Science and Medicine. Welcome, Neil.

Neil: Thank you. Nice to meet you.

John: Nice meeting you.

Rebecca: Yeah, thank you for joining us. Today our teas are

Neil: Today I’m going to go with Lapsang Souchong.

Rebecca: Sounds like a great choice.

John: Very nice, I got some of that, too. I have to keep it separate though because it has a smoky smell and it blends with the rest.

Neil: Yes it does have a strong, smoky smell.

Rebecca: And today I have vanilla chai.

John: I have Yorkshire Gold tea.
Your two thousand and sixteen paper on student attention span has gotten quite a bit of interest on a number of email lists and professional development groups and that’s where I first saw it. What prompted your interest in this topic? It’s a bit different than your usual research.

Neil: It is a little different. I have gone to various teaching institutional days and I had been told of this ten-minute rule, and naturally I accepted it because that was what the elite was telling me. And then shortly before I started looking into this, our medical school, like many medical schools in the country, was revising its curriculum entirely over the entire four years… and it was in a meeting with the Dean going through the new curriculum where he mentioned again, “I now have to stop because I’ve been talking for 10 minutes and so I need to do something different…. and I thought to myself “Well, that’s interesting, I wonder where this notion comes from?” …and I decided I was going to find out, because everyone told me ten minutes is the rule, ten minutes is the rule, but no one knew where it came from… and so I decided that I would have a look at this and if indeed it was the case I should be taking notice of this when I’m teaching… if it’s not the case, where did it come from? Where did this apparent educational myth arise? So that was really what started my interest in this… looking for where the source of this myth was.

John: I have the same sort of thing… I just took it as given, I’ve seen that said so many times and seen it in so many books and papers and recommendations, that I also took this as given. One of the things I’ve always used to judge the quality of someone’s presentation on teaching and learning is whether they start mentioning learning styles, or they put up a picture of Dale’s Cone of Learning, because those are myths that are pretty well known and pretty well established, but this one I think a lot of us had taken as a given…. there must be some research on… or we wouldn’t keep hearing it so much, and we should have known better. So what did you find when you began to investigate this? Where did this rule come from?

Neil: When I was looking through this, as you say, it’s often repeated, and people make the statements and then refer to a paper that was published many years ago… and when I eventually tracked down all the references, I did eventually come to a paper that was published in the fifties… and much to my surprise, the paper actually didn’t mention attention span at all in any of the words, which was a little curious since this is the basis that everyone uses for the ten-minute attention span. And what I found was that, in actual fact, the paper does not describe attention at all, but rather note taking…. and even more curious is that a subsequent publication by these authors even stated that they felt that note taking was no basis for discerning attention span anyway. So I think the whole propagation… and to be fair it’s the original authors, they did not say that this was looking at attention span… but somehow it’s got changed over the years… saying that they were looking at attention span of ten minutes, and that got propagated through the literature, and propagated by people who were looking not at the primary literature, but at someone else’s interpretation of the literature and it just propagates without anyone actually going back to the primary literature.

John: It’s nice that you did. It’s about time someone did and thank you for doing that.

Rebecca: I think it’s so important in the day we’re really focused on the idea of evidence-based teaching that we do remind ourselves that we should be looking at the evidence and not just taking things for granted.

John: And in your paper you also mention that there were some potential flaws in some of the early studies on note taking. What were the major flaws in some of that early work?

Neil: I think that some of the major flaws were really in experimental design. As a scientist, I spend a lot of effort pre-planning my experiments, often taking more time to plan the experiment than to do the experiment, and I found that a lot of these lacked really rigorous planning. So, for example, one study involved two lecturers that went in to observe a class, and in the paper, they stated that more often than not at least one person turned up to investigate attention. Well, if you’ve only got two people looking at student attention and only one of them occasionally turns up, it’s hard to take anything seriously that comes out of that study. So, a lot of things like that. One of the things that, as a scientist, I particularly have to take notice of is statistical rigor and we have to take care to provide instruction in what statistics we’re using and whether or not our statistics are valid. Many of the studies just state categorically, “our results were statistically significant” with no indication as to what those statistics were but then that gets repeated by the next person who just blithely states “oh, well this person states that it was significant” and just the takes that for granted, and so it gets propagated.

John: …and it might have been significant at the sixty percent level or something similar, if they don’t specify…

Neil: We don’t know because, it’s never mentioned in the papers.

John: …and one of the things you mention, in one of those early studies of note taking was that it may have been perhaps more of a measure, I think, of the content of the presentation rather than student attention.

Neil: Yes, I think it’s important when we’re considering lectures as to really fundamentally what the purpose of a lecture is, and I clearly don’t think that the purpose of a lecture is to have students take notes as the end result of a lecture. The notes should be something that the student refers to at the end of the lecture to remind them of what was covered. Now certainly note taking is important, but we don’t take notes on everything. So, for example, when I’m giving lectures I’m trying to convey a certain concept that may be difficult or it may be easy, but I try and give some illustrations of how that concept can be applied to real-life situations. Often, as it turns out, one of the things I like to do is discuss how my concepts can be applied to understanding how people use different drugs to murder people.

[Laughter]

Neil:

Now I don’t really expect students to take copious notes on how to murder someone they don’t like.

[Laughter]

Neil:So, that may be… the students would not take notes on, that’s just a little bit of fun and interest. The physiology underlying that? Then yes, that’s important to take notes on. So, you need to think about what’s being discussed. Is it important to take notes on that? Other things? No, it’s not important to take notes on. So I think there’s a balance there… that you can’t just take notes across the entire lecture, it’s really what’s being conveyed by the teacher at that particular time as to whether the notes are worth taking or not.

John: …and students could be very engaged, but not taking notes, because they are actively processing the information and making connections. And there may be no need for notes if that part of the presentation is very clear and doesn’t give them new information they need to transcribe somehow.

Neil: Absolutely agree… and just staring at a teacher or writing notes doesn’t necessarily mean that they’re focused on what they’re doing.

Rebecca: …especially if the content of the notes aren’t really being evaluated, right? It could be notes on anything!

Neil: Well, I think we’ve all done that at some point. I think we’re all guilty of that, but you’re absolutely right, just taking notes for the sake of taking note has no intrinsic value either.

John: …and in one of the studies, you mention that the students were keeping track of their level of attention where they were periodically polled and they had to indicate their level of engagement, but that was done with two different classes: one with first-year students and the other with fifth-year students, but they didn’t hold constant the instructor. So, the results were very different across instructors.

Neil: I think that that’s true broadly ,that we have all experienced instructors in our own life that have been really wonderful and engaging, and we really enjoy going to. We’ve also had instructors that we really did not want to go and listen to. So, I think there is certainly a large component of the instruction that makes a big difference in how they present. I think the other difference is that when comparing a first year with a fifth year is a huge difference. A first year really does not have any basis for knowledge, they have very little understanding of what’s going on, and so any new knowledge that they get is a huge increase. Compared to someone in the fifth year, where they may be really not learning new things, but they’re adding to what they already know, and it’s a lot easier to learn when you already know things.

John: You have more connections there.

Neil: Trying to fit things into a model that you already have to further strengthen that, that’s really not that hard to do, compared to learning a concept from the start. So, I think there’s a big difference there, between first and fifth years on a lot of levels.

Rebecca: Why do you think note taking was conflated with attention in the first place?

Neil: I think it’s conflated because it is really hard to assess attention, and we all know that when we go to lectures, we’re supposed to take notes. I think that was conflated because note taking is something obvious that we can see. We think of attention and people paying attention to what’s going on… that goes on inside the brain and you can’t really see what’s going on inside the brain; whereas you can see people writing notes. I think that was just used as a surrogate because it was something that could be measured, not necessarily because it was a valid measure of something that was going on.

Rebecca: What were your biggest takeaway is from the survey of literature that you did?

Neil: I think the biggest thing that I’ve taken away is that, as a teacher, I don’t need to worry about keeping things as ten minutes. I can focus on a concept. I was thinking about this as I went with my family to the latest Star Wars movie. I was thinking “What if I watch Star Wars only in ten-minute segments? and that those segments were not connected with each other?” I don’t think anybody would go and see that movie. So the ten minutes attention span really doesn’t hold up for that… no other experience in life do we have or do anything that’s only ten minutes. Why would it be any different from a lecture? And so I have freed myself from having to worry about going in ten-minute blocks, and I can focus more on providing a conceptual framework for what I’m trying to cover.

John: Students don’t have any trouble watching an hour and a half or two and a half or three hour movie or sitting at an engaging video game for hours at a time, and they often require lots of learning and so forth… and we don’t really worry about the amount of time it takes on those activities. When I’ve gone to talks, there have been some talks I’ve been at that, within thirty seconds, I’d like to bolt for the door (or fall asleep) and there’s been others where I’ve been fascinated for an hour or more at a time.

Neil: I think it’s the latter that we’re trying to go for… and it would be nice if a student suddenly looks at that watch and says “Oh I’ve been here for two hours …where’s the time gone? It’s been so engaging.” That’s what I, as an instructor, am really trying to aim for… is not to have a student worrying about what they’re going to be doing in the next two minutes but to realize Wow, the lecture is over. It’s been worthwhile. I’ve learned something and I’m not really bothered what the time is.”

John: That’s always great when that happens, I wish it happened more often, but it’s great when students want to stick around and find out more and they’re not ready to leave at the end of the class.

Rebecca: Sounds to me like a lot of faculty probably need to spend more time figuring out how to make their lectures more engaging and to captivate their audience, rather than simply trying to break things up into smaller pieces, right? So you might not take communication style or things like that as seriously, but perhaps something that we need to invest more time in.

Neil: I agree, I think one of the questions that I always try and ask myself is “Why is what I’m teaching important? Am I teaching it because it’s there in the textbook or am I teaching it because it really is important?” …and if it’s important, I should be able to describe why it’s important… why you need to learn this…. and it shouldn’t be abstract, it should have application to what the students are to be doing. You need to learn this because it means that you can understand what’s coming next. Since I’m teaching a lot of medical students, why it’s going to make an impact upon health care of the patients you are going to be seeing. If I’m talking to chemistry students, you need to know this is what’s important because it’s going to dictate how you design your chemistry experiments. So, no matter what you’re teaching, I think you have to come up with reasons for why what you’re teaching is important, convince yourself it’s important, and then try to convince the students it’s important.

John: That’s a point that Ken Bain made in his book What The Best College Teachers Do. He suggests that you should start with the key concepts in each class, explain to students why those are really important… you explain to them they need to be able to do these things to be able to answer those big important questions that matter to them in some way. You mentioned before that first-year students often don’t have as rich of a network of concepts, and as a result there’s perhaps a bit more cognitive load that they have to deal with. Might there be some advantage of breaking up a presentation though into small chunks and then having them actively engage with the material before moving on to the next concept — in terms of keeping the cognitive load manageable?

Neil: I think there is some merit to that. I think it should be dictated by the material, rather than a clock, and so we can look at things and see where are the boundaries that make a unit of knowledge a reasonable unit… whether you can cover that in six minutes or fifteen minutes, I don’t think that matters, but can you have a coherent unit that can stand on its own that you can put together with other things. So, content is more important than the time allotted to it.

Rebecca: I think what you said about content is really important and as a designer there’s a methodology called “content first,” and it makes sense in a classroom setting too… where you decide what the content is and then design around that, right? There’s some things are going to make more sense to do hands on, some things that are in going to make more sense to provide a lecture on… and if you let the content dictate what it is that you do, it makes a lot of sense.

Neil: …and certainly, when you’re thinking about how you’re going to organize a lecture, you have certain content that you need to cover. But the order of that content is really important. You need to start out with the foundations before you can go to the building, and to start with those foundations, that may be small chunks. I agree that can be time limited, and that you could work on, and then as you getting more knowledge more content, you start building that up into a logical, coherent molecule. But, you’ve got to start with the basics first and then you can build onto larger structures.

Rebecca: It gets funny that, as experts in a particular topic, we forget that there’s building blocks and foundations because our mental models are so much more complex. So, I think although sometimes these things seem obvious, we need the reminders to take a step back and remember what it’s like to be a novice at something.

Neil: I agree. Most of my research is focused on cystic fibrosis and I’ve been doing that for many years, but I realize when I’m teaching that to students, they don’t have the decades of experience that I have. They are not going to become experts in an hour. What I’m trying to convey to them is the broad concepts and if they get those broad concepts, they don’t need to know the minutiae that intrigues me on a daily basis. But, they have to have the broader picture and so I have to put myself in a student’s position and understand from their position what they need to learn and even what they’re capable of learning at an early stage

John: Now, have you shared these results with your colleagues and how have they responded? Has this affected how they’re teaching?

Neil: As you might expect, it’s been somewhat of a mixed reaction. Some have really liked it, some have not really been that embracing of it. I remember we had a speaker from another institution who were discussing their new curriculum and they had designed their entire curriculum around 10 minutes… and I had the temerity to ask why that was the case and the response I got was “Well, everybody knows the attention span of students is only 10 minutes.” …and so… it happens, and it’s propagated, and I think most people really appreciate the fact that there really isn’t a lot of basis for this 10 minutes, but they don’t know that. They’ve heard and been told over and over again… it’s ten minutes. It turns out that’s not the case, so we don’t worry about, this is not something that you need to worry about. Plenty of other things to worry about when you’re teaching, but this is not one of them… So, don’t worry.

John: About ten years ago here, we had a guest speaker, whose name I won’t mention, but he gave a brilliant hour and fifteen minute talk on how a lecture is ineffective…. and later at a reception I went up to him and said “that was an incredibly good lecture… it was really engaging and dynamic and everyone seemed really interested,” and he did appreciate a little bit, I think, the irony of that comment.

Neil: Yes, I agree. I think the question is, “What is the point of a lecture?’ And I don’t think a lecture is really the place for student learning. That’s not where students learn. The point of a lecture is to convey information the students can learn later, but I think there also is an important point of inspiration. It should give students an understanding of why this is important and an appreciation for students to think “this is exciting, this is really fun stuff to learn.” If I can help students make their own decision that this is fun to learn, then I don’t have to worry about making it fun to learn. They’ve decided it’s fun and they’re going to invest effort in learning it themselves. And so part of my role as an instructor and a lecturer, is to get the students to appreciate something, that I think is true, is that learning is a lot of fun… it’s a fun thing to do and that it doesn’t matter what I’m learning… whether it’s the material I’m interested in or something completely different… it’s fun to learn… and if we can convey to students it’s fun to learn, they’ll be more than happy to learn things no matter what the instructor does.

Rebecca: Such a great point.

John: It is, and I also feel the same way about faculty. That we all got into this because we were among those students who found it fun to learn, but sometimes people forget that once they’ve been teaching for a while and it would be nice if we could also encourage each other to share that enthusiasm for learning.

Neil: Yeah, I think it’s important for faculty to go and visit each other’s lectures… how we can learn from each other. I’ve been working on my lecture style for many years, I doubt very much whether is the best lecture style going. It seems to be appreciated by the students, but I can always learn and improve. I only see it from one perspective, having faculty and colleagues come to my lectures, me going to their lectures can be of huge benefit in improving everybody’s teaching.

Rebecca: Is there something specific that you’re working on right now as a lecturer to improve?

Neil: With the new curriculum that we’re bringing in, there has been a large change in how we teach, and so that’s reflecting a lot on the content that we teach, and so I’m trying to come up with ways that integrate a lot of material across a lot of different disciplines. Which is proving to be a little bit of a challenge. As a physiologist, I’ve been teaching physiology. Trying to bring in other disciplines into the classes is proving to be a little bit of a challenge, but it’s also exciting and has a lot of opportunity that I can bring a lot of different aspects in. And I think that I’m going to learn a lot and hopefully that will be conveyed to the students… that we can all learn together something that we may not have covered before.

Rebecca: Sounds exciting but also very challenging.

Neil: That’s why we’re educators, we like challenge and those go together as educators.

Rebecca: Do you think after studying attention span in this way that it’s worth more study and to have others investigate attention or is it something that we’re spending too much time thinking about?

Neil: I think there is some basis for looking at other aspects. Most of what we’re focused on so far obviously is the lecture, but as we know now, the lecture is not the only teaching modality that we use. We have small groups, we also have lab practicals, we have discussions, and so far this attention span has only really focused on lectures. I think it would be informative to also look at other ways in which we teach, look at small group learning, look at peer learning, look at practical learning. I think that is going to be an interesting avenue to explore… is what is the attention span, by whatever criteria the define attention. Is that different from those modalities than a lecture? Is it the same? Is it different for each modality that we look at?

John: …and might it also vary by the topic you’re looking at? That some topics for many students would just be more interesting than others and that, I would assume, would vary quite a bit across students as well.

Neil: I think you’re right. Not everything that we cover is exciting and interesting. There are some things we just have to do because you need that knowledge, not because it’s exciting. But, that it allows you to get the exciting parts. So again coming back to knowing fundamentals and getting that basic groundwork that allows you to do the fun stuff later.

John: …and explaining to students why they need to know those basic, less fun, things helps provide them with a bit more motivation to work through it because they see where it’s going and how it’s connected.

Neil: Yes, you always have to have an endpoint… why this is important. It’s important because we’ve got to get to this position and this provides a pathway to get there, and once we’ve got there, a whole bunch of things will open up to you that you didn’t even realize.

Rebecca: Are you planning to do any more work on attention span or are you attention spanned-out?

Neil: Well I was thinking about it, but I got bored.

[LAUGHTER]

Neil:I’m really interested in looking at these different teaching modalities to see where that applies to different avenues, because we’ve only focused really on the lecture and that certainly has been a dominant component of education at the institution. We’re moving away from that… lectures are still important, but we’re also incorporating small group learning… peer learning…. and I think it’s going to be instructive to discover whether or not attention is really critical there. How can we get students, when they’re doing peer learning, to take this into account to make sure no one’s just falling off the edge and not learning anything. So I think it’s going to be exciting to increase our understanding of how students learn, how they’re attentive, how they’re focused on what they’re doing.

John: In your review of the literature, did you think of any good ways of addressing the question of attention span? Or is it, by its nature, impossible to measure well?

Neil: I think it’s a really difficult thing to measure because it is something that is going on inside people’s brains and that’s always a hard thing to measure. Certainly, we can put people into CAT machines and MRIs. But that’s probably not a good learning environment for anything. So I think it’s nebulous by its very nature. I think the important point is: “Are the students learning anything.” I don’t think necessarily what we’re covering in the lecture is the be all and end all. Some of the experiments that were performed, that I discussed in my paper, were evaluations of what students learned that were taken immediately after the lecture. But as I pointed out, no one ever does an examination immediately after the lecture… and so those kind of studies are really, to my mind, fairly meaningless. The question is, downstream a couple of weeks later, when we examine the students on the content of that material, have they learnt it then? And, I think, that’s when we really get to assess whether they were paying attention… not by looking at whether a student is taking notes during a particular lecture, not by asking them questions immediately after the lecture, but whether they’ve really spent time going over that material, and again, and again so that they can adequately answer questions two or three weeks later when we examine them on the material.

Rebecca: So it almost might be whether or not they’re engaged and motivated enough to want to continue pursuing that information, so that they can pass those exams and things a couple weeks later… and not really whether or not they’re paying attention in the moment that it’s introduced.

John: …and that ties back to the inspirational role of lectures that you suggested earlier.

Neil: Yes, we should inspire students to want to learn. We can never just force feed students information… it’s just gonna bounce back. We have to inspire students to want to learn for themselves, and that’s what effective teachers do. They don’t teach, they get the students to learn themselves because they’re excited about learning.

John: Very good. Well thank you, this was fascinating.

Rebecca: Yeah, thank you so much for spending some time with us today.

Neil: Well, I enjoyed it and thank you for this opportunity to discuss the paper, I really enjoyed it.

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

11. Mobile technology in the classroom

Smartphones, laptops and tablets can be useful learning tools in the classroom; they can also be a source of distraction. In this episode, we discuss alternative policies that faculty and students might adopt to facilitate learning. Recent research on the relative effectiveness of handwritten vs. digital note taking is also examined.

Show Notes

Coming Soon!

  • Aguilar-Roca, N. M., Williams, A. E., & O’Dowd, D. K. (2012). The impact of laptop-free zones on student performance and attitudes in large lectures. Computers & Education, 59(4), 1300-1308.
  • Artz, Benjamin and Johnson, Marianne and Robson, Denise and Taengnoi, Sarinda, Note-Taking in the Digital Age: Evidence from Classroom Random Control Trials (September 13, 2017).
  • Bain, Ken (2011). What the best college teachers do. Harvard University Press.
  • Bui, D. C., Myerson, J., & Hale, S. (2013). Note-taking with computers: Exploring alternative strategies for improved recall. Journal of Educational Psychology, 105(2), 299.
  • Carter, S. P., Greenberg, K., & Walker, M. S. (2017). The impact of computer usage on academic performance: Evidence from a randomized trial at the United States Military Academy. Economics of Education Review, 56, 118-132.
  • Brooks, D. Christopher and Pomerantz, Jeffrey (2017). ECAR Study of Undergraduate Students and Information Technology, 2017.
  • Hembrooke, H., & Gay, G. (2003). The laptop and the lecture: The effects of multitasking in learning environments. Journal of computing in higher education, 15(1), 46-64.
  • Lang, James M. “The Distracted Classroom.” The Chronicle of Higher Education. March 13, 2017.
  • Miller, Michelle (2017). Addiction, Accommodation, and Better Solutions to the Laptop Problem. Dec. 8, 2017 blog post.
  • Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168.
  • Patterson, R. W., & Patterson, R. M. (2017). Computers and productivity: Evidence from laptop use in the college classroom. Economics of Education Review, 57, 66-79.
  • Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom learning. Psychological science, 28(2), 171-180.
  • Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24-31.

Transcript

John: Today in Oswego, it is approximately six degrees and we’ve had about four feet of snow in the last couple of days. We’re going to talk about a topic that comes up often in many of our workshops at the Center for Excellence in Learning and Teaching here, which is student mobile device use in the classroom.

Rebecca: Today our teas are:

John: Blueberry green tea

Rebecca: …. and Jasmine green.

Rebecca: So, in 2017 an ECAR Study of Undergraduate Students and Information Technology report came out and indicated that laptop ownership is up to 95 percent amongst undergraduate students and smartphones device use is up to 97 percent. I’m sure Chromebooks and other inexpensive devices have certainly increased the amount of technology… but they’re everywhere… and I definitely feel that in my classroom I don’t think I’ve had a semester yet where a student doesn’t have a smartphone or a laptop with them.

John: Another thing that the study noted is that tablet use is actually declining a little bit over the last couple of years. Still, though, about 50 percent of students that have a tablet computer as well… and most students report owning multiple mobile devices. I’ve noticed the same thing, that over the last three or four years more and more of my students in my large intro class (with about 360 or 400 students) are coming with computers. I’d say probably about 60 to 70% of them show up in class with them now.

Rebecca: Do you have any sort of policy on using these sorts of devices in your classroom?

John: I actually do encourage the use of them for specific purposes. One of the main concerns that faculty express is that computers provide a very convenient source of distractions, and I do observe that…. because I wander around the classroom quite a bit and it’s not uncommon to see students doing things other than what we’re working on in class. So that’s an issue, I think, which is at the heart of most people’s concern.

Rebecca: How do you discourage that distraction in such a large class?

John: I remind them that their time in class is limited and that it perhaps might be more effective for them if they were to use them for class related activities, which could include note-taking… or it also includes frequent use of polling. I use i>clickers in class and many of the students, probably a third to a half, are now doing the polling using either a laptop or their smartphones. What’s your policy on computers in class?

Rebecca: I teach web design courses, so obviously having multiple devices on any given student is convenient because we do need to test our work on multiple devices. So we certainly use them, in that respect. When we’re going over new material, I encourage students that want to use their devices to sit in the periphery of the room, so that they don’t necessarily distract other people who aren’t using devices… and then a lot of times we’re doing things that require diagramming and what-have-you… and so it’s just more convenient to take notes… not using a computer… unless they have a device where they can draw really easily, maybe a tablet that has a drawing application or something on it. So that’s usually my policy and most students are fine with that… and I also include in my syllabus that if you are distracting people around you then I might ask you to put it away… which I generally don’t have to do… but I have had to occasionally.

John: One of the concerns is that there’s a number of studies out there that find that when students are using computers in class, in a manner unrelated to the class, it not only harms their learning… but it also harms the learning of those around them. One of the studies on that was done by Sana, Weston, and Cepeda in a 2013 article in Computers & Education and there have been other studies that have found the same sort of results. So I use it, actually, as a learning experience because we talk about externalities in my introductory economics class, where externalities involve side effects of your actions that either provide benefits or harm to others…. and I know that when you have something that’s distracting others around you, that’s going to harm their learning as well… and perhaps we should discourage such negative externalities.

Rebecca: I think one of the things that also comes up is… distraction is one thing…. so if you’re distracted, how do you change from being distracted to having focused attention, right? ….and so… obviously the polling that you do in your classes is a great way to get students to refocus and regain attention. Are there other things that you do in your other classes that aren’t so large?

John: In my econometrics class, I will often have students work on problems in class where I’ll give them them a problem… they’ll pull up their regression software… run some results… and then we’ll talk about it in class… and generally there’s much less of an issue in upper-level classes, because the students tend to have more intrinsic motivation. They tend to be more focused on the topic in those classes as well.

Rebecca: I’d say my one class is an upper-level class but it’s still an introductory class. It’s an introduction to web design and I have a wide variety of students from first year students to graduate students that are all in the same room at the same time, but they’re all beginners and it’s an elective, so there’s a tendency to want to be there in the first place. They have some sort of motivation for wanting to learn that, so I don’t feel those same kinds of strains about focused attention that other faculty do in a general education class. But there is a lot of information out there, or strategies that you can use to start thinking about how those general education classes connect to individual majors, right? So the more you can help students find how it’s relevant to them, the more likely they will be to engage in the topic. So I’d recommend for faculty to really think about that at the start of the semester, and make that a good sell at the beginning… and connect back to that over and over again throughout the semester, so that students recognize that the general education courses really do matter to whatever it is that they’re gonna do as a professional.

John: …. and I do the same sort of thing the first day of class. I’ll generally ask my students why they’re taking this course and I’ll take responses from them sometimes I’ll do it on a poll and the results will come up… other times I’ll just have them say why they’re there and very quickly someone will say “Well, I’m here because I have to be here.” I said “ Well, why do you have to be here?” and they’ll respond “It’s required for my major.” So then the next logical question is “Why is it required for your major? What about this course is it that people think is so essential for your major that you all have to take it?” … and then that helps them think about why perhaps the things they’re learning here are relevant for their future careers…. and then I’ll tie it to that a little bit. I’ll ask them “So, what are you going to learn in this class that will help you in your future jobs that come out of this major?” and I remind them of that periodically as well… and try to make connections between what we’re doing and what their educational and career aspirations are.

Rebecca: ….This is really related to the idea of the “promising syllabus” that both Ken Bain and James Lang talk and write about quite a bit, where you’re really defining what are the big problems that the class is going to address…. and coming back to them over and over again… and that might be a strategy to really provide that focused attention.

John: mm-hmm. So, more broadly, in terms of policies, there’s three broad sets of policies that people seem to be using: one is to completely ban all mobile device use in the classroom (and we have quite a few faculty here that choose to do that), others have more of a laissez-faire policy, where students are encouraged to use computers productively… and that’s the approach I’ve generally tried to use. and others where mobile technology is is required for specific tasks… and that’s also true in my classes to a large extent.

Rebecca: Although that can be challenging, if you happen to have equity issues, right, at your university? ….and students, that three or five percent that don’t have devices, you need a way for those students to have devices.

John: Well, actually, I think the three to five percent would be with any one device that they don’t have. Virtually all students have at least one device. It’s been many years since I’ve had a student who didn’t have either a laptop or a smartphone. Most of the students come with at least one, and many of them come with both. When I teach in a summer program at Duke, for example, there are some students who may not have it. I generally bring some spare fifty-dollar Android tablets and pass them out if they forgot to bring their device. It’s not that they don’t have it, but sometimes they forget to bring it with them…. And you can do lots of activities where people share their device, as well. So the equity issue is much less of an issue than it would have been a decade or more ago. It may be in some other institutions. In lower-income areas or in some community colleges, device ownership is a bit lower, but we don’t really have much of that issue here.

Rebecca: I usually discover that the first day of class when I do a survey of my students to see what devices they have and then knowing that information, I can design activities and things that take advantage of those devices or avoid using certain devices if students really don’t have them. But I’ve also noticed, in keeping with the recent study, that my students have had more and more of those devices available to them in every class period. People who decide to have a ban on technology in the classroom… if they phrase it like that in the syllabus, and talk about that, I sometimes fear that that just sets up like rules and regulations sort of philosophy for the classroom. That sometimes doesn’t set up a classroom environment or climate that is welcoming and open. So I think if you want to discourage that use then you have to be really careful about the language that you use around that… so it doesn’t feel like a penalizing system.

John: One strategy that some faculty have used is…. you could invite students perhaps to refrain from distractive use of technology, because it could be in their own interest.

Rebecca: …and explaining that, and explaining why you might want them to put it away, I think, it can be helpful. So, rather than having a strict policy, maybe having those minor discussions here and there about when it becomes distracting, or not useful, it could be helpful.

John: … and they’re going to be living in a world where mobile technology is not going away. They’re going to be in a workplace using mobile technology and perhaps learning how to use it more productively might be useful.
One of the reasons faculty give for banning devices is that there have been some studies suggesting that students who take notes by hand perform better, or recall more information, than students who take notes on computers. One of the problems, though, with those studies (and there have been many of them) is that generally it was based on situations where students self selected, in terms of whether to take notes on computers or to take notes by hand… and those studies generally didn’t control for self selection. The most common explanation of the finding that note-taking by hand was more effective than note-taking by computers was that when students take notes by hand they don’t write as much… so when they looked at notes taken by hand and by computers, students who took notes by hand tended to write much more condensed notes, and the argument is there’s more cognitive processing going on in the note-taking… because they have to think about what the key concepts are and the additional learning is really coming from that part. The people who took notes by computers generally type much more and it appeared when you looked at the number of words typed, for example, that they were attempting to transcribe everything that was being said… or everything that was projected onto a screen (for people who were using PowerPoints, or Prezis, or Google slides, or Beamer).

Rebecca: That was one of the issues that I had with some of the studies… Specifically was that the way that they were measuring whether or not the notes were effective was whether or not they had everything that was provided copied down basically verbatim plus some as the highest score… and I would argue that that’s not very good note-taking in the first place.

John: Well, I think most of them… they actually focused on performance on some type of tests. In terms of recall, some of them were controlled experiments where there was an experimental lecture provided, and then students were tested on it immediately after, or a day or two later. But the analysis seemed to suggest that simply transcribing everything that was said is not very helpful, as you suggested there.

Rebecca: Yeah, and we know from evidence-based practices, that simply copying down, without doing any cognitive processing about what is important or what’s not important about that content…. and how does that content connect to other things, isn’t very useful. …and just reviewing your notes, in general, is also not a useful studying technique. So, if you’re only looking at those kinds of features, you’re not necessarily observing whether or not the laptop or device is inhibiting learning.

John: So, the argument for banning it is that it will reduce the number of distractions and it will force students to take notes perhaps in a more effective manner. The only problem with that logic, from my perspective, is that I remember when I was a student… I didn’t have a laptop or a mobile device, and I was perfectly able to distract myself when I found a class to be boring. There was always a notebook that I brought (I actually had only one notebook throughout my undergraduate career, I was not very good at taking notes back then) and I used it to play hangman, to write down notes to make plans for going out after class, and other things with friends around me, and I really didn’t need mobile technology to provide distractions.

Rebecca: The equivalent of texting, right? You’re still communicating with your friends by a notebook instead of through mobile devices, right? I had perfectly colored biology diagrams, I recall.
JOHN… hence, the art major thing…
….but one thing that these studies did not do is…. they found these very strong results… but they didn’t control for self selection. They observed that students who chose to take notes on computers did less well in remembering things than students who chose to take notes by hand.
There was an interesting study, though, released on September 13, 2017, by Artz, Johnson, Robson and Taengnoi (not sure of the pronunciation on those) in which there was a randomized controlled experiment conducted in, I believe, five economics classes… and they sorted students by their ID numbers. Students who had odd numbered student ID numbers were placed in one group, students with even numbers were placed in the other. A guest instructor was brought in who provided a very structured presentation in each of the classes. Half of the students, roughly, were required in the first sequence to take notes by computer and the other half were required to take notes by hand…. and then they reversed that a month later…. where they had the same groups but they switched the odd and even numbers. When they were tested on the material two days later, there was no significant difference. What this result seems to suggest is that the earlier studies were subject to sample selection bias, and the basic problem is that students who were more likely to choose to take notes on computers were students, on average, who would do less well whether they took notes by hand or by computer.

Rebecca: Students who weren’t as strong necessarily academically would be the ones who are more likely to choose to take notes on a computer… maybe that’s because they weren’t as interested in the subject matter…, maybe they didn’t really understand the subject matter and they were choosing that device, in particular, because they could “multitask.”

John: …and one of the things we do know, and this this has been found in many studies, is that none of us is very good at multitasking. The students dramatically overestimate their ability to multitask and they underestimate the amount of time they actually do multitask… and pulling away your attention to look at a text… to respond to a text… to jump to a website… to respond to a Facebook, or an Instagram ,or other notification… has a fairly strong adverse effect on your recall… on your ability to remember things… and students significantly underestimate how often they do that, as measured by studies where they’ve they were using a proxy server to actually track students’ use of mobile devices.
The main conclusion of the study is that it really doesn’t matter whether students take notes by hand or whether they take notes on a computer.

Rebecca: That leads me to kind of question whether or not students are just not very good note takers, right? We don’t spend a lot of time, especially at the college and university level, teaching students how to take good notes… and they think that copying verbatim whatever they see on the board, or what have you, is how to take notes, rather than taking the time to reflect, process, and figure out what things mean and putting it in their own words. So, I wonder how many faculty actually take any time to discuss note-taking or what might be effective in a particular class. I remember in Small Teaching that James Lang suggested using an outline and providing students with outlines that the students could fill in with blanks to follow along and focus their attention but also structure their notes in a way that might be useful. So, that’s one technique faculty could use.

John: Right, it could be an outline, or it could even be distributing… for those who use PowerPoint slides or other types of presentations …distributing those with some gaps, because if you provide students with too much detail then they tend not to use it very much… they tend not to annotate it quite as much. One of my former colleagues, Bill Goffe (who was on the show a few weeks ago), when he shares PowerPoint slides (or at least in the past when he shared PowerPoint slides) he’d leave the graphs off of it. So, he’d have some text there and some basic outlines, but then students would have to fill in part of it. So he’d share the notes in advance, but there would be things they’d have to fill in… which fills that same sort of purpose, of having a broad outline but then requiring students to make some connections on that as well.

Rebecca: Similarly, I think students generally don’t have very good metacognition, right? ….and so I find that using quizzes in class and going over those quizzes is a good way for students to fill in their notes. What I do is a series of review questions most class periods at the start of class that interleave different topics that we’ve had throughout the semester. Students take those quizzes and then I have them self grade it in a red pen… in a different color… and we go over it… and I require them to take notes about why their answer is incorrect and what the correct answer is. So I’m forcing them to do that reflective piece about what the thing means, and put it in their own words, essentially. If they just regurgitate something that I had said, and they can’t actually explain what that means, then we take that opportunity during the review questions… time to figure that out or take the time to do that.

John: In my class, probably more than half the time… sometimes 60 or 70% of the time, I’m giving students problems where they have to solve them, and I use Eric Mazur’s think-pair-share type of methodology with clickers, where they first answer the questions individually and they give their responses… and then if about half of them get it incorrect (I try to aim for questions where about half of them will get it correct the first time) then I have them discuss it and work on it and then they vote again… and generally there is some significant improvement and it’s that retrieval practice that’s probably much more effective than just simply lecturing to students. I do give some short lectures in class for 5 or 10 minutes at a time and students will often comment that they need more time to write things down and I’ll remind them that perhaps what they should be writing down is not every word that I say or every word that they may happen to see but the things that would be most helpful for them and making those connections… because I share any slides that I use with my class, so they don’t really need to try to transcribe everything. If they really want a transcription of everything, I record the class. They could watch it. We use Panopto here. They could watch it later if they need to go back and review something. They don’t really need to use the notes for that and they should be focusing on trying to use their time more effectively to help them understand the things that they don’t understand as well… and the retrieval practice that’s being done helps them… or at least the goal is, to help them understand what they don’t know… to improve that metacognition as you mentioned.

Rebecca: One of the things that I started doing after we’ve had our reading groups on campus, is spending more time in class talking about evidence-based practices, so that students are more aware of how they learn so that they can be more effective… But one of the things that this study is encouraging me to think more about, is to maybe spend a little bit of time in class to go over note-taking strategies… and when I’m finding that students are struggling with that, to kind of review those again. I do the evidence-based stuff frequently when we’re doing review questions because I often ask “Now, why do we do review questions?” before we do them to encourage students to take advantage of that opportunity… and remember that it’s not really a penalizing thing, but really about learning. So I think the note-taking is something that I want to take on maybe this spring and think about how to help students get better at that because every time I’ve asked a student who’s asked for help on something… and I say ”Well, where is that in your notes?” …their notes are like a disaster and they’re not organized. They’re in multiple notebooks, they’re in random scraps of paper or whatever. So I think finding methods to help students do that more effectively is worth the time and effort and I’d encourage other faculty to think about taking that on.

John: Normally, ask our guests what they’re going to do next, but now we’ll ask each other. What are you going to do next, Rebecca?

Rebecca: So, related to laptops and things in the classroom, I think that I’m going to be a little more cognizant about actively having students use their devices. So if they have them, that we’re taking advantage of the fact that they’re in class, and I’m definitely gonna find a way to spend some instruction time talking about note-taking. How about you, John?

John: I’ve been expanding, gradually, in talking more about why I’m doing things and to help motivate the methods that I’ve been using in class with retrieval practice, spaced practice and interleaved practice. It doesn’t always get through to students, but I’m gonna keep working on that… but I’m also going to be trying a couple of new things this spring semester. One is following up with the discussion we had with Jeffrey Riman, which came out in episode 10, I’m going to be using VoiceThread in at least one of my classes this spring, and I’m also going to flip my econometrics class a bit more than it has been flipped in the past… and I’m also going to write a couple new chapters for the book that we’re using. So I’ve got a lot of plans for this semester. I hope I manage to get them all together before the semester starts.
Thanks for joining us.

Rebecca: Catch you next time.