102. Team-Based Learning

A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, Dr. Kristin Croyle joins us to discuss how she transitioned from  explore using interactive lecture to collaborative learning, and then to team-based learning. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego.

Show Notes

  • Johnson, R. T., & Johnson, D. W. (2008). Active learning: Cooperation in the classroom. The annual report of educational psychology in Japan, 47, 29-30.
  • A discussion by Dan Ariely explaining why asking for shorter lists of positive features in a relationship can engender positive feelings appears in this March 24, 2014 video clip.
  • Michaelsen, L. K., Knight, A. B., & Fink, L. D. (2004). Team-based learning: A transformative use of small groups in college teaching.
  • Team-Based Learning Cooperative
  • Epstein Educational Enterprises, What is the IF-AT?
  • Sweet, M., & Michaelsen, L. K. (2012). Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement. Stylus Publishing, LLC.
  • Croyle, K. L., & Alfaro, E. (2012). Applying team-based learning with Mexican American students in the social science classroom. Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement, 203-220.
  • Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251-19257.
  • 74. Uncoverage – David Voelker – Tea for Teaching podcast episode discussing the uncoverage movement in history, March 27, 2019

Transcript

John: A large body of research finds that active learning approaches result in larger learning gains than traditional lecture approaches. In this episode, we explore one faculty member’s transition from using interactive lecture to collaborative learning, and then to team-based learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Kristin Croyle. Kristin is a Psychologist and our new Dean of the College of Arts and Sciences at SUNY Oswego. Welcome, Kristin.

Kristin: Thank you.

John: Our teas today are:

Kristin: Earl Grey

Rebecca: I am having Mama’s work tea, because Ada made it this morning and she calls it work tea, which means she pulls the tea bag tag out and puts it in the big cup. Also, it’s just my normal English Afternoon. But, that was a better story.

John: And I’m drinking Spring Cherry green tea.

Rebecca: We invited you here today to talk about collaborative and team-based learning in your classes. But before you do that, you’ve noted that you had a strong preparation in teaching before you got started. Can you talk a little bit about that. We’ve talked a little bit about that on the show before and how a lot of faculty aren’t prepared…

Kristin: Um hmm.

Rebecca: So, could you talk about how your preparation may be informed what you’ve done.

Kristin: My graduate program, I went to the doctoral program in clinical psychology at the University of Montana in beautiful Missoula. And that program takes the preparation of their grad students very seriously, but across several areas not just in clinical work and research, but also knowing that some of them are going to end up in positions in which there will be teaching. So, while I was there, that very first semester I was brought in, they had a structure for teaching their introductory psychology classes where graduate students were assigned our own classes where we were the instructor in the classroom, but we had a supportive network around us. So, the syllabus was already there, the textbook was already there. We collaborated in writing tests. We had a structure of TAs that supported us and they would have recitation sections in which the TAs also received development. And we joined in that so we could see how more hands-on kind of things could be done with students in smaller groups. We even assigned our final grades together. And some of those pieces are pieces that are areas of skill that people don’t often think about developing. So, that first semester, all I had to do was think about working within the structure: How am I going to handle the day-to-day teaching and learning in the classroom? I didn’t have to worry about course design because the course was already designed in front of me. And I also didn’t have to, at the end, think: “When you assign grades, is that rigid? Do you really have to follow the exact, you know, 90/80 that it is in the syllabus? Or what if there are natural breaks around 88 or 89? Is it okay to flex that? What kind of power does an instructor have that is fair to students and evaluation?” I got to do all of that in a collaborative setting with a very experienced faculty member as a guide. There was also a credit-bearing course for teaching psychology that we were encouraged to take… which I really enjoyed. And then I was given opportunities to function more independently. When they needed a stats teacher over the summer, and they knew I was living there over the summer, I got to teach on an adjunct basis, but still with the support of faculty around me. So kind of putting students in the deep end, but with a high level of support around them, I felt very prepared when I was done with the graduate program to enter into an assistant professor position. And I still appreciate the preparation that they gave me.

Rebecca: I think with the preparation like that you’re probably far more willing to experiment and do new things as a faculty member too and to maybe even break away from what faculty around you are doing. Do you find that to be the case? Or were there other faculty doing some of this collaborative work in the department that you were in?

Kristin: Yes, and no. One of the experiences I had at my previous institution, which was the University of Texas – Pan American that then transitioned through a merger to be the University of Texas Rio Grande Valley. I was talking with a colleague in another department about the kinds of things we were doing in the classroom. And I still remember him saying “Oh, I didn’t know you could do that in the classroom and that was like teaching.” He had a very restricted idea of what teaching was, and what would be acceptable to colleagues, which he had never had the opportunity to test with other people around him. And that was something that I arrived from day one… that you talked about your teaching… that you can do many different things in the classroom. And it’s all teaching, as long as you are trying to work with students to create a learning environment and they are learning, then it counts as teaching. So I did come in with a much more flexible idea, then certainly some of my colleagues who hadn’t had an opportunity to ever have those discussions. And of course, some people are hired into departments in which those discussions don’t ever happen, so they may persist with those misconceptions for many years.

John: Or throughout their entire careers at times. [LAUGHTER] The scaffolding that was provided is really nice, because we’ve talked to a few people who’ve been in teaching training program or had some training in graduate programs. But usually, it’s not quite as structured as that and that’s a nice feature.

Rebecca: Yeah, I came from a program like that, but it was like very front loaded. It wasn’t that ongoing…. So I felt a lot more prepared, because I did have a lot of those experiences, but I didn’t have that same kind of supportive network throughout. Which is incredibly valuable.

Rebecca: So, you want to take us through what some of your collaborative experiences have been in the classroom and the ways that you set up some of the team-based learning exercises, maybe starting with what are those?

Kristin: Sure. So kind of the way that I journey through my teaching, particularly when I was an assistant professor, I felt comfortable in the classroom, but I didn’t feel expert. I felt like I was still trying to figure out what was going on, which is a perfectly fine way to be and a good state for learning to occur. So I felt like I was a talented lecturer, like I can engage students. I teach in psychology, I also think psychology is naturally very engaging, but part of that is because I really love the field. So, I felt like I could engage students and that they would listen and that they would be interested. But I started to become dissatisfied that there was always a core of engaged students and I had no idea what was happening with the other students in the class. And then sometimes I would be disappointed when we have tests or homework. Everyone said they had no questions. Clearly that was wrong. I was wondering how do you engage the majority or all of the students in their learning so that they aren’t coasting through class believing that they understand until they really don’t. And then I also felt like I was kind of fooling myself into thinking that students were with me when they were not with me. So I had an opportunity at that time to do some intensive cooperative learning training along the model of Johnson and Johnson collaborative learning. And that model from the University of Minnesota, it focuses on the importance of cooperation in the classroom, and that in cooperative settings, students learn more, develop a stronger sense of self efficacy around their learning; that they together are able to achieve more than they would individually. And it also has impacts on retention… that if students are feeling like they are individually known and valued in the classroom by their peers, they’re more likely to continue showing up to class and to develop relationships outside of the classroom that supports them along the way. So through that training, it was intensive, it was like eight hours a week, one day set for several weeks. The very first day, I could see what a difference I was going to make in my classroom. So, for example, I was using group assignments in class and they had all the same disadvantages that group assignments and most classes have, because I had no idea how to structure the group work so that it would be successful. I was doing group work to save me grading time, honestly.

Rebecca: That’s why a lot of people go to group work.

Kristin: Yes! Without understanding that all I had to do was some structural changes, and then it would actually be effective for learning as well, instead of just saving me grading time. In that cooperative learning training, I learned how to structure intensive group work that could be the length of an entire semester, or it could be the length of a single class day. I learned how to structure less intensive moments of team time. So how do you do a think-pair-share that works versus how do you do a think-pair-share that doesn’t work very well. So, that within the course of that training… actually just within a few days… I suddenly had, instead of 10% of the students in the class engaged on a daily basis, I had 100% of the students engaged on a daily basis. So, that was a huge breakthrough and I continued that way for several years.

John: What were some of the structural changes that you made that did lead to increased engagement?

Kristin: So, the cooperative learning approach of Johnson and Johnson, is kind of theoretically heavy, in the sense that they outline the pieces that are necessary for strong collaboration to occur. And then they turn it over to you as the instructor to say, “How do I build those pieces in?” So, for example, they emphasize positive interdependence as one of the essential components of cooperative learning… that when you create a group and a group activity for them to do, the activity has to be structured in such a way that each person is necessary to contribute. You can’t structure it in such a way that you can have three people talking when one person is only needed, and there are specific recommendations on how do you structure it so that everyone is needed. At the same time you have to build an individual accountability as another required component, so that, even if each person is needed, people can still slack off, say, “Yes, you all can’t do as well without me because you need me, but I don’t really care about what is happening here.” There has to be a level of individual accountability that’s also built in. Along with that, some of the skills that I thought were most important, they build an emphasis on group processing and social skills, so that if you have people consistently working together in class, they may not have developed the social skills to do that effectively, especially over time. You can work with someone for two minutes on a think-pair-share and really be bad at social skills. But, if you have to work with them over an extended period of time on a project and things are going south in terms of group conflict, it’s the instructor’s responsibility to help them to develop the social skills to work together. For example, on the first day of class, when I first start having students talk to each other so that they know that’s going to be a pattern in the class, I give them something quick to talk about. And I say introduce yourself to the person next to you… spend two minutes talking about this. And then I’m going to ask you about what you talked about. And then I run around the class real quick… pair up people who aren’t participating, introduce them to each other so that they understand this is a part of the class. So, then I follow back. So, what pieces are important there? …that I explicitly instruct them, you turn with your body… you actually make eye contact. And I will point out as people first start doing this, look at these two people, they are looking at each other, because many times students won’t do that, and it’s very hard to have a cooperative interaction if you don’t make eye contact… and I will say, “Who was the person you talked to? Tell me their name.” So they understand that I was serious when I said, introduce yourself, tell me something about them and that there’s individual accountability through just random calling on… that they need to participate in the cooperative portion. And then there’s also the self-reinforcing aspect of it that five minutes later, when I say to talk about something else, they realize they already know somebody in class, they have a connection. The next day, when I come in, they’re not quiet, they’re already talking to each other, they’ve created those connections.

John: A nice thing about that, too, is for people who are uncomfortable talking about themselves in class, having one person tell you something about the other person, it’s a little bit less pressure, it’s a little less revelation to the whole group. There’s some evidence that that type of thing is more effective in providing a more comfortable environment.

Rebecca: Kristin, can you also talk a little bit about a specific example of a cooperative activity where all of the members are held accountable, and all have a role? …just to provide an example for people who have less experience.

Kristin: So cooperative learning can be divided into informal and formal cooperative learning. Informal cooperative learning tends to be much shorter activities that can be done kind of on the fly if you already have an idea in your mind of how you might want to do that. Formal cooperative learning tends to be more intensively structured… longer-term activities. So that could be a single class session. If you’re going to do an activity that takes an hour, that would be more formal… or if you’re going to do something that takes an entire semester. The pair-and-share that I just talked about is an example of informal cooperative learning. Something like a jigsaw classroom activity can be structured as a formal cooperative learning activity. And it already shares almost all of the components: there’s individual accountability, because each student is given a specific role. There’s also positive interdependence, because the success of everyone depends on each person doing their role. So there are ones that are already structured with a built in component. The pieces that aren’t built into something like a jigsaw classroom activity, would be the group skills and group processing, and the ways that you can build that in. You can, for example, ask groups to reflect on what went well. I typically emphasize that more than asking them to reflect on things that went poorly, because asking to reflect on what went well tends to maintain a positive atmosphere, but also helps them to cover both bases at the same time anyway.

Rebecca: …or realize that my list for what went well is not very long… [LAUGHTER]

Kristin: Right. So, a common group processing thing I would have students do after their first more lengthy or more formal cooperative learning activity would be: list three things that your team did well together and one thing that you could improve on. And another thing I might ask them to do is to provide positive feedback to each member of the group at the end of the activity. And the kind of feedback that they provide is usually pretty specific, and helps to shape their behavior throughout the rest of the semester. So when they say things like, “I like it when you disagreed, and you said that this other thing would be a better way to go” that provides important feedback, and it helps to encourage better processing going forward. But I will go around and give individual social skill feedback too. But it’s usually things like, “Oh, I see you’re sitting so far away from your group, I’m not sure they can hear you, let’s scoot your chair in so that they can hear you.” Or I might ask, “Oh, do you know this person’s name next to you? What’s her name?” …and we’ll make sure that people maintain the social and cooperative connections that enable to do that kind of good group.

John: Just as an aside, it’s useful if you’re asking about things that went well, to keep the list fairly short. I’m reminded of a study that Dan Ariely talked about where they did a controlled experiment where in one case, they asked people to reflect on three things they liked about their partner and another case to list 10 things I liked about the partner, and then they surveyed them on the quality of the relationship. And those who were asked about three things generally rated their relationship with a partner fairly high. But when they were asked to come up with 10 things, they struggled with that and they rated that relationship lower. So keeping the list short…

Kristin: Right.

John: …is really good so you don’t…

Kristin: There’s kind of analogous thought about keeping things like gratitude lists. If you list too much stuff, it can have a negative effect, because you start to identify things that you really don’t think are that important, and it makes you think the whole thing is less important.

John: And if you want to get the opposite effect, ask people to list 10 things that were bad, and then they’ll struggle beyond the first few. You talked about having continuous relationships or persistent relationships with collaborative learning. Did you try to keep the group relationships consistent for the same groups throughout the term? Or did you vary that?

Kristin: I varied it. There are some good data to suggest that in collaborative learning… they refer to them as base teams… that base teams have a persistent positive effect, particularly on things like student engagement and retention throughout the semester and throughout the year…. that you have a team that is expecting you every day. But when I was doing cooperative learning, I didn’t restructure my courses. I restructured the day. Does that make sense?

Rebecca: Um hmm.

Kristin: So I didn’t have a reason for base groups. And I felt strange imposing them on the students without a reason. Besides, they would maybe be socially a good idea. I had to completely rebuild my courses from the ground up before I started using base teams. And that’s when I transitioned to team-based learning.

John: …and in team-based learning, persistent teams are recommended as part of the process.

Kristin: Absolutely.

John: Could you tell us a little bit about this transition to team-based learning. What prompted you to introduce that? …and how it worked?

Kristin: So I was happy with how courses were going. People were interested and engaged. I had students telling me, “I know every single person in this classroom.” and when you’re teaching a class of 30, or 40, or 50, that’s unusual. “I know everyone in here, I feel really supported.” I feel like things were going well. But I was unsatisfied with what I was teaching. I wasn’t clear, in my own mind, about what persistent learning outcomes I wanted for my students. I had not sat down and really thought through if I were to follow up with a student in a year or five years, what would I want them to recall from this class? What would I want them to be putting into use in their lives or in their careers? I had never thought that through. And I was fortunate enough to run into team-based learning at that time, right as I was primed to start thinking about this questions. Team-based learning originated by Larry Michaelsen. He was coming from the perspective of enrollment increases. He had been assigning some pretty challenging work. He was a faculty member in business. And as his course enrollments increased, he started to wonder how can you maintain the same kind of interesting, really challenging in class… by case work, for example… with a large enrollment. So he developed team-based learning to address that piece, but it also requires you to completely rethink the design of the course. And to start from the course outcomes: “What do you want the persistent outcomes to be?” …and then structure the course forward in that way. So in team-based learning, after you make a decision about your course outcomes, and what you really want students to be able to do, then you structure the course in a modular fashion. And each module has certain steps. So the beginning is student preparation, then when they come into class, you test. You say, “okay” …and it’s called the readiness assurance process. So you want to know what students are ready to do after they’ve individually prepared, and what they’re not ready to do. So they prepare, they test. And then, since it’s a team focus, they also test as a team. After that you have a good idea as an instructor, what are they ready to do? What are still the fuzzy areas? What do they really not get at all? What are their competencies as a team already, even if every individual student doesn’t have it, and then you can do some corrective lecturing, basically, so many lectures that fill in some of the gaps. And that’s all part of the readiness process, because you’re getting them ready to do some interesting application work in class. And the rationale for that is… and actually what I had been doing prior to that, was giving interesting application material to work on at home individually, while doing lecture and cooperative learning in class. But the interesting application material was actually the heart of the course, and the much more challenging piece. So it was better to bring the hard piece where they needed support into the classroom. And the piece they were ready to do, which was to do their own self study back into their own lives. So you do this readiness assurance process to make sure they are ready for interesting application, and then the majority at the time for the module you spent on application. Doing that after I had already worked with cooperative learning was really helpful, because all of that application work is done in a team setting. So when you already have some experience with how to build teams, how to maintain and develop their social skills, that’s really, really, really helpful. That’s a short version.

John: One of my colleagues, Bill Goffe, who was on one of our very early podcasts, noted that when he gave the group test, the performance always went up significantly, so that they could see the benefits of the peer discussion that was part of that. And he was really impressed with it. And he noted that, oftentimes, if a student didn’t show up for class one day, they get a hard time from their classmates from the group because they let the group down. And he said his attendance had never been better than when he was using a team-based learning approach.

Kristin: Absolutely. And a lot of people who do team-based learning, use the same methodology for doing the team testing, which is honestly really cute. It’s a scratch-off form. And the scratch-off form is used so that the team gets immediate feedback on each option. So on any particular item in a multiple choice test, if they want to select “B” they scratch off “B.” If it’s not there, then they continue to scratch until they get the right answer. For one thing, they love it. But also they are getting immediate team feedback. If this person is not speaking up, if they say I think it’s “B” and then they stop advocating and then it turns out to be “B” later than the team immediately knows, by the time they get to the next question. “Okay, we need to incorporate more feedback from all of our team members, wait a minute, this person who’s not speaking up actually has a lot to say.” In the course of just a few multiple choice questions, it brings their team development forward leaps and bounds. And they kind of have fun with a scratch off, which is also a bonus.

John: And it also gives them incentives to come prepared and to listen to other people in ways that they might not otherwise.

Kristin: Yeah, and their team will give them grief, if they say “Oh, I don’t know, because I didn’t read,” their team members will be like, “But we are depending on you, you need to read, we all read.”

John: And it also gives them a little bit, perhaps, of improvement in metacognition because they’re getting that immediate feedback, and it’s being coupled with the reactions of the peers. So if someone was insistent on a wrong answer, and they dominated that discussion, they might be a little more careful in the future and more willing to listen to the other people and reflect.

Kristin: Exactly, and it doesn’t have to wait till next week, it can happen right away. Right on the next question. The team application activities are also structured in a particular way. In team-based learning, they talk about the four S’s for the application activity, the first one is that you have to select a significant problem. So what they’re working on is something that will be important to them, something that they will identify with, or that they recognize is worth their time in thinking about and trying to think through. The second one is that they need to be working on this same problem. You can’t say teams one and two are working on this, three and four are working on this, five and six are working on this. Third one is that they structure in so that they make a specific choice as the outcome. Because it’s easier to solicit team feedback if everyone is making a specific choice rather than having kind of an open-ended narrative response. And it helps to stimulate whole group discussion as you’re moving. Now it can sound like it’s limiting to say that you have to make a specific choice, but you can do in a very broad way. And the fourth one is simultaneous reporting. So all the teams are asked to report at the same time on what the choice was that they made, so that they can’t piggyback off another team who’s putting in effort. So, as an example, one of the courses that I taught in the psychology major in Texas was the tests and measurements course in psychology, and test and measurements starts with a stats review. They’ve all had statistics, it usually comes prior to tests and measurement. But it’s the first time that they have an opportunity to work with statistics in kind of a decision-making way. So you start with a stats review. So one of the activities that I would do, I gave them two hypothetical first-grade teachers with how many questions 10 of there students got right on a spelling test. And the two distributions had the same mean, but one was fairly normal, and one was highly skewed. So they had to do their quick statistics review… Do the mean, median, mode and standard deviation describe the shape of the distribution. But the question I was asking them was, “If you were the principal, which teacher would you offer an after-school tutoring program to for extra pay? And which teacher would you potentially nominate for a teaching award?” They found that question to be a really interesting question. For one thing, students think a lot about what good teaching is, and what constitutes a good teacher. So they already come in with very strong opinions. And they also understand the complexity of, you know, if everybody’s passing but people aren’t excelling, is that good teaching? Whereas if most people are failing, but a few people are getting an “A” is that good teaching? …and how the data contributes to good decision making, but can also be kind of manipulated to contribute to decision making in not such a good way. So instead of just saying, “Let’s review the stats, here they are,” it was a question with a specific choice that they simultaneously reported on. And then we could discuss together. And of course, their answers are different. There’s different rationales in both ways. So then we could discuss together what their rationale was, if they want to debate they can debate a little. It generates a lot of student enthusiasm, and everybody’s doing it instead of just 10% of the class.

John: And once they’ve committed to an answer, they have a stake in and they really want to know, that’s something we’ve seen a lot of things we’ve talked about in the past, too.

Kristin: Absolutely.

Rebecca: When you were doing the team-based learning, were you sticking specifically to problems that were on a class-by-class basis still, like you were discussing in the co-operative setting, or were you doing some longer term activities that went across multiple class periods?

Kristin: I had the… what I consider gift… to often be teaching in a three-hour time slot, which is my very favorite time slot. So I would have activities that would extend two or three hours, but typically not between classes, I found that to be more of a sweet spot, at least for me. At my previous institution we had a very high commuter population. And I promised, in both models, that I would never ask them to do something out of class with their teams, that was one of my rules… that it was just simply too burdensome for students who have multiple outside of school commitments… family and work, or living potentially 150 miles apart, which was not unheard of. I promised them no out of class stuff. I structured that intentionally so that the individual preparation that they were doing, they could do anywhere on their own time schedule, but they were expected to be there. And their team expected them to be there to be able to engage in class. And it was also one of the ways that you talk people into it, when they say “I worked with other groups who were all slackers and we would always set times and they wouldn’t show up.” And I said “That’s not going to happen in here. We already have a time we’re all going to show up together.”

John: And the philosophy that’s very similar to the flipped classroom approach where you let students do the easy stuff outside and then give them assistance with or have them work in a framework where they’re getting more assistance with the more challenging issues.

Kristin: Absolutely. I think TBL [team-based learning] is definitely a flipped classroom approach.

Rebecca: I think the other thing that helps too with that model… of making sure you’re not working outside of class… really helps students with really different backgrounds start working together, because you might have students who are more traditional who are on campus. And so for them to meet outside of class is often not such a big deal. But then if you have students who are working or have families, and there’s a disconnect in the class, even, between those two populations, that helps make that more obvious and work a little bit better,

Kristin: Right. Absolutely. Yeah. And I didn’t want to set up anything where people were made to feel like unvalued team members, because they couldn’t do what was asked of them because of other commitments. Since that was in my control, I wanted to make sure that people felt welcome.

Rebecca: I’ve tried to even do that with long-term projects. In the field that I’m in, we tend to do things that go across class periods, but there’s always the “Are we going to do this outside of class or are we going to do this inside of class, and I try to have them do anything that needs to be collaborative, and decision making, in class, and then things that can be done on their own, even if that means doing some creative work, or whatever, outside of class. But those are independent things that can be done for the same reasons. And I find that students will try to manipulate that system, so that they’re gonna: “Oh, we’ll just do it outside of class, because we don’t want her to know whether or not we’re on top of something,” or whatever. But I call them out on it, because it’s really devaluing some of that exact thing. People have other commitments and things.

John: You mentioned, you started to use a backwards design approach where you started with the things you want them to remember five years later. Did you have to cut back on the breadth of the coverage in the class, to some extent, by doing that?

Kristin: Yes, I did. When I was going with the straight up cooperative learning approach, I did not have to cut back on the content at all. Without the full redesign, I found I could cover the same amount of material in straight lecture versus in a cooperative setting. But it was all coverage. It was just a different kind of coverage. When I approached it from a backward design perspective, and I really was able to focus on the objectives that I thought were important, I did have to reduce the amount of things that we were covering. I have no regrets about that, of course, because I completely recognize that covering material isn’t just covering it. What are students going to do with something I covered in class? They didn’t cover it, I was the one who was learning it and talking about it. So I’m much happier with an approach in which I am consistently hitting on the objectives that I really want them to recall, and that they are working hard to apply those throughout the semester.

John: If they’re not going to remember it passed the final exam, covering more material isn’t terribly useful.

Kristin: No.

John: We talked about that in a previous podcast with David Voelker, who talking about the coverage approach in History…

Kristin: Right.

John: …which is the same logic.

Kristin: Exactly. And I actually now consider that to be a complete waste of time. So why am I spending class time on something that I actually don’t really care if they remember, it’s not the most important thing to me, and they really don’t care if they remember.

Rebecca: You have some compelling arguments for why team-based learning and collaborative learning are good options. If one wanted to start moving in that direction, what would you suggest their first steps be?

Kristin: For team-based learning, there are a couple of great books that are very easy to approach. There are several great resources for team-based learning. Larry Michaelsen published a book in 2008, for example, that covers that from front to back. It gives examples of applications in different disciplines. There’s also a book published a few years later on team-based learning in the social sciences and humanities. That also covers the basics, but has applications that are more specific to social sciences and humanities. Team-based learning has really caught on in medical education and in business education. So in the original book, there are more application examples that are in MD preparation or in business schools. So if you’re looking for other examples, the second book might be a good choice as well. And that one is edited by Michael Sweet and Larry Michaelsen.

John: And in fact, I read your article, or

Kristin: Oh, did you?

John: …your chapter in there as background.

Kristin: I’m glad someone read it.

John: Now I have to read all the others. But, I, at least, did read that. It was very good. So for faculty who are moving to this, what are some pitfalls that they might run into? Or what sort of problems might they encounter?

Kristin: Team-based learning as being a much more structured approach… Michaelsen does a really nice job of laying out the pieces that he thinks are critical. And I agree they are critical. So, for example, he talks about explaining, and testing the model with students on the first class day, and you cannot skip it. So the very first class day, I give students an example individual application test, like they would get for their readiness assurance. It includes basic psychology knowledge that may or may not be present in the culture. So they have some chance of getting some of them right and some not. And then I have them do it as a team. And the team scores, of course, are always dramatically higher than the individual scores. And the team testing process is so much better. [LAIGHTER] It’s more pleasant and interesting and collaborative than they expect it to be. That simply going through that, it allays many of their fears about what a team is going to be like to work with. Plus, when they see that the team has tripled their individual score, they’re like, “Hey, maybe I could depend on other people to help me learn, and maybe this will pay off for me.” So going through an explanation of what the rationale is, having them experience it a little is really, really critical in helping them stay open minded while they experience it. And then regularly throughout the semester, I will keep reinforcing them with those messages. I’ll say, look at this amazing thing you guys did. You used all the intellectual resources around you, and you analyzed this difficult problem and came up with some great solutions. I’ll remind them how much they’re learning and what kinds of challenging tasks they’re able to do as a team when they have the preparation to do it, which helps as they’re starting to think “Well, wouldn’t it just be easier if I could do this by myself?” It helps them to kind of remember, ”Well, yes, but you wouldn’t be doing this, you would be doing something not as challenging, not as integrative.”

John: and probably not learning quite as much either&hellp;

Kristin: Yes. He also emphasizes an aspect that is also emphasized in cooperative learning… of helping the teams develop and giving them feedback, helping them give each other feedback. That’s also really critical, especially very early in the semester, as they’re starting to develop group norms and bond together to make sure that you don’t short the time in class for them to have some group processing time and to build their team skills. So, for example, when I taught last spring, I had a student who came to me after I think it was the second week. So it’s very early in the semester, and she said, “I really need to reassigned teams. My team hates me, they won’t make eye contact with me.” She was really upset. And I’m reluctant to reassign people teams, because often what they’re experiencing, they take with them. It’s not always a function of that team process. So we talked some, and I tried to get a handle on what she was experiencing. I knew where she sat, I had an idea of the team composition. And I asked her to try one more day, just one more day. And then we would talk about reassigning her teams. And that day, I was sure to build in plenty of time for group processing, where they talked about what they were doing well as a team and something to improve. Their team turned around immediately. She was a relatively assertive person, which I already knew. I knew that she could handle this. So she went back to the team. She was able to talk with her team about not feeling heard. They immediately turned around in the way that they were with her. And by the very next class day, they were a relatively high functioning team. They did well all semester. They brought doughnuts for each other. I mean, it was a really nice supportive group. What they needed was the time in class to do some processing. And if I, as the instructor, had been moving too fast, and not giving them time to do that, and not giving them a prompt to do that, it would have been a really negative experience for her. So, also building in time for the team to develop and prompts for them to do that.

Rebecca: So you mentioned liking to have a three-hour teaching slot.

Kristin: That’s my favorite. It’s not required.

Rebecca: So, in that amount of time, how much time would you designate towards this group processing, for example, to give people an idea of what that proportion or the amount of time to dedicate so that you don’t shortcut it and you don’t rush through it?

Kristin: If I were to do an activity that might take an hour, I might spend 10 minutes for group process, it doesn’t have to be very long, or even five. And you don’t have to do it every time, you could do 10 minutes after the first one or two more intensive activities, and then not do it for another few times… and another five minutes just every so often to help them resolve their underlying dissatisfactions and to recognize that what they’re doing is not just application activity, it’s also group interaction. So please take time to do both. Another really important required component that I didn’t mention is peer evaluation, I always incorporate peer evaluation as part of the grade.

John: How did you form the teams in these classes?

Kristin: They’re heterogeneous, first, with a very open process so students can see it happening and know there are no shenanigans… that this is all very open… talking about the rationale that people of different backgrounds bring different strengths. So you want a group that has people of different backgrounds, so you can have a larger kind of learning base between you. So usually, I’ll pick a few characteristics that might be important in that kind of background. And I will line them up around the room based on those characteristics. And if it’s 200 people, it’s a really long line. And then we count off. So when I teach introductory psychology, students who have had a high school psychology class usually are starting a big leg up on the other students. So I’ll include that as a characteristic. Sometimes I’ll include the distance that people are coming from, because then they have different experiences, depending on what class I might also include if their student athletes, just because if you put too many together in a team, then they’re all gone on the same day. They have interesting backgrounds, but they also have patterns of attendance and of absence that need to be adjusted around. And we’ll count off all the way around so people can see how the teams are made. But heterogeneous teams are really, really critical. Having students with pre-existing relationships will throw off the team process in a way that automatically excludes people that don’t have pre-existing relationships… plus they tend to be lower performing teams. And I don’t want to set that up on purpose.

John: One of my colleagues once did this in a class of, I think it was about 350 students, but he just sorted them alphabetically. So he had them organize himself that way, and it was a fairly long process. But, it was kind of amusing for those of us wandering by and just seeing…

Kristin: …this huge line… Yeah.

John: He didn’t do it that way In the future, he used other criteria.

Kristin: I’ve had colleagues that I’ve talked with that think that this is a long process. It’s not. You can sort 200 people in 10 minutes, and then you’re done for the whole semester,

John: Doing it alphabetically…

Kristin: takes a lot longer.

John: …can be more challenging, because they were self forming that… it didn’t convert rapidly.

Kristin: The other thing I never do is I don’t put the students who didn’t come the first day into a team, because there are characteristics about why they didn’t come the first day. If you put them all together in one team, they share some of those characteristics… It tends not to be a very high performing team. So I make sure they’re sorted out among the other teams. But that was one of the things that I learned in cooperative learning. That, before I did cooperative learning training, and I was assigning group work, I would assign people based on if you didn’t come the day we did the assignments, you were in another group. And that group typically did not do very well. And as an instructor, it’s my responsibility to create a learning environment in which students can excel, it’s on them whether they do their part. But if I’m setting up a team in ignorance, with predictable characteristics, so that they’re going to have a failure experience, that’s on me to correct. And it’s not on them. So afterwards, I felt guilty when I had come to a new realization. But, yeah, it’s my responsibility to set up an environment in which those students can be successful in their teams.

John: In your chapter in that book, you mentioned that when you switched over, it did affect your course evaluations a little bit. Can you tell us a little bit about that?

Kristin: Just a little bit. But yes, it did. So when I was doing straight lecture, I was shooting for engaged lecture. And in psychology, you can build in little experiences, especially in introductory psychology, where the topics are changing frequently, you can always build things in that are kind of interesting. You can do a little optical illusion here and a little bit of memory trick there. And there’s these ways to build it in, but it is still basically straight lecture. And I got high evaluations for that. I was careful about trying to build those in every day, you know, every few minutes. And when I went to cooperative learning, where it was essentially the same approach, but in in a much more engaged and cooperative fashion, those evaluations stayed very high. Students knew each other, they were happy in class. When I went to team-based learning and I was actually asking every student to participate all the time, and be prepared in class in a way that their contributions were much more obvious than mine. My evaluations did drop just a little bit, not a lot, but a little. And I am grateful that I was teaching in a context where I knew that my department wouldn’t care. They were more interested that I was doing good teaching. And they understood the many factors that influence student evaluations. But I also recognize that it’s incumbent on me to help students understand how they are learning, what kinds of things encourage learning and retention, and then you kind of let the student evaluations fall where they may.

John: When I read that, it reminded me of that study that came out a few weeks ago from Harvard in their physics program, where they found that students in active learning classes did demonstrably better on tests, but they perceived their learning as being lower. So there was a pretty strong inverse relationship between their perception of learning and actual learning. That seems to be fairly common, there have been a number of other studies where what students think to be most effective, is often not what most enhances their learning.

Kristin: Right.

John: Do you have any other advice for our listeners, who might think about using either collaborative or team-based learning in their classes,

Kristin: The one thing I would say is that teaching a cooperative learning or a team-based learning structure class is a lot more fun. You have to be willing to give up control, because when you’re lecturing, you have absolute control… meaning even that students can’t ask you weird things, because you haven’t opened the door for that to happen. But when you structure the learning experience, and then you give up the control to the students, it is an exciting environment to be in. I wasn’t as tired when I was coming out of class. I was energized, you could feel the difference in the room just walking into class… they were excited and talking with each other. When I would circulate around before class started, they’re talking about the class instead of talking about other stuff. It completely changes the environment in the classroom in a way that I think really matches what I expect out of a university education for students, it creates a environment of intellectual enthusiasm around the topic that you’re teaching.

Rebecca: So we always wrap up by asking what’s next?

Kristin: That’s a great question. So right now I’m 100% administrative. And since I’m in a new position, in a new institution, I’m gonna spend some time figuring out all the newness pieces. But I’d like to go back to the classroom, at least for a course here and there when I can. There’s nothing different about students than there is about people. So I also think often about how what we do in the classroom, what we understand works and what we understand doesn’t work, how that applies in administrative settings as well. We know for example, that people tend to try and find the shortest path. So if they’re trying to learn something, they want to put in the least effort to learn it. If you ask a faculty member to do a task for the department, they are obviously going to choose the easiest path to do that… not necessarily the best path. So how do I take the experiences of learning and teaching, that in some ways are better understood to an environment of administration that in some ways is not as well understood? What kinds of lessons can I apply there as well?

Rebecca: Well, thank you so much. It’s been a really interesting conversation. I’m sure it gives a lot of people things to think about as they move forward in this semester and future semesters.

Kristin: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

98. Developing Metacognition

Many students arrive in our classes with relatively little understanding of how they learn. In this episode, Dr. Judith Boettcher joins us to discuss how well structured project-based or problem-based learning activities can help students develop their metacognitive skills so that they become more successful as learners.

Dr. Boettcher is the author of many books and articles on higher education and has long been a leader in the field of online education. The Online Teaching Survival Guide: Simple and Practical Pedagogical Tips, co-authored by Judith has been an important resource for faculty transitioning to online teaching. At Oswego (and many other institutions), many faculty have been using materials that Judith has developed for ACUE (the Association of College and University Educators).

Show Notes

Transcript

John: Many students arrive in our classes with relatively little understanding of how they learn. In this episode, we examine how well structured project-based or problem-based learning activities can help students develop their metacognitive skills so that they become more successful as learners.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Judith Boettcher. She is the author of many books and articles on higher education and has long been a leader in the field of online education. The Online Teaching Survival Guide: Simple and Practical Pedagogical Tips, co-authored by Judith has been an important resource for faculty transitioning to online teaching. At Oswego many of our faculty have been using materials that Judith has developed for ACUE (the Association of College and University Educators). Welcome Judith.

Judith: Thank you very much, Rebecca and John, it’s great to be here.

John: We’re really pleased to have you here. Our teas today are… are you drinking tea?

Judith: I actually… yes, made a special point. Here’s my cup, which you can’t see. But I chose one of my favorite teas, which is a lemon and ginger tea from England, of course,

John: the Twinings version.

Judith: No, this is a Hamptons tea from London.

John: We have the Twinings version of that in our tea collection.

Rebecca: I’m drinking Jasmine green tea today.

John: And I’m drinking oolong tea today.

Judith: Sounds good.

Rebecca: Judith, your mug looks really interesting. Is it abstract art. Is that what was on it?

Judith: It’s actually pears.

Rebecca: Okay, I only saw the bottom part of it. I can see it now.

Judith: Yeah, right. Well, I will confess that at some point, I finally decided to clear every vendor cup out of my cupboard.

Rebecca: That sounds refreshing.

Judith: Yes.

John: I have vendor cups all over… in my vehicles… in my offices, everywhere.

Judith: Well, that was part of my retirement process that I went through. I said, “Okay, that’s it.”

John: One of the things we had trouble with is picking a topic because you’ve worked on so many topics, but we settled on having you talk a little bit about how students can work to improve their metacognition using project-based or problem-based learning. But before we do that, could you talk a little bit about what metacognition means?

Judith: I would really like to John, partially because I had this book with the title of metacognition that I was reading when I was with a family event. And one of my relatives said, “What in the world is this? Meta what?” [LAUGHTER] So, an easy way I like to think about metacognition is the definition of just it’s thinking about thinking. It’s a definition, I think. that we can all just really grab on to and we can really use. But then I kind of like to expand that definition into one that’s in the How People Learn report that I go back to pretty regularly and that is that “metacognition is the process of reflecting on and directing our own learning.” And I really like that one, because it’s got the two steps, I think, what we want to really kind of focus on with metacognition, and that is reflection… really stopping thinking, pausing… and then actually directing our own thinking, because that leads to action. So then we have reflection and action, which I think is the core of metacognition skills.

Rebecca: One of the things I think we often talk about in education context is this reflection piece. And we always tell students to reflect, but we don’t always give them the time and space to do that.

Judith: Yeah. [LAUGHTER] Very much.

Rebecca: So can you talk a little bit about project-based and problem-based learning and how metacognition connects to those rather than standard ways of operating in the classroom?

Judith: Yes, I would love to do that. In fact, that was one of the first things when we’ve got into online learning, was that there was a real struggle as to how do we maintain the security and everything of people taking tests. And so it turned out that we decided that one of the best ways of gathering evidence of student learning was not by doing these tests… that we would actually have the students do projects. And that kind of evolved into the following process. And that is that in some of the work I’ve done for ACUE, and also in the book, I’ve mentioned that I really like to design a course that starts with the students selecting and doing a project, actually in week one or two of the course. And then that students actually focus on that project throughout the entire course. And that’s the mechanism by which we gather evidence of student learning. It kind of also avoids this whole process of buying papers and buying other kinds of things. Because you really have milestones along the way.

I’m going to stop and talk about the first step for just a moment. And that is choosing a project. One of the wonderful things about choosing a project is that then students actually have to stop and then think about the kind of project that they want to do. It gives them an opportunity to actually customize a course to their interests, which starts getting past that big motivation problem that sometimes teachers might say, “Well, how do I motivate them?” Well, we give them the opportunity to choose something that is of interest to them. So they choose a project, and then they actually write up what that project is that they want to do. But that’s not the end of selecting the project. Before the project is really kind of finalized that they’re going to be working on, they actually then sit down and talk with some of their other students and the other peers. They switch and swap their proposed project descriptions, so that they actually talk out loud about the project. And then hopefully, by talking to their peers about it, they get some additional ideas, and they refine it a bit. And then it goes to the faculty member. So the faculty member doesn’t get it right away, but it goes through this first the individual students thinking about it, and then the other students thinking about it, and then the faculty member can take a look at it. And that’s only milestone one that can take up to about the third week of a course. But by then, hopefully, that gives them a real focus of the course rather than having just the topics thrown at them from week one through week 16… that they’ve got a real focus of: “Oh, how is this going to affect my project?” We can come back to that I just want to mention briefly then there’s like four other milestones, five other milestones for every project throughout the course. And the first is that project description, then the second one would be planning how do I plan to do it, which is a really important metacognitive skill. And then another milestone would be some type of other “Just checking in, how are you doing?” …kind of a thing. And then there’s two final things. One is where they actually share their project with the other students, like in a mini conference, whatever. And then the final thing is the actual final thing, which might not be a paper, it might be a video or an interview, it could be one of a number of things that would go into their student portfolio then. But, that gives the student a focus throughout the entire course that way. That’s a lot, isn’t it?

Rebecca: It is a lot. [LAUGHTER]

John: When you have them give feedback to each other, do you recommend that that’s done synchronously or asynchronously If it’s an online course?

Judith: You know it’s actually best, I think, that the students do it somewhat like a brainstorming event. What you and I are doing right now here with Zoom, students can do with FaceTime, they can do it with just an online gathering chat, whatever. If they’re really, really busy online students, and they have to do it asynchronously, that can be done too with email. I mean, that works too. But it’s often really great for the students themselves to talk out loud. We don’t have them talking enough, I think. They read passively, and they kind of think and everything else, but we don’t have ways for them to use their voice to talk about what they’re thinking.

John: It sounds like the project’s really nicely scaffoldied. But how do you bring in the metacognitive development? How do you get students to improve their metacognitive skills? Is that something they’re explicitly thinking about? Or is it something that’s done as part of the structure of the project?

Judith: Well, of course, it depends on how the faculty member wants to do it. As I had the chance to go back and look more intensely at the metacognitive skills, it occurs to me that thinking metacognitively is such a basic intellectual skill in many respects, particularly now in our 21st century. It’s as fundamental as reading and writing. And, you know, I think we need to look at our entire curriculum from pre-K through whatever as to really how do we explicitly teach and model and coach metacognition skills, which includes with that initial project proposal, we can talk about the fact that what they need to do is think and be sure to build on their interest to give them the criteria and coaching as to how that’s going to really work for them. It’s really a problem when they have to select a project. So what are the constraints on that problem? What are the features or benefits that they’re going to get from this? …and to build that into the assignment and actually writing out the initial proposal and then meeting with their colleagues and peer students, that’s all part of the coaching and the modeling of the metacognitive process.

Rebecca: I think your emphasis on talking about things is really interesting. Because I find that students often will passively write something, and it makes no sense to anybody. But as soon as they try to explain it out loud, they realize it makes no sense. Because when it’s in writing, or they’re just reading in their head, they don’t often have that realization. But as soon as you try to say it out loud as a sentence, it’s not structured in a way that makes sense. They catch themselves or they realize, “Wait, there’s a big hole in what I’m talking about here… nobody knows what I’m talking about.” You can tell it by other people’s faces looking blankly back at you.

Judith: And then they realize that they don’t know what they’re talking about.

Rebecca: Yeah. [LAUGHTER]

Judith: So yes, in fact, that’s apart of the power of talking with their peers about the project, because the peers would most likely say, “Well, wait a minute, why did you choose this? Why are you interested in this?” And so that’s when they have to dig more deeply. It almost goes back to that Socratic questioning, “Why do you think that? Why is that important to you? Do you think this is going to make any difference in your life?” One of the things I really recommend in online learning is that the students, when they read and look at the objectives for a course, to stop… and in the first week, actually, to have the students process those learning objectives, and then personalize one or two of them and say, “This is what I really want to make certain I know when I finish this course. This is how I’m going to be a different person.” Thinking “What skill am I going to be able to do when I finish this course?” They kind of set that goal and set those expectations early on. Hopefully, students setting those goals and objectives become again, more personal to them, and something that they can check themselves. And then as they go through the course, “What progress am I making on this particular goal? How is this coming?” …kind of fun.

Rebecca: Are there ways that you would suggest a faculty member model metacognition early on in an online course?

Judith: Well, I think a really easy way for faculty to model meta cognitive skills is… say, for example, in biology or one of sciences, one of the things I love to look at is the biography of a scientist and looking at, “Well, how did they come to this point? What made them think X, Y, or Z as opposed to A, B, or C?” and we find it’s really rooted in their personal lives and their thinking that they’ve been doing. So faculty member can somewhat do the same thing, in fact, very early on in their introduction to the course. Now, what do you teach Rebecca?

Rebecca: I teach web design.

Judith: You teach web design, okay. So in your introduction, then, to your students, you can say the reason I love web design is as follows. And you can go back into your life and your experiences and say, This is what happened to me… and this is what I was thinking… this is how I got to this point. So you really speak out loud, and you share your processes by which you arrived at that point. And we can do that for any kind of thinking… as we’re talking about an experiment. We do want faculty to share their expertise and their dissertation, for example, they say, “Wow, you know, this is why I’m interested in this. And you know, after 25-30 years, I’m still interested in this because of this.” So they share their thinking processes. Another really easy time is when students ask them a question. And obviously, this can happen online in the synchronous activities. And the students might ask them a question about, “Well, gee, Professor, so and so what does this really mean in this instance, or in this context?” And the faculty member can say, “You know, I really am not sure about that. And I would like to, before I answer that, I would like to do a little more research and thinking, and I will get back to you on that.” So it sends the message that the process of thinking is something that we keep doing all the time.

Rebecca: I think that moment of admitting that you’re not always the expert in everything in the moment, is always a good thing for students. And they respond really positively to that.

Judith: That’s right. It also gives them the opportunity to say, “You know, Professor, I would like a little more time to think about that. Let me get back to you on that.”

John: How can we tell that students have improved their metacognitive skills? I like the idea of having the peer instruction do that. Or to work with students to help them recognize what they know and what they don’t know. But how can we measure that? Or how can students know that they’re more metacognitively aware? How can they observe improvement?

Judith: Well, one of the things I did as I was thinking about this was, you know, I was kind of preparing for you asking me the question, “What are the actual metacognitive skills?” and I came up with five thinking skills that we really want to encourage and model for students. The first one is an easy one. And that is, number one, we really want to encourage students to think. I know that sounds somewhat simplistic, but oftentimes think about in the online environment, we give them an assignment and students, they don’t even read it, really, they kind of scan it. And they kind of assume without thinking that that’s what that assignment is going to give back. And so one metacognitive skill is to pause and stop to really take some time to think and process. In fact, I kind of like to think about the assignment as a briefing. You know, we use briefings in business and politics and detective crime solving and all the rest of it we use briefings. Maybe as a faculty member, you want to encourage the students to think of an assignment as a briefing and a briefing is, “Well, this is what we know now, what do we want to know about next? How are we going to find out what that next is? And when you finish this assignment, what do you expect to know or think about or become clear about?” So step number one is really pausing and thinking. There’s a book out recently called, was it Kahneman? It’s that Thinking Fast and Slow or Slow and Fast?

John: Yes, Thinking Fast and Slow. It’s a great book.

Judith: Thinking Fast and Slow, right. Well, you know, he makes the point that we live in such a fast-paced world that our first response to anything is that, “Oh, I know the answer to that.” It’s like Jeopardy, and you really don’t stop and pause and think. And so, because it’s really easy, that’s the short-term memory, we don’t have to think about it. We know that in order for learning to occur, that we really have to stop and give time for information going into our short-term memory to get into the long-term memory. The only way anything goes from the short-term to the long-term memory is for us to connect what we know already with the new information coming in. And you know what? …that takes time. So that’s the one thing for all of us. Time is so seriously, we don’t want to take the time. But if we don’t take the time, the only way learning occurs, you know, if we grow dendrites in our brain, and so if we don’t take the time the dendrites don’t grow, and nothing lasts. It’s kind of a fast bullet shot, so to speak. And then we forget it. As soon as we use it in a sentence, it’s gone. So the idea that we have to really discipline ourselves to stop and think and process, what is it that I’m needing to do? What is it that I want to learn? What do I want to get out of this? And when I finish this assignment, how will I know that I’ve really finished the assignment. Which brings us to the next thinking skills. One of the things I really liked, I use myself a great deal. And this is kind of a hard won practice of my own. And that is that, whenever I was given an assignment whether it was at work or wherever, whenever I had a hard time getting started… you know, getting started is sometimes one of the most difficult parts of anything, right? So anyhow, I finally figured out that one of the reasons it was hard getting started was that I didn’t quite know what it was I was going to do. I finally realized that my best practice is that I visualize what it is going to be when I finish. I get requests to review articles for journals, etc, etc. Well, I know now that my visualization works is that I’ll be done when I finish answering all those questions, you know, read the article, finish answering all the questions, I compose my response back to the editor. And that’s when I’m done, okay? So with any kind of a project, we want to encourage the students to say, “Okay, what is my assignment going to look like when I’m really done? When I have finished reading this article, or reading this core seminal research project? When will I really be done? Will I be done when I can explain the research to someone else? Will I be done when I can actually implement these ideas in a web design project? Is that when I’m going to be done? Just what is that project going to look like when I’m really done?” So we just celebrated the Apollo landing on the moon, in July. I just had the recent opportunity to see the Apollo 11 program that they created out of the original footage from everything that was happening at Cape Kennedy and Houston… the pictures, it showed like, literally, a room full… it almost look like 100 guys, and they were all guys at that time… 100 guys sitting at computer terminals, and you think when they worked on that project, how did they envision success? They had to envision success as actually a man being on the moon. How did he get to the moon? What did he land on the moon in? And then how are we going to get him back from the moon safely? I mean, think of all the things that had to be planned and worked on and everything had to be coordinated to make that thing happen.

So thinking skills, we have to think, we have to visualize, and then we have to plan. Once we know what that final vision is, we need to then plan each of those steps along the way. And again, we can model some of that and coach that by building the planning into the assignments for the project. We definitely want the students to give me a plan with the date, the milestones, the resources they’re going to be needing,]… if they need to make appointments or interviews when that’s going to happen. So we help them realize that projects just don’t happen. They happen after all of these various steps. And then, of course, we build in step number four, evaluating and pausing to then debrief each step along the way: How am I doing? Do I need more time? What else might I need? What else do I really want to know, if I do have enough time? I mean, this really does happen with web design, right? We get to a certain point, we think of it’d be great if I did this. Great if I did that. Do I have enough time to do that? Do I have the knowledge to do that? Or do I need to learn a new skill to insert that into that. So those are all questions that we want to ask encourage the students to plan for and to ask along the way. And then of course, the final one is the final debriefing. When you hand something in, you get feedback both from your peers and from the faculty member. And in many online courses, you do have these little mini conferences where you invite alumni or experts or just friends in to say take a look at this and see what you think.

Rebecca: I like your framework of the briefing. And it’s actually one that I use in my classes pretty regularly.

Judith: Oh, great.

Rebecca: When I’m doing long term projects, I have students on a weekly basis do basically a little briefing of what did they do? What do they need to do? What are the planning steps? So not just like that big scope of the whole project, but on a very routine basis, checking in with their schedule and checking in with their plan and what they’re struggling with to demonstrate what they’re doing, and what their thought processes is as they’re working on it. So I asked them, “What were the big design decisions you made this week? And what did you base those on?”

Judith: Wonderful. In fact, I sometimes like to use the example of Mark Harmon, the actor in NCIS. I love those programs, actually. Like every morning, he kind of just drives into the office and says, “Okay, what do we have.” And each one of his team members have to then report “Okay, since we saw you yesterday, this is what we’ve done. And this is what we know.” And then from that briefing, decide on their next steps, “Okay, we need to do X, Y, or Z.” It’s a really kind of a nice example of planning.

John: Building it into the project just seems to make an awful lot of sense. If you want students to improve their skills, have them apply those skills and structuring it so that they’re doing it is a very reasonable way of doing this.

Judith: And I love the fact that, Rebecca, as you were saying is to ask the students to say what have you been thinking for? And what made you arrive at the decision that you’d like to do something a little different there? So yeah, again, verbalizing the thought processes, is part of the metacognitive abilities.

Rebecca: I think one of the things that surprises students with that assignment is that they don’t realize that they’ve made decisions.

Judith: Oh, interesting.

Rebecca: That happens in projects. But also, if they’re writing a paper, whatever, they’ve made decisions, but they don’t necessarily think of it like they’ve made a decision. They’ve just kind of moved forward. So it causes them to stop and pause and do that first thing that you were talking about, and think for a second before they move forward.

Judith: Yeah.

Rebecca: I do have to say, that since I instituted that, my students are far more articulate when they’re talking about their projects. [LAUGHTER]

Judith: When they’re doing their own projects, they get their passions involved. There was another thing I wrote recently on curiosity, how important curiosity is and how we want to really build that in. In fact, one of the things when I was doing a workshop, I suggested to faculty, I said, “You know, let’s get the students to stop answering questions. Let’s get them posing questions. And let’s give them problems that, in our discipline, we don’t know the answers to, because what fun is that? Ok, so we set up the situation that we the faculty know the answers, and then they have to figure it out. So switch it around a little bit and say, here are some problems and we don’t know the answers, how am I to approach this? And then we’ll give them the challenge.” And that really, particularly John, I’m sure you’ve seen this with the TIP program, the students come up with things that you never would have expected.

John: It’s one of the reasons I enjoy it so much.

Judith: Yeah.

Rebecca: So one of the things that I think can be a challenge is if you’re doing a big project throughout the semester, we don’t want it to end up being just one big high-stakes assignment. Are there methods or ways or strategies that we can manage bigger assignments to have some lower-stakes moments or ways so that there’s not so much pressure, and that they’re allowed to make mistakes and improve and learn?

Judith: Yes, and I’m glad you asked that. Actually, somewhere in a couple of tips online that I wanted to refer to. It’s, I think, tip… is it 38 and 60, that talk about project-based learning online… that’s something to think about. But I also have a chart that recommends the grading process for an online course. And so you assign points to each of the milestones in the project. So you assign points with the project proposal and selection, but you also then have smaller team-based events where it’s worth a little bit. Also, the discussion board is a really important aspect of online learning. But it’s hard. How do you evaluate and grade that? So I recommend that about 15%, even 20% on the discussion board. And it’s pretty much a given like the classroom discussion is… so long as students are reasonably consistent about following the rubric for the discussion boards. So you’ve got a little bit there, you have also shorter essays and shorter leadership opportunities in an online course where they might summarize a week’s discussion. So there’s other kinds of activities within an online course. At no point is everything dependent on that one big project, but it does require the students to invest time and energy in that.

John: Many people often think of a need to develop metacognitive skills most for our less able students, thinking of the Dunning-Kruger effect, and so forth… that those who know the least often overestimate their learning by the most. But I think those issues may apply for students at all levels.

Judith: Yeah, we assume that great kids, actually, automatically have these skills and think that way, and they don’t.

Rebecca: Which I think is also a really great thing to talk about, too, because we often talk about metacognition, and helping students who are struggling, do better. But it’s also a great way to challenge students who are already doing well, to do better.

Judith: Absolutely, a great point.

John: For the last 32 summers, I’ve been teaching in the Talent Identification Program (or TIP) at Duke University. And one of the things I’ve observed is that many of the students there have generally been able to breeze through their regular classes without ever having to really learn how to learn. So they sometimes face a little bit of a challenge when they arrive at TIP, and suddenly they’re faced with a challenge. But a really nice part about this is that it’s an ungraded program, so that they can develop their learning skills without worrying about what sort of grades they’re going to get in their classes. And it’s much better to do it there than it would be at some future point, perhaps in a physical chemistry class, or a differential equations class, or some other class later in the career, when they haven’t really had to develop those metacognitive skills that are going to be useful in their future. Developing metacognitive skills really are important for students all along the spectrum.

Judith: That’s a really good example too, John, it’s an interesting phenomenon to talk about, really.

John: How can we tell whether students have higher levels of metacognition? Or how can we tell when students have not developed their metacognitive skills?

Judith: Well, you know, this is obviously a great question. And so let me share just a little bit about my thoughts on the students. How do we know when the students are totally clueless about their thinking processes? And I think one really red flag, particularly in online courses, that these are the students who rather consistently post comments on the discussion board: “What are we doing now? What was the assignment about? When is this? Oh, there’s a rubric, I didn’t know that.” These comments often are the ones from students that they really haven’t taken the time to really read the assignment and to make plans on that. These are also the students that we know for online students that they’re trying to do too many things. They’ve got 1000 things going on with their families, and work, very likely. And they just simply don’t take the time. So I think… just simple reminders all the way along the way in an online course from the faculty member. In the assignment, to be really clear about the assignment and to say, “Be sure to think about the following.” For example, if an article has been assigned, that you as a faculty member say, “Okay, this is why I’ve selected this article. Here are some of the core concepts that are really important in this” and guide the reading to say, “Why do you think the person did this? Be sure to be able to answer these kinds of questions?” And then also to be clear about maybe what you want to do when you finish this reading assignment, to talk about it. If necessary, explain it to your 12-year old. See if you can explain it to that person. So being really clear in the assignment would be, I think, super helpful for all students in an online course, because again, it’s the kind of thing a faculty member would do in the classroom, you’d say, “Okay, I want you to read this article. And you know, this is one of my favorite articles. This is why I want you to do it” …and all the rest of it.” I think we need to do more of that in the assignments in the online course. That’s one thing.

The other students that are clueless are the ones that keep thinking that all they have to do is passively read and reread and reread. We hear that all the time from students that “Oh, well, I read it 15 times. I don’t know why I don’t understand it.” Without saying, while giving help and modeling, as you’re reading this, think about the following.

Just as an aside, I signed up to take a course on brain, dendrites, and synapses from the Hebrew University of Jerusalem, and oh, my gosh, lesson three, they said, “Well, you know, if you’re not familiar with the electrical circuits, you probably need to go into the Wikipedia and get all this information.” I’m thinking, “Oh, I’m lost right here.” But the idea was that there was some foundational content that I didn’t have. And so again, if in an online course, if some of their content is dependent on some of those other core concepts that they might not have, to remind students that, “hey, this is a difficult reading. And you may want to get some further help in X, Y, or Z.” So again, going back to that whole briefing, and using the assignment as a briefing and giving them clues how to do well in the reading. Some of those comments really have to do with whether students are struggling or not. Because we want to emphasize more clearly the reasons for why we make choices. When we design a course, we really make lots and lots of design decisions and selections as to why I want my students to read X, Y, or Z. And I think sharing that rationale and sharing those reasons, I think, really helps give insight into the metacognitive thinking of the faculty member as well.

Then John, you ask the question, where we talked about little bit about how even when students are doing well, they may not be thinking metacognitively. And I think that it’s important to recognize that these explicit thinking skills about thinking, about visualizing, and about planning, etc, that it would be a good idea to build those visualization and planning into the assignments.

Now, one of the challenges with online courses, and I’ve seen this for years, is the fact that some students don’t have good places to study, they can be living in very busy environments, they don’t have an office, they don’t have a really quiet place to go. And so one of the things that we can do is actually ask the students to post a picture of where they’re going to be doing their studying. It kind of gets them thinking, “Okay, where am I going to be doing my studying? Is it going to be a place that I can really concentrate?” There was one study hint… that I actually just pulled it out, because I gave it my granddaughters…. and it was called “How to study” …and the little hint in there was that when you are sitting down to study, what you do is you find, select or design, some kind of a flamboyant little hat… think Cat in the Hat kind of thing, you know… just flamboyant things, design some kind of a flamboyant hat so that when you put that hat on, you tell your brain and you tell the other people around you that you’re going internal now… you’re really going to think… you’re going to work, you’re going to study. We almost do that nowadays… when I go into my Starbucks, or my local coffee shop… that’s where I do my writing, by the way… I put on my Bose headphones, because again, that signals to the people around me that, “Hey, I’m working, I’m really concentrating.” So if we asked the students to really stop and think, “where am I going to study?” If I’m a family person, and I’m working, “When am I going to study?” That I have to schedule my week, and the times and places that I’m going to do my work and do my studying. And oh, by the way, I kind of have to get my family and my friends on board to say “Yes, John or Rebecca, they’re studying now I can tell they’re studying, you got to leave them alone, give them the time and the space to leave them alone.” For online students I think this is super important for them to build a schedule ahead of time. And again, it recognizes the fact that metacognitively takes time and it takes space in order to do it well.

John: That reminds me of a couple of our previous podcasts, a few episodes back Mathew Ouelett from Cornell, when we were recording, he mentioned that he has a drawing or a painting, I think he did of a tomato, I don’t remember if it was a painting or if he colored it, but he has a big poster with a tomato on his door that he puts up as a signal for faculty to leave him alone because he’s engaged in a pomodoro technique, and he wants to be just focused on this. In a much earlier podcasts, our very second one, Judie LittleJohn was here and we talked about a metacognitive cafe online discussion forum she and I had both used. And one of the things she uses in it, and Rebecca has introduced that in her classes too, is exactly what you suggested, having students describe their study space and perhaps post a picture of it. And then to address all those issues about how well it works, how they deal with distractions, and so on. This ties in nicely to some of our earlier discussions,

Judith: Well great, actually as I was in my Lucky Goat local coffee shop, actually getting ready for this podcast a couple weeks ago, it turns out this young business person came in and he had a briefcase and all the rest of it and I could tell he was kind of settling in… a little too close to me… but that’s okay, there weren’t many seats available. But anyhow, he was talking pretty loudly. And I thought, “Oh, he’s on his phone.” You know, there’s everybody who talks to themselves these days is on the phone, right? And anyhow, we ended up talking to each other partially because he was talking. He said, you know, what he was doing was he was talking to himself about what he was going to do during his time there. And obviously setting his own personal goal. And then he had learned that from his mother. [LAUGHTER] But that does remind me of something else I do want to share. And that is when I do sit down… my own metacognitive practice… when I sit down to do a task… say, I go to the coffee shop, and I got a couple of things I need to do, I write out a mini plan. And list the time I’m starting… list my first subtask, the second sub tasks with approximate time to completion, and everything else. And of course, it doesn’t happen exactly like that. But I do get done, and I do manage to check the pieces off. But then another really important metacognitive practice that I’d like to share is that when I finish that task, I say “What is my next important step?” In fact, David Allen and his book Getting Things Done on Stress Free productivity, thee whole environment, even in business. His primary question is, “What is your next step?” So before you stop where you are, particularly in a writing project, you write down what your next step is. Because it’s totally short circuits, that transition. Because when you sit down, “Yeah, what was I doing now? What do I have to do? What is my next step?” Hey, I can’t tell you the number of hours that that particular little hint has saved me in terms of really making progress and stuff. So that’s another little hint to build into all that project planning. “What is my next step? What do I have to have next?” And sometimes for online learners, particularly, it’s not sitting down and studying, they have to get a resource, or they have to get a book or they have to make an appointment or they have to do something. But then that’s something that doesn’t have to necessarily be done exactly in order. To kind of almost wanting to build a little calligraphy thing, saying “What is my next step?” is a really, really great little hint.

Rebecca: I think experience certainly teaches us that, but it’s something that students who have less experience don’t explicitly know to do. Because we all know that we’ve tried to cut corners and hurry through something. And then if we don’t do that… I know I’ve spent hours figuring out now what was I doing? Like it’s been a while since I’ve worked on this project. Now, where was that? What was my filing system?

Judith: Yes. And what was I thinking? And oh, dear, I wish I bought that along with this other thing with me. Yeah, exactly.

Rebecca: I also wanted to circle back a little bit to the workspaces too. One thing that I found with asking students to talk about their workspace is also to show them professional workspaces. And to talk about the different kinds of environments and how space can help facilitate certain kinds of activities, but also can short circuit certain kinds of activities as well.

Judith: I think that’s really, really helpful. I know, I can’t tell you the number of conversations I had with people when I was in a work environment, and management was making all kinds of decisions that were not conducive to good collaborative work or good independent work. I mean, it’s a real discipline to think about.

John: And you need both spaces, but you will often only have something that’s better for one or the other.

Judith: Exactly, exactly. For any kind of a course, as we’re talking about metacognition, we get overly focused on “what” the students are learning as opposed to the “how.” So just in a capsule comment, that as we are designing online our classroom based courses that we really include in our assignments.. in the design and the various activities… that we include both the “what” of the content and the “how…” How do I get there? And I think we started getting there when they start setting those personal goals. Because some of the courses now are starting to include a goal or a learning outcome that they want their students to think like a scientist. You’re not just learning biology, we want you to think like a scientist. Or we want you to think like an engineer, or an entrepreneur, or whatever. But getting this… it’s a different mindset. You just don’t want to learn what biology is and learn about the content of biology, but I love finding out about the biographies of like Einstein and a few other folks. It’s just fascinating to get into their heads as to the process that they use to do that. Eric Kandle, by the way, is another Nobel Prize winner. He’s got some fabulous book out about his thinking and his processes by which he actually investigated and learned about memory. So… fun.

Rebecca: I think that using biographies is a great way to introduce students to that a little bit.

Judith: Yes.

John: A number of software packages, such as Lumen Learning’s Waymaker package, Norton’s Inquiszitive, and some of McGraw-Hill’s and CENGAGE’s products include attempts at building student metacognition in their products. For example, they’ll ask students questions and they’ll also ask them about their confidence in their responses. The Norton Inquizitive package, in particular, sets it up in a somewhat game-like situation, where they get to bet points on how confident they are. And then it gives them feedback on how they did versus how they perceived they were doing. Do you think this type of approach might be useful?

Judith: I was just reading a research study about that, John, and the students… Who was that by? I would have to go back and find it, I’ll have to email it to you. But the results showed that the students who were less confident…

John: …did better.

Judith: Yeah. Who was that?

John: I don’t remember. I think it was on the POD list, but maybe not, it might have been on an economics list. Consistent with the Dunning-Kruger effect, the students who did relatively poorly, had relatively high self perceptions of how well they did that were not reflected in their test scores.

Judith: …So important to be asking the question: “What do I know?” and “What don’t I know?” Because that’s a core, isn’t it? If we can answer that question. In fact, one hint that’s in that book called Make It Stick… which is a real good one… one of the key things I took away from that was the technique or practice of having in a textbook, taking the heading and turning it into a question and then seeing if you can answer that when you finish reading something. So again, “What do I know when I start? What do I know when I finish? And am I able to answer that question? Or am I able to pose another good question based on that?” So explaining to ourselves what I know and what I don’t know, I wanted to go back to that study that we were just both talking about, because my question that occurred to me as I was reading this is that, you know, the problem, I think, is that the study really didn’t really give the students an opportunity to verbalize why they felt confident or less confident, which is, I think, a whole missing piece of that. And I don’t know how they obviously would make designing a study… actually, my dissertation, in fact, I will bore you with the title of it is “Fluent Readers’ Strategies for Assigning Meaning to Unknown Words in Context.” And the thing is that, half the time, they didn’t know that was an unknown word, they just assumed they knew the word. So again, if we don’t know we don’t know something, we can be very confident. [LAUGHTER] But anyway, in order to get the answers to that question of the study, I actually had the students talk out loud to me, they verbalized and I would go back and ask them about the word and say, “What were you thinking when you came to that?” …and all the rest of it. So they had to verbalize their thinking. For me, it was a good study, it really worked. So again, going back to the value of verbalizing,”What do I know?” and “What don’t I know?” and “What do I think I know?” And Rebecca, we got that when you said, “Well, you started talking about what to do, and then you realized you didn’t know anything.”

John: As you suggested before, a really good thing with any project is to think about where you’re going next. And we always end our podcast with the question, “What are you doing next?”

Judith: What a great question. Okay, what am I working on next? I did mention that I just finished a little video for the Distance Learning Conference in Madison in early August. On my back burner, and I’ve been saying this for a couple of years, I would really like to write a book on concepts. What I’d like to do is… I mentioned I love working with faculty, and when we design courses it so often on a topic-to-topic basis. And yet, as I started working with faculty towards the last 10 years or so, I started asking faculty to tell me what their core concepts of a course is. And, you know, faculty, that’ve been teaching for 10,15, 20, even 30 years, they would pause and they’d say, “Uhhhh, [LAUGHTER] you know, we just teach topics,…” as opposed to core concepts. And we think about core concepts are what stay with us when we finish a course, hopefully. So the question really is, what do you expect your students to know, five years, 10 years down the road from what you have been spending all this time and energy on? So I’d like to write a book about thinking about concepts and how to design a course, and focus on problems and concepts rather than topics.

Rebecca: I like that, I think that would be really helpful. Can’t wait to read it.

Judith: [LAUGHTER] I can’t either.

Rebecca: You gotta visualize, you gotta visualize. [LAUGHTER]

Judith: Thank you. You can give it right back to me.

John: We talk a lot about backwards design. But a lot of the classes that many of us teach were not designed in that way. They did not start with those major course learning objectives, and then work backwards to get to that point. And they’re just series of topics taught in the same way that they were taught to them when they were students. And they were taught in the same way as their previous generation taught them. And there’s not always a lot of thought going into that. And that sounds like a really good project.

Judith: Well, thank you, I may call you and see. I’ll need faculty to work with on that project. So I may contact you for that.

Rebecca: I’ll sign up.

John: Be happy to.

Judith: I’d be willing, God be willing that I get going on that. [LAUGHTER]

John: I think we all have a few projects like that. But eventually they often happen.

Rebecca: Eventually.

Judith: Yes, well, actually talking about it is a good thing. Because the more things we write down, and the more things we actually talk about are more likely to happen.

Rebecca: And I have just told the world so it’s gonna have to happen, right?

Judith: Oh, dear. [LAUGHTER]

John: Although if you change your mind, we can edit it out.

Judith: Scratch that…

John: I know I come up with some things after a podcast where I say I want to do this in my class next semester. And once it’s in a recording now, I pretty much have to do it. [LAUGHTER]

Judith: Well, John, before we break up here, when you do go to Duke, what do you teach?

John: I teach economics, introductory micro and macro economics.

Judith: Okay. Sounds great.

John: It’s been a lot of fun. I love doing it. The kids are just so amazing.

Judith: Well, kids are amazing at that age. They really are. It’s wonderful to see them evolving to young men and women. You know, I’ve got eight grandchildren. My oldest is now 19 and a half. In fact, she did microeconomics online, both one and two this summer.

John: Thank you for joining us.

Judith: Well, thank you very much. I really enjoyed being here.

Rebecca: This is a lot of fun. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

97. Emotions and Learning

As faculty, we often don’t take emotions into account when planning our courses or curricula. In this episode, Dr. Sarah Rose Cavanagh joins us to discuss the powerful role emotions play in student learning. Sarah is the author of The Spark of Learning: Energizing Education with the Science of Emotion and of Hivemind: the New Science of Tribalism in our Divided World and numerous scholarly publications. She is the Associate Director for Grants and Research at the D’Amour Center for Teaching Excellence at Assumption College, the Co-Director of the Laboratory for Cognitive and Affective Science, and also Research Affiliate at the Emotion, Brain, and Behavior Laboratory at Tufts University.

Show Notes

  • Sarah Rose Cavanagh – websitetwitter
  • Caulfield, M. (2017). Web literacy for student fact‐checkers. Pressbooks.
  • Cavanagh, S. R. (2016). The spark of learning: Energizing the college classroom with the science of emotion. West Virginia University Press.
  • Cavanagh, S. R. (2019). Hivemind: The New Science of Tribalism in our Divided World. Grand Central Publishing.
  • Lemov, D. (2010). Teach like a champion: 49 techniques that put students on the path to college (K-12). John Wiley & Sons.
  • Lemov, D. (2012). Teach like a champion field guide: A practical resource to make the 49 techniques your own. John Wiley & Sons.
  • Lemov, D. (2015). Teach like a champion 2.0: 62 techniques that put students on the path to college. John Wiley & Sons.
  • Parker, P. (2018). The art of gathering: How we meet and why it matters. Penguin.
  • Harrington, Christine. “61. A Motivational Syllabus,” Tea for Teaching podcast, December 25, 2018
  • Bain, K. (2011). What the best college teachers do. Harvard University Press.
  • Lang, J. M. (2006). The promising syllabus. Chronicle of Higher Education, 53(2), C2.
  • Knapp, Jennifer, “41. Instructional Communication,” Tea for Teaching podcast, August 8, 2018
  • Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational psychology review, 18(4), 315-341.
  • Pekrun, R., Frenzel, A. C., Goetz, T., & Perry, R. P. (2007). The control-value theory of achievement emotions: An integrative approach to emotions in education. In Emotion in education (pp. 13-36). Academic Press.
  • Smith, Kentina (2017). Stimulating Curiosity Using Hooks. Noba Blog. June 7

Transcript

John: Before we get to our regularly scheduled program we have a small request. Our 100th episode is around the corner and we’re collecting stories from our listeners about episodes, guests, or ideas that have influenced or impacted you, your colleagues, and your students. Please share your stories on teaforteaching.com.
We now return to the regularly scheduled podcast.

Rebecca: As faculty, we often don’t take emotions into account when planning our courses or curricula. In this episode, we discuss the powerful role emotions play in student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Today our guest is Dr. Sarah Rose Cavanagh. She’s the author of The Spark of Learning: Energizing Education with the Science of Emotion and of Hivemind: the New Science of Tribalism in our Divided World and numerous scholarly publications. Sarah is the Associate Director for Grants and Research at the D’Amour Center for Teaching Excellence at Assumption College, the Co-Director of the Laboratory for Cognitive and Affective Science, and also Research Affiliate at the Emotion, Brain, and Behavior Laboratory at Tufts University. Welcome, Sarah.

John: Welcome.

Sarah: Oh, thank you.

John: Our teas today are:… are you drinking tea?

Sarah: I am not. I am a coffee drinker. And I just had a very large coffee and I’m moving on to water now.

Rebecca: So many coffee drinkers on this show.

Sarah: Yup. It’s important. [LAUGHTER]

Rebecca: I’m drinking English breakfast, despite the fact that it’s no longer morning.

John: I’m drinking Tea Forte Black Currant tea.

Sarah: Mmmm. That sounds tasty

John: It’s very good.

Rebecca: So Sarah, we asked you to join us today to talk a little bit about The Spark of Learning. In that book, you argue that faculty should design all aspects of the course to target student emotions. Yet as teachers, we don’t really think about emotions, necessarily. So she can talk a little bit about why considering emotions is so important.

Sarah: Sure. Well, I think when you look at what’s required for learning in the classroom, you’ll see that there’s numerous cognitive resources that are required for learning. They have to pay attention to the material, you have to be willing to work on the material and your working memory, you have to be motivated to put effort and energy into that work both in the class, but then also outside of the class when you’re working on assignments. And all of these cognitive resources are limited, there’s only so much of them to go around. You can only pay attention to so much at once, you can only work on so many bits of information in your working memory. So we have to think about how can we motivate students to direct those cognitive resources towards the class material, toward the work of the class. And I believe that emotions are a critical ingredient in doing so because emotions attract attention. They were motivated to pay attention to work on emotional material, things that are self relevant. And we think that emotions evolved in the first place in order to motivate behavior: to push us toward things that are good for us, to pull us away from things that are dangerous or irrelevant, and also to tag information as important to remember. …and thinking a little bit about the emotional design of a presentation style, of the assignments that we choose, of the class activities, and even of how we assess students. All of these are strategies by which we can get students more motivated and more engaged.

John: One of the things you talk about in your book is the importance of first impressions. Could you tell us a little bit about why that’s so important to open the class with something that engages students’ emotions?

Sarah: Mm hmm. Great. Well, I think that students come to the class, they have busy lives… lots of things pulling them from work of the class… and when they first come into the class, we need to spark their curiosity, we need to get them engaged, and to focus them on the work of the class. I had a speaking engagement in Tennessee on the subject of learning and their planning committee was reading Priya Parker’s book, The Art of Gathering. So I picked it up in the airport, and I was reading it and she talks not about classrooms, but any gathering or meeting space. And one thing that she said that I love that I thought was very consistent with this idea of first impressions is you shouldn’t start with logistics. She says, “Don’t start a funeral with logistics.” Don’t stand up and say, “here’s the parking information.” And I think that we can use that lesson in the classroom. Like why start a class with “Oh, here’s the learning management system.” And “here’s what happens if you plagiarize” and all of these logistics that are kind of boring, and kind of ugly. [LAUGHTER] Why not start with the idea that we’re watching this intellectual journey together? Here’s what drew me to psychology or literature, chemistry, here’s what I think that you’re going to take from this class, here are the things you’re going to learn… to start with that passion. That’s going to form student feelings about the entire semester. And so I think that first impressions are important.

John: So perhaps going over the syllabus, interminably, on the first day may not be the best strategy. [LAUGHTER]

Sarah: Right.

Rebecca: To follow up on that a little bit, though, syllabi have all these policies and things… is there a way that we can tap into this emotional connection in a document like that, that can feel very policy oriented and rules oriented?

Sarah: Well, I think…a couple things. One, I wish I could remember the person’s name, but probably five years ago now, I saw some person’s blog posts on Twitter or something. She was a historian. And she had redone all of her syllabi, with images and famous quotes and made them really beautiful and kind of exciting to look at. And even though it was late in the summer, and I was already a little stressed about everything that was going on, I was inspired to redo all my syllabi similarly. And so I think just putting a little design into your syllabus can make it a more attractive document. I think my colleague, James Lang has a Chronicle post about starting syllabi with kind of what we were saying about Priya Parker and the Art of Gathering with a promise, “here are the exciting things that we’re going to be covering” instead of, “we’re going to read these books and cover these principles.” So in that section, when you say what the course is about, I think is powerful. And then in terms of policies, certain policies are a good idea to include on the syllabus. But I think the language that you choose matters quite a lot. And back in the day, I think I had a section on issues of courtesy. You know, “don’t pack up your bags while I’m still talking,” “don’t use your cell phones,” all these things. Now that section on my syllabi talks about “Let’s respect each other, and here’s my commitment to you: that I will start and end the class promptly on time that I will return your assignments to you within a reasonable time frame, but I will respect all of your contributions. And in return, I would ask that you not pack up your bags, while I’m still speaking and these kind of things.” And so I think framing some of the policies in terms of both what’s exciting that’s going to happen, but then also in the sort of communal language rather than punitive language, I think can go a long way to make this a little bit more inviting.

John: I’ll throw in a reference to a past podcast… we had Christine Harrington, who talked about her book: Designing a Motivational Syllabus. And also, Ken Bain had written about the “promising syllabus” way back. And I think that’s inspired a lot of these discussions. And I think they’re all very good suggestions. We should all do more of that, I suspect.

Rebecca: So we talked a little bit about the setup in the beginning of the class. Some of it is also just deciding what assessments there’s going to be and what the assignments are going to be. So can you talk a little bit about how we can plan for emotion in those kinds of design aspects as well?

Sarah: Sure. And here, I’m going to cite Reinhard Pekrun, the researcher and psychologist, and he has an entire theory of academic emotion. So he was having a lot of respect before I ever did. And his theory of academic emotions, he calls the control value theory of academic emotions. And by control, he means autonomy. So giving students choices, giving them flexibility, and the sense that they’re crafting their own intellectual journey, not just that they’re submitting themselves to yours. [LAUGHTER] And then value really being about some of these things that we’re talking about in terms of emotional engagement, but also the whole idea of relevance . So the students see the relevance of the assignments and the assessment. And relevance is multifaceted, it can be relevant for their personal lives, or their future careers… It could be some transcendent purpose, here’s why we should be evaluating this topic in order to improve society at large… that the students should see the value. And so kind of the opposite of busy work. We’re not just doing this for no reason, there’s a purpose, there’s a relevance. And so I think, using his framework, and thinking about ways that we can help students shape their own intellectual journey, and which assignments they’re going to do with the topics, you’re giving them choices of topics… on exams, giving them choices of essays, things like that. And then value, always illustrating the relevance and the importance of the work that they’re doing, I think are ways that we can think about assignments and assessments.

John: You also talk a little bit about using emotional contagion in classes to help build motivation. Could you talk a little bit about that?

Sarah: Sure. That whole topic really engaged me in reading and researching and writing… kind of turned into my second book project. But I think that we are incredibly social beings, we’re individuals, but also have this collective aspects to our psychology and how our brains work. I think that in the classroom, we’re in a social setting. And there’s certainly lots of research evidence showing that emotions, in particular, are contagious, that they kind of spread from one to another. I think one of the ways that that topic is relevant in the classroom is from instructor to students. And so putting a little bit of thought into your presence and the kinds of emotions that you’re showing: are you showing passion? Are you showing enthusiasm? Are you engaged yourself? Are you interested and present yourself? …that level of curiosity and passion can spread through the class. There’s student to student emotional contagion. And I’m sure anyone who’s taught a while has had these experiences both positive and negative ways… the ways in which enthusiasm and motivation can kind of spread among the class and the ways that negative emotions can spread throughout a class. And there’s a big literature on the topic of reactance, which is a term that refers to when the students sort of collectively decide that you, the instructor, are unfair, or uninteresting, or something else, [LAUGHTER] and kind of bands together and bond over that. And so thinking strategically about how to minimize those occurrences, are also ways to think about emotional contagion in the class.

John: So, on those days when you’re not feeling as energetic and enthusiastic, what can we do to help create that emotional contagion effect?

Sarah: Yeah, coffee. [LAUGHTER]

John: …or tea.

Sarah: Yes, or tea… coffee or tea. But, that’s a fascinating question, and one that’s a little under studied. And so I looked at the research literature, and there are a couple of research studies on the whole phenomenon of faking it, and doing emotional labor. So putting on a happy face, and an enthusiastic face, even when you’re not there. And it’s mixed. There’s a power in authenticity. But sometimes we also have to engender some enthusiasm that we might not necessarily be feeling. I think that prior preparation can also go a long way. Some of these ways of being more emotionally engaging, I think, can be in your choice of activities in the class and videos that you’re showing. And so thinking ahead of time, if it’s kind of a dead time of semester for you, thinking of things you can do in the classroom to mix it up, because you know that your energy might not bring that energy.

John: And you also suggest that mindfulness training might be useful in helping faculty become more focused or more present in the classroom.

Sarah: Yeah, mindfulness is super interesting. I think it’s one of those topics that are so multifaceted that they’re hard to break down and study from a psychology perspective, because mindfulness itself has attentional components it has components of acceptance. But research shows that mindfulness is really good at bringing people to the present moment. And I think that some of these present and performance related topics… a lot of it is “are you there with the students,” instead of off in your own mind, creating your shopping list or thinking about your manuscript that’s overdue. [LAUGHTER] And so I think bringing yourself back to that present moment, and reconnecting with the students… making eye contact, thinking carefully about what you’re going to say. That is the essence of mindfulness training, bringing yourself back to the present moment, and so it may benefit your work in the classroom.

Rebecca: Can we talk a little bit about those negative emotions.

Rebecca: You know, sometimes that happens… you’re having a bad semester or something goes wrong. And then perhaps that contagion effect really does happen in your class, and you need to bring it back.

Sarah: Yeah.

Rebecca: Do you have some strategies on how to bring it back.

Sarah: I think that those emotions tend to build within the class itself, when students aren’t feeling heard, when they are not feeling that autonomy, and they’re not feeling that control. And I think a lot of those emotions are just around perceptions of unfairness and status and authority. So some of the ways do work on that, I think, are being transparent and having open conversations with the students doing mid semester check ins… you know, giving them a voice, and a way for them to.. Instead of telling each other what they don’t like about your class… to tell you. And then that, in demonstrating that you care… that you want to know what their feedback is, especially if you’re able to make slight changes, because they might have a point… and none of us are perfect. But having that open conversation and valuing their voice, I think, is a way to try to alleviate some of that reactance. The literature on reactance shows that the best defense is a good offense… preventing it in the first place. Some of the ways that the research suggests to prevent it is, again, that presence and immediately… this whole concept of immediacy cues, things like eye contact, using inclusive language, varied vocal tone, things like that that shows students that you’re there with them, have been some of the best variables that predict lower reactants and lower negative emotions over the semester.

Rebecca: There’s some really great tips on immediacy in the episode we had with Jennifer Knapp.

Sarah: Oh, good. I’ll check that out.

John: You also talk a little bit about self disclosure as a way to building more immediacy. Could you talk to us a little bit about how self disclosure might be done productively? And when does it go too far?

Sarah: Yeah, I think self disclosure does two things that explain why it’s effective. One, it’s a way of being present. And secondly, it’s also a way of using storytelling in the classroom. And we know that stories are kind of cognitively privileged… that they work… they’re effective in the classroom. I read a couple of qualitative studies in which they had a sort of student think tank somewhere asking them about instructor self disclosure, and the times that they felt that it was very effective, and the times that they felt that was less effective. What students reported was that it was most effective when instructors shared stories about their own intellectual journeys, especially times that they had trouble with this material and how they worked their way around it. I always tell my students that I failed to get into a single graduate school the first time around and they love to hear that. Because it shows that when you look toward your goal, it’s not always smooth sailing, we all hit bumps in the road and have to re-strategize. Some degree of personal one-on-one disclosure is also effective… talking about the game you were at with your kids over the weekend, or your favorite movie, and things like that… just because it makes you a person instead of just an authority figure at the front of the room.

Rebecca: I thought we were all robots at the front of the room. [LAUGHTER] I didn’t understand that we weren’t that.

Sarah: Yeah, it always surprises me when my students perk up whenever I share something personal. And I’m like, I’m this old fogie, like… It surprises me that they’re interested. But they are, I think, for those reasons. I think reasonable boundaries, they don’t need to know about… [LAUGHTER] what they don’t need to know that. They don’t need to know everything.

Rebecca: We’ve talked a little bit about design and thinking about getting students motivated together, and us helping them get motivated and them motivating each other. But you also talked a little bit about the strength of emotion in being able to just process and remember things. Can you talk a little bit about that, and maybe some strategies that we can incorporate into our classes related to that?

Sarah: Sure. and I think primarily, the first thing that I always think of with emotions, in that sense, is grabbing attention. And we have lots of literature showing that, on a very basic neurological level, emotional stimuli arrests attention. And I ran into a blog post after writing the book that I wish I had run into before writing the book by Kentina Smith, and she talks about using emotional hooks in the classroom. And I love that term. And what she means by that is kind of sectioning up your class into whatever makes sense for your length of your class and for your material. And then beginning each segment of your class, of your material, with an emotional hook… that they hook them in. And that can be using videos… stories, again, are really great… reading passages that are emotionally interesting. Again, demonstrating relevance for career or for something else. I was running a workshop at Northern Illinois University and one of the professors shared what she did… she was in a nursing program… and in one of her freshman classes that were really a lot of work… and students often got discouraged… she would have the students who had just graduated and now were in their internships come back and talk about how the material that they learned in that class… how they were using it in the field at this moment… and how they were so grateful to have those skills. And I thought that was amazing. That was a really powerful way of hooking students attention and saying, “Okay, this material might be a little boring, but it’s really important.” That isn’t too flashy. I think sometimes people worry that what I’m talking about means that we’re just purely entertaining the students. And I don’t think that’s the case… and so using those emotional hook.

Memory is interesting, it’s a little trickier. Because there is some evidence, I shouldn’t admit this. But when you do something really emotional, that students remember the emotion, and then not what comes next. Because they’re so caught up in the emotion. But I don’t think much of what we’re doing in the classroom is making students super emotional, but just like giving them a little bit of a prime, we’re more likely to remember things that are novel, that are interesting that would get us a little outraged, that get us a little passionate. And so I think that at a very basic level, emotions benefit these cognitive resources.

John: One of the emotions you talk a little bit about is frustration, and that it can be useful sometimes to confuse students a bit. Could you talk a little bit about?

Sarah: Sure. When I talk to people about ideas in the book, they sometimes think that I’m advocating that students should be happy all the time, that it should be nothing but positive. And I don’t think that. I think that some frustration is a natural part of the process of learning. There’s experience-sampling studies where students are learning new skills from computerized tutorials, and also reporting on their emotions, like on a dial at the same time. And it shows that, as the students learn new skills, it’s a repeated dynamic cycle between initial confusion because they don’t know this yet… they start strategizing and start trying things… working on it… and then they’re frustrated. Then they solve that level or skill or problem and they achieve learning, and then they have this flash of pleasure. And then the tutorial system brings them to the next level and they’re confused again. And that learning seems to be that repeated dynamic cycle. I think that that’s very true. I think helping them navigate that through self disclosure… through transparency… saying, “Hey, you’re going to get frustrated and that means you’re learning. That means that this is something you haven’t encountered before.” I think this can help navigate them through because you don’t want them to get so frustrated that they get anxious and worried. So normalizing and acknowledging that that’s part of the process… But I think it is, I think it is part of the process of learning.

John: We often have students from very diverse backgrounds, though, in terms of their prior knowledge. How can we design activities that will provide an optimal amount of challenge for students, when students come in with so different backgrounds?

Sarah: That’s really tricky. [LAUGHTER] I think that’s one of the trickiest things about our job. And I think routinely assessing where your students are at, can be a strategy. I mean, it’s still going to tell you a lot about the average, which is not going to tell you as much about the diversity of experiences, but having kind of your finger on the pulse of where your students are, either through quick quizzes, online check-ins, but even through the questions that you ask. I read Doug Lemov’s book, I’m forgetting the title, [Teach Like a Champion] but he worked in high school and studied star teachers who were having with amazing outcomes, even in high schools that had low resources. And one of the recommendations that comes out of his analysis of those teachers was asking questions in ways that really reveal the student level of knowledge. Instead of saying “Everybody’s got that?” or “Does everyone understand? …asking those questions so that you can have a gauge of where all of your students are. Smaller classes… you can do more personalized, focused things. One of the works that I read had talked about getting progress feedback, as well as discrepancy feedback. So having papers be due in segments, and not only showing students where they needed to improve, but also telling them where they have improved. I think that sort of personalized attention we can’t all do when we’re teaching classes of 500. But, if you’re teaching a smaller class, some of that personalized stuff can help.

John: Can peer instruction, perhaps, help leverage some of that when you ask questions that are challenging for some and easier for others.

Sarah: Yes, I love that.

Rebecca: Sometimes students may get too frustrated and give up. How do you get them back to a place where it can be productive again?

Sarah: Again, being transparent, kind of my go to, and talking about the fact that that’s likely to happen at different points in the semester for different students and helping them do that. I think, knowing your college’s resources in terms of student mental health, in terms of academic support, and being able to refer students out to those, I think is important. And I think even just small things like sending an email. And again, they realize that I have this bias because I teach small classes, relatively. But you know that a student is struggling and you can observe that they’re hitting kind of a rough point… sending them a personalized email and saying, “Hey, do you want to drop by office hours, this is when they are…” and feeling seen by the professor and knowing that there are resources, I think, can be very helpful.

Rebecca: One of the things you mentioned up front was the idea that we want them to get curious and engaged and own their learning. Can you talk a little bit about ways other than just the choose your own adventure kind of opportunities where they have choice, that we can leverage students curiosity and get them really hooked?

Sarah: Yeah, I think asking questions, kind of the idea of puzzles and mysteries. Every field has their unsolved mysteries. And I find that students really respond when I present debates that are ongoing in the field. And I think that works on two levels. There’s not a set answer and so they’re curious, because we’re always most curious about things that we’re not quite sure about. And they also feel the freedom to contribute, because they know they’re not going to get it wrong, because no one knows. But also putting them in this position where they feel like they too could join this quest. And they might be able to push knowledge, if they were to go on to graduate school. So, putting them in the shoes of a contemporary psychologist or biologist and here are the things that people are yelling at each other about on Twitter, because no one can agree. “What is your opinion?” is a way to get students curious.

John: We’re recording that in mid-August, but will be releasing it shortly after your new book, Hivemind comes out? Could you tell us a little bit about Hivemind?

Sarah: Sure, it’s a complicated book. I see it as having three layers. On it’s base layer, it’s really a contemporary overview of social neuroscience, the current state of knowledge in terms of how we are, as I was saying before, not just an individualistic species, but we also have this collective aspect. That as Jon Haidt says, we can be hive-ish. And that’s why the title Hivemind. And so at its base level, it’s kind of like a bird’s eye overview of what’s going on in social neuroscience: How do our brains relate to each other? How do we engage in this sometimes almost collective consciousness and things like that. And then the second layer is how smartphones and social media, the invention of those devices and technologies, are amplifying our social natures, both in good ways and in bad ways… on evaluating that evidence. And then the third layer is sort of our current political polarization moment, and what we can learn from social neuroscience and social media as to what’s going on in the world.

John: How have the changes in technology led to the changes in polarization that we’ve been observing?

Sarah: Yeah, it’s a fascinating question and one that would be a great question for a class because I don’t think we know for sure. But anyone who has a smartphone or is on social media, I think, has seen evidence of this polarization and felt like it has become more extreme. And certainly, there’s some polling about in the States, Republicans and Democrats and how comfortable you would or wouldn’t be if your child married someone of the opposing political party. And those sorts of studies are definitely showing greater polarization. And there’s a lot of principles in terms of when you get together with a group, and you begin discussing your opinions and you’re sharing your opinions, that your opinions become more extreme, because you’re hearing it echoes back… the whole phenomenon of good polarization and echo chambers. So there’s evidence that that’s making all of that worse. I think that there is also evidence, though, that we may be paying too much attention to the polarization and that talking so much about the polarization, in some ways gives us permission to be polarized. And I think that there’s evidence from social psychology that we form much more extreme “us versus thems” when we feel under perceived threat. And certainly we are under numerous threats. But I think that also we are kind of buying into a collective panic and fear. Ironically, in part, one of those panics, I think is about smartphones and social media. And I think we’re overly panicked. It’s really, really complicated. And I think it’s really, really fascinating. And I think we’re not sure quite yet.

Rebecca: I know that a lot of faculty have talked about how the polarization, the spread of misinformation through social media, is impacting conversations and things that are happening in their classes.

Sarah: um hmm.

Rebecca: Do you have any suggestions for how to navigate that, using some of this emotional research that you’ve been focused on?

Sarah: Sure, I think that, I’m going to go back to my transparency again. But having ground rules, especially if your class is focused on a topic that is likely to generate some of this heat, starting the semester with ground rules about respect, about open dialogue, and then also with the tapping back to control and then autonomy, giving students some power over that. So, on Twitter, people are sharing stories about how to charge the class to sit down and develop, with an agreement about how we’re going to debate things together. And students would make suggestions and some of this is done on Wikis… that’s really interesting work. So I think acknowledging that, and I think this is going to vary a lot on different campuses. And I’ve seen that… I do some traveling around doing workshops and talks, and I see that variability. Different campuses very politically in terms of whether they’re left leaning or right leaning… The students vary in their degree to which they’re politically active or interested. On my campus, I find that students are reluctant to debate some of these issues, and that we have to bring them to the table. Whereas I talked to people in some other campuses where they have to cool down the whole class because everybody’s jumping in. So I think the strategies will vary a lot based on your student body and the topic that you’re teaching. But I think ground rules about respect, especially collectively sourced, can be very powerful… and getting students some say,

Rebecca: It sounds like maybe this book is coming out just in time, so we can all prepare for 2020. [LAUGHTER]

Sarah: Yeah. [LAUGHTER]

I was at Wellesley College, I think, a year ago. They were asking me about the topic of the book, Hivemind, and they were saying the same thing. They were like, “Oh, this is so timely.” And one of the women, as she looked at me with such dismay, and she was like, “I really hope it’s not still timely by the time the book comes out.” …that we resolve some of these issues. But now it’s coming out in a few weeks and I don’t think we’ve solved much.

John: Is some of it, though, a shift from national media, where the major newspapers and TV stations and so forth had to appeal to a broader audience, so they aimed at the middle of the spectrum? And now we’ve diversified, as has happened in many other areas with music and arts as well, so that now any particular point of view can develop its own hive, and extreme views can spread perhaps more easily,

Rebecca: …like the long tail idea?

Sarah: Yes, I definitely think there’s a lot to that. And I think that some of those things are, when we’re not looking politically, necessarily are really positive. It allows social media and has allowed people of like mind to find other people of like mind in terms of like hobbies or interests, or people who share their life experiences. I interviewed some people in the book who have had those experiences: there’s no one that understood them, or if they were disconnected from their heritage, and they were able through social media to connect. But I think that it is more dangerous when it’s news sources and politics.

John: One of the issues I’ve seen in my classes in the last several years is that people used to disagree about policy outcomes, but they generally didn’t disagree about basic facts and evidence. And now I’m seeing a lot of that in classes in ways I’ve never seen until the last few years. How can we deal with that type of an issue?

Sarah: Yeah, there’s some great people working on this issue. Mike Caulfield has a whole fact checking literacy. It’s a free online PDF, a book, and he has what he calls “Four Moves to Fact Checking.” And what I really love about this is it ties into the emotional piece and understanding how humans work. Because other approaches to fact checking in media literacy are really laborious. There are 12 steps… and I think unrealistic for how we engage with information. And he has, I don’t know each one of his moves. But there four moves for checking facts in which students can quickly advocate for certain information and look for the background… look for actual scholarly sources on it and get to a better place of is this actually information that’s true. And I do it with my own students, my intro Psych students, we do a little fact check on a couple different memes [LAUGHTER] to get them used to that sort of thing. Because if we can’t agree on facts, then we’re going to be in a lot of trouble.

Rebecca: It sounds to me like talking about emotions in general, no matter what your class ia, could be a benefit in helping students understand and sort through the difference between an emotional response to something versus a cognitive response to something,

Sarah: Right, I think so too. My research background is in emotion regulation. And in the book, I advocate for using cognitive reappraisal, which is an emotion regulatory technique in which you reinterpret the situation or the emotion that you’re having. And there’s some really fascinating work being done using cognitive reappraisal to people on two sides of intractable conflicts, and it is effective… and I think, using emotion regulation and regulating our own information, especially as it intersects with facts, especially facts that are political. I absolutely agree it’s going to be a critical strategy.

Rebecca: Do you have like a Cliff notes version of that, that you could share with folks who are maybe not in your field, that we could share that information with students?

Sarah: Yes. Sure. I think that’s one of the basic examples that I give for cognitive reappraisal is, you know, if you’re fired, you got a pink slip at work. And you could interpret that on the one hand as “You are a failure, you’re never going to have another job, that this is a devastating loss.” And that’s going to lead you down a directory of a certain emotional response. Or you could rephrase it as “You know, the company is downsizing and it’s nothing personal, that you would always want to just shift careers to these and this is an opportunity to do that.” And that set of appraisals or interpretations is going to set you on a very different emotional path. I think that reappraising some of these “us versus them…” You talk to people on either side of the political spectrum and about the opposing political side… and there’s also a lot of dehumanizing speech: they’re monsters, they’re evil. I think when we engage in those appraisals, it’s just going to drive us further and further apart. And so reappraising those, yes, disagree with this person on this policy, but trying to see their perspective… going to have that conversation, framing them as a human being who has different opinions than you, rather than a monster or a creature, I think, are powerful ways of trying to step back from some of the heat of this polarization.

John: We always end with the question, what are you doing next?

Sarah: I want to answer it on two levels. One on the like Spark education level. With my colleague, James Lang, we’re focusing our attention and have some grants out the door on grading. And so you think about emotions in the classroom, emotional moments in the classroom, I think being graded and handing back a grade… students’ reactions to grades as one of the most emotional moments. There’s a lot of literature showing that students find receiving grades demotivating. Sometimes if they get a lower grades than they expected, they won’t read any of that careful feedback. And it can be unreliable, from professor to professor, from student to student, there are biases… gender biases… racial biases… in grading. And so I think we kind of need to fix grading, and that’s what we’re turning our attention towards next. On the writing side, I’m working on a book proposal that’s going to remain mostly secret, but it’s gonna be something fun. [LAUGHTER] I don’t want to think about politics anymore. I sometimes joke that writing Hivemind, it’s like I sat down and developed, like, “How many hate lists can I get on? “ And that’s like the farewell to the chapter outlines.

Rebecca: So now you need balance, you need to get on the good list, right?

Sarah: So I might do something like a little fun. It will still be psychology and neuroscience, personal anecdotes, and interviews and things like that, but one that has nothing to do with politics.

Rebecca: Sounds like a nice place to be.

Sarah: Yeah. [LAUGHTER]

Rebecca: Well, thank you so much for joining us. It’s been really interesting and I think faculty as they’re getting started in the new semester will take advantage of some of this information as they move forward.

Sarah: Awesome. Thank you. This has been such a pleasure,

John: Thank you. And I’m looking forward to the arrival of Hivemind which should be in early September, I believe.

Sarah: Yep. September 3,

John: it will be out by the time this podcast is released.

Rebecca: Yeah, September 4.

Sarah: Oh, that’s so cool. my publicist will be so pleased.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

96. Inclusive Pedagogy

Many of us strive to be inclusive in our classrooms but may not have the training to be as effective as we want to be. In this episode, Dr. Amer F. Ahmed joins us to explore inclusive pedagogy and to encourage us to consider our roles as both instructors and learners in intercultural contexts.

Amer is the founder and CEO of AFA Diversity Consulting LLC. He previously served as Director of Intercultural Teaching and Faculty Development at the University of Massachusetts at Amherst, as faculty at the Summer and Winter Institutes for Intercultural Communication, and as a member of Speak Out: the Institute for Democratic Education.

Show Notes

Transcript

Rebecca: Many of us strive to be inclusive in our classrooms but may not have the training to be as effective as we want to be. In this episode, we explore inclusive pedagogy by considering our roles as both instructors and learners in intercultural contexts.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Amer F. Ahmed. He is the founder and CEO of AFA Diversity Consulting LLC. He previously served as Director of Intercultural Teaching and Faculty Development at the University of Massachusetts at Amherst, as faculty at the Summer and Winter Institutes for Intercultural Communication, and as a member of Speak Out: the Institute for Democratic Education. Welcome.

Amer: Thank you.

Rebecca: Today our teas are:

John: Are you drinking any tea?

Amer: Not at the moment, but I like jasmine tea and green tea.

Rebecca: Yum!

John: I’m drinking pineapple ginger green tea.

Rebecca: Oh, that sounds yummy.

John: It is.

Rebecca: I am drinking my good old English afternoon tea.

John: …for a change.

Rebecca: Yeah. [LAUGHTER]

John: We’ve invited you here to talk about your work on creating inclusive learning environments. Could you tell us a little bit about what you’ve been doing and what you recommend?

Amer: Yeah, well, in recent months, I’ve been spending a lot of time with various campuses, working with faculty, working with teaching excellence of faculty development units, and diversity officers, on building capacity around inclusive teaching and inclusive pedagogy at various institutions around the country. It’s a big area of emphasis and focus these days for a number of institutions. It’s a tremendous challenge that many institutions are facing in terms of the classroom environment for students in higher education. My work has been on diversity, equity, and inclusion in a number of different arenas within higher education. But more recently, beyond just the broader strategic and institutional strategies and efforts that I work on, there’s been a lot of focus on the classroom and working with faculty on building capacity around that.

Rebecca: Can you talk a little bit about what you mean by inclusive pedagogy. I think that that’s a term that’s being used a lot, but not defined often.

Amer: Yeah, I think that one thing I learned by working in a faculty development unit was that many faculty have not spent a lot of time in their training and development around teaching in general. Quite honestly, as scholars, we’re trained to be researchers. And then as a result, as a default, we often teach the way that we were taught. And the reality is that there’s historical systems of inequity that are built around who’s privileged in terms of what cultural norm feeds our privilege in the way in which teaching and learning has been traditionally occurring. And Paulo Freire talked about banking and depositing… just the faculty member and the teacher as an expert, just dumping information into students as passive recipients and regurgitators of that information and knowledge. And I think that teaching, really… many people say it’s an art and the idea of pedagogy as a process, right? …that we engage with our students. An inclusive pedagogy, I think, really emphasizes who we are as teachers and learners, and that we all are teachers and learners, but that who we are and our identities and our backgrounds and experiences are all resources for learning. And then the question becomes, what is the process for us to harness the benefits of all those unique backgrounds and experiences and identities that we each bring as related to the content of the course, or of what we’re focusing on in the learning environment? And so I just think that a lot of times, we’re really focused on the content, and of course we should be focused on the content, but less focused on who is in the room, engaging the process of learning.

John: How can we tap into students’ identities? How can we find out information that’s relevant for the course?

Amer: Yeah, well, I think where I try to start is recognizing that we can’t know everything about everybody, right? And again, that’s where we have to think of ourselves as educators as learners as well. We don’t know it all (about anything, certainly), let alone the idea of who our students are. And as a result, can we develop some core competencies and skills around understanding who we are in relationship to who we encounter and have some intercultural skills that position us to be able to learn who our students are, and to draw from who the students are. So then it gets even back to the course design of: have we designed our course to leverage who are students are… to bring that forward. And then to be aware of our biases, when we’re aware of we are in relationship to others, we might realize that, oh, maybe I have some pre-existent stereotypes or perception of what it means to be X, Y, and Z. And instead, can I build a process where students are really articulating who they are, how they understand what we’re engaging in the content of the course in relationship to their backgrounds and experiences. And so I think that, for faculty, I think a lot of the fear is, “I’m going to mess up, I’m going to say the wrong thing.” So can we create a learning environment where it’s okay to make mistakes, but we’re going to do the best we can to understand as much as we can about one another and position ourselves to be able to draw from that to learn?

Rebecca: You said something about designing your course to leverage identity and leverage who’s in the room and who the learners are. Can you give an example from a specific course of what that kind of courses I might look like that does take advantage of that?

Amer: So I taught a global implications of hip hop, race, and spirituality course last fall at UMass Amherst. And one of the projects that I had the students work on was, after we learned some kind of key principles and issues as related to hip hop, and learned some examples of hip hop in different places in the world. I asked them to bring in an example and share an example in the course of hip hop somewhere in the world, that met some of these principles and concepts and ideas that we were talking about. And for me, it was just so fascinating to learn about all these examples. I mean, I’m familiar with a lot of examples of hip hop in different places in the world. And there was plenty that I was not familiar with… examples from Russia, examples from Iran. And it was really interesting to see how students were drawing from their backgrounds and experiences as oftentimes, not always, as a rationale for why they picked that example. So for one student, his roommate was Iranian and he learned a lot about Iran from his roommate. And that’s how he learned about hip hop in Iran and so he wanted to share that with the class. We have other examples of the Dominican-American students wanting to share examples from the Dominican Republic. So not every example was drawn directly from their own personal identity, some of it was just from their experience, but they felt connected to it in a different way, because they had the room and permission to connect who they were. And then we did other things in the course, to really try to harness that. But they understood that their background, experiences, their trajectories, were valued. And then part of how that was also articulated in the course was in their reading responses. I made it very clear to the students that I don’t want just a summary of what the reading was, I’ve read it, you know, I know what’s in it. What I’m curious about is, how do you understand yourself in relationship to what you’re reading? How does it connect to your background and experience? And I think that creates way different responses from students, and for me to affirm when they’re connecting the content to their experience, when I’m validating that that’s what I want… that’s what I like to see. Because whether we like it or not, they’re going to elevate us as faculty members. So they need to know that it’s okay, that that’s what we want. And the incentive is in that. I think for us as faculty, the course becomes less rote. How many times have you heard a faculty member saying, “I taught the same course, again, last semester, I’m teaching it again, this next semester.” You know, no two courses should ever be the same, because you never have the same people in your class. So the question is, what have you done in the class to be able to harness who’s in the room… to make it a new experience every time for you, as well as, of course, a new experience with the students.

Rebecca: It sounds to me like you do a lot to set up a very safe space for learning and discussion. Are there some things that you do at the beginning of the course or in your syllabus to actually set that stage to have those conversations and make students feel comfortable about sharing those experiences?

Amer: Yeah, and “safe space” has become a little bit of a loaded phrase these days. Can you truly make a learning environment truly safe given some of the trauma and backgrounds and experiences that people are bringing into the classroom? And so obviously, many people have been talking about brave spaces these days. Can we find ways to be courageous? But part of how we do that is to try to create mechanisms of safety, to whatever degree we can, for students to want to be courageous and brave and sharing who they are in the classroom. And so for me as a person who started my career in student affairs, just norms… working through creating a set of norms and agreements with your students at the beginning of a course. And this is something that’s widely done in co-curricular learning spaces, as you bring folks together for dialogue. But what I’ve learned is that a lot of faculty don’t do that. And many faculty feel like that’s a waste of time, I’m trying to get to the content. And it’s just one more thing to do. But I think it’s important for students to feel like they’re able to articulate what it is that they feel like they need to be able to be their full, whole authentic selves… participating and engaging the classroom. And sometimes that means students being able to articulate their comfort level with verbal communication, whatever it is, confidentiality, different kinds of expectations that they put out. And as a faculty member, you’re not telling them necessarily, they might say exactly what you were thinking, but the sense of ownership of what’s happening in the classroom… and that I had some kind of say over how we’re going to engage, so that I can feel comfortable bringing myself forward. And so what I do is I create a Google doc. So whatever they come up with, I put that into a Google doc and I make that available to everybody throughout the course, if anybody has concerns about the list that was created by them, they can always let me know and revisit it if they feel like there’s something that’s not working or that I’m not ensuring that those agreements are being held to. But again, it means that I’m not telling them how I expect them to engage. They’re articulating that… again, different ownership over what’s happening in the classroom. And so that means that we’re decentering ourselves in the process, and more of a facilitator role of the learning that’s happening, I think, for a lot of faculty, that seems ludicrous. Like, I’m the expert, I’m the one that went and did all this work to be able to share. But I think the question is, what is the learning that we wanted to see occur? Is it about us downloading this information, and students may or may not grasp all of it, or feel connected to it and be disinterested and disengage in it? Or is there a way for them to connect to it, where they actively engage the learning where they’re more centered, and the idea of student-centered learning where who they is centered more. The faculty member may be decentered more, but that opens up the space to be able to bring more of who everybody is into the learning process.

John: It sounds like one of the important components then is devising learning activities that bring this out, that gives students the opportunity to express themselves and their identity through the activities or through the assignments. Is that correct?

Amer: Yeah. And that’s the reason why faculty need each other as resources. And they need faculty development and teaching excellence offices and units as resources, because every faculty member cannot be expected to come up with all these different kinds of activities. Faculty need support, they need support to be able to do this. But there also needs to be incentive, there needs to be some kind of value in the institution for it to be worth their time. Because it’s like, why am I going to take all this time, energy and effort to be a better teacher, if my entire path to tenure and full professor doesn’t value that in any way, shape, or form, right? So that’s where my system lens comes in around that. So it’s a combination of faculty wanting to teach, and for our academic affairs areas to provide the resources and support a faculty to actually want to develop these skills,

Rebecca: You mentioned the role of teaching center. Can you talk a little bit more about the role that you see teaching centers in helping faculty move forward? What kinds of services or tutorials or what have you?

Amer: Yeah, this is a really evolving space in higher education from my purview. I mean, I’m fortunate that I get to see a lot of different institutional environments, situations in working across so many contexts. Again, we have so many different kinds of institutions, some institutions have really robust resources, and some have one person. And some of those one-person offices are understandable on a really small, private liberal arts institution, but maybe without a lot of resources. But I think what I see universally is that the resources that are made available to faculty are usually voluntary. And then the tendency is that we see junior faculty more likely to tap those resources and I think that it may create goodwill amongst faculty, but it doesn’t necessarily translate into reaching a broad swath of faculty. And so that’s really, I think the big question is, are we going to have resources that actually reach a number of faculty, and are there going to be some incentives and or expectation of faculty utilizing those resources. Increasingly I’m learning more about trying to make more resources available online, and not just links to articles, not just some basic resources, but literally full blown professional development… learning opportunities around effective teaching. But the next piece is the inclusion piece. So there’s a varying degree to which inclusion is focused on in these Teaching Excellence offices. And so what I found as a diversity, equity, and inclusion professional coming into that area, and finding myself to be one of the few people of color around in the field, I mean, obviously, around the country, you’ll find a decent amount. But generally, there’s not a lot, I didn’t come across a lot… So, I haven’t so far. There are some out there. Not to say that you have to be a person of color to advocate for inclusion. But it might be in a lot of context: “Oh, and by the way, we want to try our identities matter and we want to try to be inclusive in some kind of way,” as opposed to a real emphasis and real commitment to embedding it into every aspect of how we engage teaching excellence. And I think that that is something that is very much in process and a lot of places. I see there to be a lot of bifurcation between how we talk about teaching in general, like a lot of people don’t talk about student-centered teaching as a practice of inclusion. A lot of people don’t talk about backwards design of courses as a process towards making a more inclusive classroom, but it is… and so how do we connect in a more clear and articulate way how those mainline, mainstream, faculty development teaching excellence practices connect to broader efforts and work of inclusion? That bifurcation, I think, perpetuates faculties perception that the inclusion piece is not relevant, especially if they’re in a field that they think the content of their work is not relevant to those conversations.

Rebecca: I think it’s interesting that in a series of episodes that we’ve had on inclusion, this kind of theme bubbles up frequently… that evidence-based practices are a good way to start to be inclusive. And focusing on teaching and being student centered is a good way to be inclusive. So it’s interesting that that kind of bubbles up once again in this conversation. I think it’s also interesting to hear you talk about because I feel like I’ve experienced this a bit, that there’s teaching center stuff and that’s like one silo. And then diversity/inclusion is another silo. And accessibility and disability is another silo. But they’re all interconnected and we don’t often interact necessarily or work on things collaboratively in a way that could be useful. I think your background in student affairs also is another area where that’s its own silo. And those folks don’t necessarily interact with the academic folks as often as perhaps they could, because there’s a lot of different expertise in both of those silos, essentially, that benefit from one another.

Amer: Yeah, the student affairs piece was exactly where I was going to go. It was just shocking to me to move across from student affairs to academic affairs, and find out that norms and agreements were just not something that most faculty did and was not even like on the radar. I just was shocked by that when I first encountered it. I’ll never forget my first staff meeting… and coming from a student affairs background, you’re student centered, you’re thinking about students all the time. And I just remember, it was just in a staff meeting, saying, “You know, why don’t we get a student perspective on what they think faculty need?” And I was just looked at, like, I was an alien. I mean they were just like, “What are you talking about?” “Why would we ever ask a student?”… you know, and it’s like, because they’re the recipients of what faculty do… you know what I mean? So they have another perspective that could be valuable in getting us to think about what faculty need, not just hearing from faculty about what they need, but hearing from students too. So there’s all these different ways in which se silos end up creating challenges and I feel blessed and fortunate that I’ve worked across them. And it gives me a lens and perspective, but I increasingly find that that’s not typical as I work across the country.

Rebecca: Do you have some examples of really successful ways that folks have worked across areas or have been a little more integrated in the way that they think about inclusion and evidence based-practices and student and academic affairs that are worth maybe sharing as a model?

Amer: Well, I would say that anywhere that that’s happening, there’s a robust diversity, equity, and inclusion apparatus, structural work that’s working collaboratively across the institution. Because those areas, if they’re going to be effective, they have to be collaborative with Academic and Student Affairs. A senior Diversity Officer at a cabinet level, needs to have a good relationship with the Provost, and needs to have a good relationship with the VP of Student Affairs. So most of the examples that I know, there was a robust infrastructure around that, and where that more synergistic work is housed varies. Sometimes that can be within a Division of Diversity, Equity and Inclusion, where they’re doing some academic support resources, they may be working with a teaching excellence office collaboratively. I can think of Wake Forest as a place that I knew some of those things were happening. But I still think that, in a lot of places, too much of this is dependent on personnel-dependent relationships, and not structurally positioned to really create the expectation that these areas and some dotted lines in the org chart to really say that we think that these things are directly relevant and important and need to be connected. But yeah, too often teaching excellence and faculty development units are not at all connected to the diversity apparatus. I think it’s starting to happen because the Chief Diversity Officers are increasingly focused on the academic affairs area, and the need to engage that tough slog and the fact that students are protesting all over the country about their experiences in the classroom, but a lot of it usually depends on your Provost. And do they see the connection? Are they committed? Do they want to have a strong relationship with their senior Diversity Officer at a Cabinet level? Some institutions, their senior diversity officer is a Vice Provost for Diversity and Inclusion under the Provost and those are the places that I think you tend to see more of a natural connection because they’re within the same division of the institution. But oftentimes, in my experience, that silo between Academic and Student Affairs is a chasm, which is true in most institutions. But I think in a lot of those institutions, and they might have a separate focus on diversity within Student Affairs that is really operating almost autonomously from what’s going on the classroom stuff.

John: Let’s bring this back a little bit more to inclusion in the classroom. You mentioned a couple things that faculty can do. One is having students create rules for engagement in discussion and creating more activities that naturally bring students more in. Are there any other suggestions you have on what faculty who would like to start making their classroom more inclusive could do to make some progress in this direction?

Amer: Yeah, there’s obviously work that you can do in the content in terms of who are the authors, what perspectives they’re bringing of the content. Because if every single person that you’re citing for the content of your workshop is a white man, then at this point, most fields have a broader selection of people to draw from, or at the very least, highlighting key contributors to a field who are from backgrounds that have been historically marginalized, and noting their contributions. And so that’s a long way of saying there’s a curricular way to get it as well, that’s important. I’ll never forget my first English class in college, and it was a requirement, I went to Miami University in Ohio, and, you know, white male teacher, but he decided that all our reading was going to be World Literature translated into English from around the world. And I was writing my assignments, I thought, “Okay, whatever, I’m just going to do my homework and respond to these readings.” And again, it’s also about does the faculty member value the perspective that you’re bringing, and he made a point to make clear to me, like “You’re articulating perspectives, that are really different from anything I’ve ever heard, and from anybody else in the class.” And for me… and I think this is particularly true for younger students… is that I had never heard anybody say that to me before. Like, I didn’t think there was a value of being South-Asian and Muslim. I just thought it would made me different and weird from the majority, I didn’t think that was an asset. I didn’t think that there was something valuable to that. I didn’t know that what I saw and my perspective, that that was a resource for what was happening in the classroom, but he did. He valued that and he wanted to leverage that and he wanted to help me understand why it was valuable, so that I would be more willing to share my perspective, if I wanted to, towards what was happening in the classroom. And that’s why you have to set up the agreement about how we’re going to engage, so that I’m going to want to share that. Because I think, oftentimes, faculty in the desire for that student who might be a different background from everybody else to share, they may end up tokenizing, unintentionally, that student. And so that’s why it’s better to build it into the process, where you’re drawing it out from students, and they’re really making the connection on their own.

John: Because if you’re going to ask students to be representative of some group, you run the risk of stereotype threat and so forth, and making them feel more marginalized. Right?

Amer: Right. That’s part of those core intercultural skills and competencies we have to learn is that our identities are complicated. For students to be able to self articulate how they understand what they’re engaged in, in the learning, as related to their experience, it’s all about creating an environment where they’re going to want to do that.

Rebecca: I think kind of highlighting the idea of a personal note on an assignment. that is thoughtful… could be brief, but demonstrates that you’ve read, you understand, and that you’re interested,… that can go a long way in setting up the environment when everybody’s around so that private encounter can be really important to more public interactions. And I think that we don’t always think as faculty like the power in doing something, frankly, that’s fairly simple like that.

Amer: Yeah. So I had their weekly readings… and again, I made it really clear that I want to know about what you think, how do you connect your background experience to what we just read? How does this resonate for you? Don’t regurgitate it, because I read it. And the thing is that now they’ve spent some time connecting it to their experience before they’ve gone into class. And so for some students, they’re not comfortable just improvising in the moment in class. And so what I’m saying is that, when we engage in the conversation in class, you can draw from what you wrote, you don’t have to come up with it on the spot. Some students, they’re more comfortable with that; other students they’re going to want to look at what they wrote to really be their prompt. And here’s the other thing, as a faculty member, I know that they wrote it. And so if they don’t feel comfortable speaking or engaging, I’m not going to penalize them for that, because I know that they read it and I know they connected to their experience already. And obviously, you’re going to try to do what you can small group work, dyad work, other kinds of ways of getting them to engage, because some students are just not going to be comfortable engaging in a large group setting.

Rebecca: You mentioned a few minutes ago about intercultural competencies that faculty need to obtain. Can you outline what some of those are, so that faculty that are newer to this area, or really interested in inclusion but really haven’t thought about the competencies that they need to obtain… the little checklist of things to think about?

Amer: Yeah, and I will say that it’s really important to note that it’s a lifelong process, right? For all of us. We’re all learning, we’re all encountering, we all have assumptions and I think that sometimes I think it’s important to highlight that we all are in that process, because sometimes it feels like we’re saying, some of you have to learn and the rest of us, we already got it. Maybe because I was South Asian and Muslim, I had to adapt and adjust to more types that I’m more aware of more types of things automatically through my experience. But that doesn’t mean that I don’t have a whole lot to learn still. Let me just give you a quick example. I was at the National Conference on Race and Ethnicity a couple weeks ago, and I’m sitting in the car with three Pacific Islanders and we’re going back to our hotel from a social gathering and I find out that three of us are Muslim in the conversation. Now, I have to admit, I did not think that I was going to be in a car with two other Muslim people, given that three of them were Pacific Islanders; that was just my assumption that I made that clearly turned out to be incorrect. Now, I didn’t articulate that until later… I mean, I told them, because I was like, yeah, I have to be honest. But there was enough trust in those encounters and relationships. But my point is that we all are capable, we all have that learning to do, we all are going to make our assumptions and so forth. Some of the core competencies around intercultural development are self awareness… for me, the foundation is self awareness, we have to be able to spend some time reflecting on who we are, how do we understand ourselves and our experiences, our biases, our styles, our identities, including social identities in terms of race, ethnicity, sexual orientation, class. For folks from other countries, maybe race might not be something that they’re used to thinking about and that’s fine. So for somebody coming from India as a professor, okay, well, if it’s not race, then I know that there’s caste and there’s religion, and there’s other historically based systems of inequity. How does that shape your understanding? How are you positioned in relationship to those things? How does that shape your understanding of the world in their experience? I always find it to be interesting that we are asked to be self reflective as researchers, but not as teachers. I think that’s really an interesting thing. So absolute foundation… because to me, if you don’t have that foundational level of self awareness, you don’t have the reference point that you need to be able to empathize, which is the next key competency, and that when I say empathy, it’s validating someone else’s experience as true for them. We don’t have to agree and this is another area in which academics struggle, right? A lot of times we think that well, because I’m entitled to my point of view, no matter what, then I don’t have to be empathetic, because I don’t agree with you. And that’s not necessarily the case. So if a woman is saying to me, a woman identified individual, shares with me that she feels uncomfortable every time somebody is around, and I say that I’m sure they mean no harm, it doesn’t make me a bad person, it just means I’m not being empathetic. I’ve just dismissed how she feels and what her experience is and so it creates unnecessary barriers between us. If I did something like that, what’s the likelihood of that person’s gonna want to come to me the next time something’s going wrong for them? So when we work on it, it makes us more approachable. It makes us more trusted in these things. Another competency or skill is tolerance for ambiguity and I think this is a big one. Being okay with the fact that you don’t know all the details all the time and that’s okay. I did not know I was going to be sitting in a car with two other Muslims out of the three other Pacific Islanders in the car. But quite honestly, when they disclosed it, I wasn’t like, “Wow, I didn’t see that coming.” I didn’t do that because I’m like, okay, I didn’t know. I sat with the ambiguity, right? …rather than trying to make them feel strange for the fact that they’re Pacific Islander and Muslim. So for me, I get people asking, “What are you?”

And I’m a little bit racially ambiguous. And I’m like, “Well, I’m a person…” …you know.

“Well, where are you from?”

I’m like, “Well, I’m from Ohio, originally, and…”

”No, where are you originally from?”

And that can happen the first time you meet somebody. We don’t have a relationship… we haven’t established one… and I don’t necessarily feel like sharing my entire ancestral lineage with someone the first time I meet them. And some people are okay with that. Some people like being asked that. To me, I get asked that so often I’m like, “You know, I’m good.” I feel essentialized and tokenized in those situations and that creates a barrier… Again, unnecessary. So can we sit with that ambiguity? And that’s tied to things like patience, but it is good to be curious, a lot of people are like, “Well, isn’t it good to be curious and want to know”and I’m like “Yeah, that’s great.” Now with that curiosity, be patient and sit with the ambiguity as long as you can. But it’s important to be curious, because if you’re not curious, you don’t even want to know. So it’s important to be curious. These are some of the core competencies and skills that it’s helpful for everybody to work on, to position ourselves to be more likely to be successful. And then it’s like, knowing that we’re all going to make mistakes, and can we create an environment with enough trust to where we understand that mistakes will be made? And I think that’s important as well.

Rebecca: And the key there, right, is that there’s mistakes with both faculty and with students, right? Anybody can have mistakes.

Amer: Anybody is capable, so then it becomes how we navigate that and I think that’s part of those difficult conversations… concerns that a lot of faculty have these days.

John: How would you suggest faculty address that if they or a student makes an insensitive comment that offends other people, what would be a good approach?

Amer: Well, there’s a whole set of things tied to our whole conversation about how you create the environment. So there’s a prep in terms of how you create the environment for navigating moments like that. But then there’s like, what are you going to actually do in the moment? …and one of the things that some of my colleagues and I have talked about is that you’re allowed to pause… you know what I mean? …like to take a moment and really try to reflect. I think, also, it’s really helpful to ask clarifying questions. Can you help me understand what you mean by what you’re saying? Or where are you coming from? Can you help clarify? Because I think sometimes when we react, it’s not always necessarily operating from the clearest place and so asking the person who’s sharing to be a little bit clear about where they’re coming from, and the basis of their rationale for why they’re saying what they’re saying. That preps work and working on your intercultural skills, those are the things that are going to help you to be more likely to recognize that something is occurring. I think one of the number one things that students get upset by is it something that they view as problematic has come up and been said or asked and the faculty member didn’t notice it, didn’t recognize it, didn’t note it, didn’t say anything about it, didn’t address it, just kept on going. So there’s two things here. One is that if that happens, you’re allowed to go back the next class, if you reflect or a student contacts you and say, “Hey, you know, there was something that happened in the last class that I just want to address.” I know, folks are like, “I gotta get to my content,…” but you have to remember that you may have just lost a bunch of students in your class… they’re not going to trust you and they’re not going to go with you the rest of the course, if you just keep going. So you still have an opportunity to come back at the beginning of the next class, and to say, “Hey, I was reflecting” and to address it then, so that the rest of the students know that you are aware, and that it does matter to you, and that you’re going to try to do whatever you can to address it. And you may have to say we’re not going to resolve this here, but I do want to acknowledge that there were some concerns or x, y, and z. I think it’s important that we know that there were different sentiments or feelings or whatever. So those are some of the initial things that I really try to get folks to think about.

Rebecca: One of the things that I heard you say without directly saying it, I think, is that sometimes our gut reaction might be judgmental.

Amer: Yeah.

Rebecca: And it comes out that way, rather than from a place of wanting everybody to learn.

Amer: Yeah, I think this is an important thing for a number of people, of a number of trajectories and backgrounds. And what I’ve been talking about a lot is the difference between reacting and responding. Responding requires critical reflection, reacting is like that you have a stimulus, and then you do exactly whatever your response is to that stimulus. This is important for everybody. But I think if you’re from a marginalized identity, I think this is a big one, because students can say things that are triggering for you that you may have been traumatized or marginalized as a faculty member, I think that’s part of the reason why it’s important to do a lot of self work and reflection. And I think part of what we need to talk about is faculty getting the time to be able to be reflective, and that that being a value, that that is actually valuable for faculty to have the time to be reflective about who they are and what they bring to the classroom. Because the thing is that when we react, that’s when we’re more likely to draw from our implicit biases, that’s when we’re more likely to commit a micro aggression against a student, that’s when we’re more likely to do those things. And so we need the opportunity to reflect, to take the time to really understand who we are in relationship to other colleagues, with our students, so that we’re more likely to bring our best self into the classroom. That also involves faculty getting the opportunity to engage one another around these conversations. The number one thing I’ve noticed around the faculty development spaces around teaching is that they love the opportunity to talk to one another about what they’re experiencing, and what’s working for them and where their challenges are, and so forth. And they need the opportunity and space to do that. And I know that’s hard. Sometimes it involves faculty unions, and contracts and stuff, but I think we just got to make it part of what we do and ee got to create space for faculty to engage each other on these things.

Rebecca: Are there things that we think we should also address that we haven’t addressed yet?

Amer: I do want to note that I know that we’re in a very intense political and social climate in multiple trajectories and I don’t want to sound like I’m creating any false equivalencies. There’s hate, and there’s people being targeted for their identities and that’s a factor for what’s going on and that’s horrible. But there’s also, what I refer to as the culture of campus social justice elitism, in which I think we’ve created a new hierarchy around the language and discourse of social justice. Actually, there’s a reason why I talk the way I do around this stuff, and not constantly using an elitist form of discourse of social justice. And part of that, for me, is rooted in the fact that I was an activist before I came into the work… and more connected to grassroots activism. What I would say to my students sometimes is, let’s take all your big words, because they’re replicating what the academy is doing. It’s teaching them these words and languages and it’s like a way of showing that they know, which is where all the incentives are in the academy. None of the incentives are around not knowing, they’re all around knowing. So even around social justice stuff, I’m going to be performative around how much I know. One of the things I used to say to my students when I was at the University of Michigan, and I was like “let’s go to Detroit, where some people are organizing in the community. Let’s take all those words. And let’s just see how that’s going to go. These are the communities that you say that you advocate for and… you know what I’m saying?” …and I think they know. I think part of what we have to recognize is that it’s not just what students are doing, they’re being positioned to do certain things, whether it’s the impact of technology, whether it’s the way the Academy is structured, whether it’s where they are developmentally if they’re young adults, we have to continue to account for that. And so part of why we have to do our work is so that we don’t take it so personal. And yes, it’s hard. It is frustrating when students come at us in some of the ways that have been happening these days. And quite honestly, I think part of the reason why faculty are engaging these resources these days more is because they’re scared to death that they’re going to get blasted on social media, because they’ve heard it happen to a colleague or someone they went to graduate school, and they really don’t want that to happen to them. I wish that wasn’t the motivating factor for some faculty, but increasingly it is. So I’m not going to say that I have a magic wand. And I get, on a general level, the challenges of our time and the moment. But I don’t think that that’s a reason to not engage these processes and not to be committed to it. And we have to do that with authenticity, and recognizing that we also don’t have all the answers. So all we can do is just do the best we can. And if we’re committed to it, we can go down a path towards creating a more inclusive learning environment for all.

John: And whatever brings faculty to this if they create a more inclusive learning environment, it’s all to the good.

Amer: Yes, exactly. Absolutely. One of the reasons why I made sure that I prefaced what I said with “I don’t want to minimize the fact that there are people being attacked for their identities these days.” First of all, I’m part of one of those groups that gets attacked incessantly and demonized so I fully understand that. But secondarily, I think part of it is that we’re in this binary dualism of like, if you say one thing, that means you’re the opposite. Or if you say one thing that that means that you’re planting your flag in the ground. And this dualism means you’re either on one side or the other. And I think the academy shouldn’t be about dualism, I think it should be about exploration of knowledge, which is much more nuanced than dualistic camps on things. So I really do think that we need to actually start valuing and emphasizing not knowing, and I think that would actually make our teaching better.

Rebecca: I love that idea. Not knowing and being curious. That is really what the Academy is actually about. That’s what learning is about. It’s actually the not knowing.

Amer: That’s what it’s supposed to be about.

Rebecca: Yeah.

Amer: But I do think that the systems of the academy position us to replicate the idea that the only thing that matters is knowing; that critical thinking, even just epistemologically, we say that critical thinking is… in many cultural contexts, intellectual critical thinking knowledge is only one paradigm of knowledge, and that there’s other forms of knowledge that we can draw from. And that’s part of what we have to be open about. And that’s part of what our students are bringing from their various trajectories that they’re coming from… many different types of ways of knowing and being in the world.

John: We always end with the question, what are you doing next?

Amer: Well, the most immediate next thing is that I’m, in terms of professionally, is that I’m giving a keynote at a Jesuit institution diversity conference, I’m really excited about that. I’m very interested in the idea of connecting more intentionally religion and spirituality to broader intersections of diversity, equity, and inclusion, I think that oftentimes gets separated out. And I think for a person like me, who is part of a community that’s targeted, partially because of racism, but partially also because of faith, that I think is something that we need to spend more time being willing to engage. And I think too often in the academy we’re dismissive of religion and spirituality as something that is intellectually weak.. You know, weak minded or something. So it’s something that I’m particularly interested in, and I’m actually going to be co editing a volume focusing on that, which I’m really excited about as well.

Rebecca: That sounds really interesting and definitely fills a space that’s very empty.

Amer: Yeah. And particularly on a practical level, like how do we actually support and work with students and various constituencies on our campus around that?

John: Well, thank you for joining us. This has been a fascinating conversation, and I hope it helps lots of people in moving towards a more inclusive environment.

Amer: Thanks so much.

Rebecca: Yeah, thank you so much for your insights, This was really, a really productive conversation.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

95. Specifications Grading

Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, Dr. Linda Nilson joins us to explore how specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

Linda is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Specifications Grading: Restoring rigor, motivating students, and saving faculty time as well as many other superb books, book chapters, and articles on teaching and learning.

Show Notes

  • Linda Nilson – Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Nilson, L. “80. Self-Regulated Learning.” Tea for Teaching podcast. May 8, 2019.

Transcript

John: Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, we explore how the use of specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson. She is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning.

Rebecca: Welcome, Linda.

John: Welcome.

Linda: Thank you very much. It’s great to be here.

Rebecca: I think it’s more of a welcome back, right?

John: Yes. Welcome back.

Linda: Welcome back. Yes, yes, yes. Good to be here again.

Rebecca: Are you drinking any tea today?

Linda: As a matter of fact, I am. Yes, yes, yes. Of course I had coffee this morning as well. But I am drinking tea. I am drinking a berry tea, but it also has black tea in it. So it’s still a bit of a stimulant anyway, but it tastes really good.

Rebecca: Sounds good.

John: I have ginger peach green tea.

Rebecca: And I have English breakfast.

John: In an earlier podcast we talked with you about your work with self-regulated learning and one of the topics that came up with that was specifications grading.

Linda: Yes.

John: So we’d like you to tell us a little bit about your book, Specifications Grading: Restoring rigor, motivating students, and saving faculty time… especially that last part. [LAUGHTER]

Linda: Yes, saving faculty time. Yeah, I figured that would help sell the book. But it’s true. It does save faculty time. And one of the things that inspired this book was just my hearing so many complaints from so many faculty over the years, about grading and the aftermath of grading and returning grading material to students. And the constant steady stream of students trying to get another half a point, just arguing… just conflict… constant conflict… students being stressed… faculty being stressed… faculty getting larger and larger classes with less and less help… fewer and fewer TAs, if there ever any TAs. And I got tired of it. But there was a part of this that I did not invent. I heard it from a faculty member in the School of Management. And she was doing that pass-fail grading. It was saving her tons of time. She had huge classes, online classes. And she just invented this and she was also sick and tired… getting complaints from students and students not paying attention to her feedback, which of course, took her hours and hours and hours to write and return. So I took some ideas from her. But I also wanted to tie grades somehow to outcomes. And this is where another aspect of specs grading comes in. And that is with respect to bundling assignments, or turning them into modules or whatever. But I prefer the term bundles because it’s much more universal. So anyway, this was a solution to a problem. And that’s what a lot of my work has had to do with, making the faculty members job easier and more rewarding.

Rebecca: I think that you’ve been spying on me for the last many years. [LAUGHTER]

Linda: Yes I have. Yes I have. [LAUGHTER] I’ll admit it.

Rebecca: I knew it. I knew it. So I think John and I are both really interested in the idea of saving time, as are many faculty. But you also talked about, in your book, the history of grading… how its evolved… the 4.0 system. How is this different? And how does this relate to the history of grading?

Linda: Well, let’s look at the history of grading first. Grade started, well everywhere, in 1783. It was Yale’s idea. And what they started doing in that year was an achievement-based student classification system. They were not using As and Bs, what they were using were Latin designations of like optimi outstanding and pejores for failing, like as in pejorative, right? Anyway, then in 1800, Yale dropped the Latin designations and started using numbers 0 to 4. Sound familiar? But that was Yale doing that now. In 1850, the University of Michigan initiated grades, but for them, it was strictly pass/fail, and it only took 50% to pass. So we talk about grade inflation now… look backwards. [LAUGHTER] Mount Holyoke, though, just a few years later set passing at 75%. Harvard, also a little bit later, invented the A to F system, but passing was only 26%. So if you were wealthy enough, you had to know less, right? Okay. But anyway, that’s where ultimately grades came from before 1783. This was in Europe. And this started hundreds of years before our notion of grades. There was something… it was like a Jeopardy game, where graduates… or graduates to be… the hopeful graduates… were answering questions in a tournament style, but the stakes were really high. And so yes, if you are winning throughout, you were really showered with honors. But if you were at the bottom, you lived years in shame. It was terrible. We talk about high stakes. Oh my. So anyway, grades were invented after universities. Socrates didn’t talk about grades, right? It’s a relatively new invention for sorting students.

Rebecca: How does the specification grading relate to this letter system or this 4.0 system?

Linda: Well, it kind of takes a break away. Because first of all, all this grades that I was talking about was with respect to courses, there have been pass/fail courses. Sometimes they worked well, sometimes they didn’t work so well. They work pretty well in medical school ‘cause we’re dealing with a highly motivated students who really understand the need to know. But other than that, most students would do the absolute minimum to get their C-… whatever, it didn’t work very well in terms of like motivating students to learn, they learned the absolute minimum. With specs grading, the pass/fail is within the course, the assignments and tests in the course. But, you don’t pass with a C or C-. You set the passing level at what you would regard as a B level. And this is what restores the rigor. I think we have been sold a poor bill of goods, when we don’t set our students to high standards. We say, “Well, maybe they can’t do it, and then they’ll get…” They can do it. Come on. They’re just not doing it because of our partial credit system and our point system ‘cause they can always get another half a point, right? …just by wearing you down. [LAUGHTER]

Linda: Really, they do and they know it works, it worked in high school, so it works now. Now it may have worked the previous semester, these students aren’t stupid. And they can do what we ask them to do for “B” work… certainly at the undergraduate level, or actually at the graduate level as well. They could do it. But they choose not to, because they can survive otherwise. They could do well enough to pass or get their “B” or even get their “A.” So why should they sweat it, they can always get the partial credit. And students get partial credit. It’s almost like going back to the University of Michigan 50% passing for assignments and tests. I mean, we pretty much lowered the bar, because we want so many people to get over it. Well, people are getting over it without preparing… without doing a decent job. And they’ll spend 10 minutes… 20 minutes… the night before on an assignment knowing that no matter what they do, they will get partial credit and they will pass. So again, they’re not stupid, it’s just that, for them, college is a game that doesn’t have much to do with learning. That’s not where the focus is, and at least with specs grading, there is a lot more focus on learning, because you are tying the assignments and tests to student learning outcomes. And that’s a really nice part of the system in that those grades, the ABC, mean something in terms of outcomes achievement, and these, in turn, might be tied to program outcomes as well. So all of a sudden, you’ve gotten rid of an entire step at the departmental level of having to measure program outcomes. And why do you have to do this? Because accrediting agencies know that our grades don’t mean much of anything, as far as learning is concerned, as far as what they’re interested in, which is outcomes achievement. And outcomes achievement, it’s an up-or-out sort of thing. You can either do something or you can’t, and as long as there are standards set for that, and certainly the accrediting agencies want standards set for that. So let’s say “Okay, so your goal is a student can write a good quality, maybe not great quality, but a good quality business proposal.” Okay, fine. So if that is your outcome, what really does that entail? What exactly are you looking for in terms of a very good business plan? And that should be incorporated in your directions to students. But we don’t articulate that, do we? We really don’t. We speak very vaguely, “Well a business plan should have this and here’s your rubric.” Well, first of all, students often don’t understand the language, but is simply not detailed enough such that all students can understand the directions and actually follow them. We need to put more detail. Usually what we assigned undergraduates is some sort of a template or a formula to do this. And we’re not talking about that. We’re not sharing this template or formula with our students. And that’s what we need to do… not to say that you can’t allow for creative work. Matter of fact, with this system, you can and you don’t have to worry about tearing your hair out in different ways. “Well, I don’t know how to grade a movie.” Well, no, but you can talk about certain qualities of that movie, as in just simple things like the length, or perhaps the number of scenes that you want to see. And certainly they’ll be a learning goal and a communication goal connected to it. And that’s pretty much all you have to do. For most of our assignments, they are formulaic, but students don’t know the formula. And we need to tell them, and those are what you would call our specs for an assignment. So that’s really what we need to do. So all our work is up front and laying out what those specs are. It’s like a one-level rubric. But we don’t have to worry about different levels of the rubric, where there are four-level rubrics and five-level rubrics. We don’t have to worry about that. All we need is one. So we can afford the time to actually specify what we’re looking for… what that template involves.

John: And then the focus is helping students reach the standard, rather than negotiating with them over the grades for what they’ve received.

Linda: Exactly, exactly. Yeah, forget negotiation. There’s some things that ought to be non-negotiable. I mean, we’re not sloppy when we grade, I know, we get sick and tired of it. But we are so diligent in our grading. And then we have a line of students outside our office saying, “I deserve another two points on this because Susie said the same thing. She got 12 points, I only got 10.” …like write an essay about it… justifying it.

John: One thing I really like about it, though, is that right now we have these two levels of assessing students’ learning, we have these complex assessment plans for each department. And then we have a grading system, which often bears little resemblance to the assessment. And this is making the assessment transparent and obvious to students. And it’s forcing both faculty and students to focus on the learning objectives for the course.

Linda: Yes, as we should be, you think about it, what does it mean, when a student gets an A? Does that mean that that student has achieved all the objectives? all the outcomes at the level we want to see? Well, maybe for some of the students who get an A. What does a “B” mean? Now we’re starting to get into really ambiguous territory. Does that mean that the students sort of achieved or barely achieved all the outcomes or maybe achieved some really well and then didn’t achieve others? And a “C,” forget it. You can’t tell what’s going on there, especially given the way we give out “Cs” these days. So maybe the “C” student achieved one outcome well, but which one? Yeah, I mean, no wonder accrediting agencies pay no attention to our grades. They don’t even want to know them. And I can understand why, if they’re focused on learning outcomes. And so yeah, we’ve got this whole extra level of work that we have to do and the department chairs have to do and the Provosts have to do. I mean, who needs it?

Rebecca: Can you talk a little bit about the relationship between specs grading and contract grading, because it seems like they have some things in common?

Linda: Yes, they do have something in common. With contract grading, students individually work it out with a faculty member as to exactly what they’re going to do. And if they do all those things that they will get there “A” because I’ve never heard of a student contracting or a ”B,” it just doesn’t happen. Now, contract rating goes back to the 60s, and the first half of the 70s. And what was happening was since there was this individual relationship, first of all, faculty didn’t specify enough about what they wanted… what constituted “A” work… same problem we have had up to now… and faculty weren’t all that much different then, in fact, they probably gave less guidance. But in any case, there were the specifications. And then faculty and students would develop this relationship, this individual relationship. And it became particularly difficult for the faculty member to give anything but an “A” to the student. And the student would have thought, “Well, why didn’t you tell me to do this to get my ‘A,’ because that’s what I contracted for.” So contract rating was: 1. sloppy, but 2. highly individualized. With our class sizes, we can’t even talk about contract grading. And because it was something that was mutually established, between the faculty and the students, and specs grading is nothing like that. This is all in the faculty members’ hands. Well, at least, setting out what is required to get the “As” and “Bs” and “Cs” and “Ds” . But students choose what grade they’re going to go for. And according to what grade they believe that they need, but also according to what kind of a workload they want to shoulder for this particular course. And you know, maybe all they need is a “C.” And you know what, that’s okay. And we don’t look down on that student if the student says: “Well, all I need is a ‘C.’ We do otherwise. We say “Wow, this is a lazy student.” That’s what we’re thinking in the back of our heads. We have negative thoughts about students who settle for low grades. But with specs grading, you don’t have to feel that way. This student chose a seat for whatever reason, I don’t care. My course is not in that student’s major. So I’m not going to take it personally. So, that’s nice. But the fact that that’s making that choice, gets rid of all kinds of grading complaints, or that thing at the end of the semester: “What can I do to bring up my grade?” Well, it’s very clear what you could have been doing. If you went for a “C” now you could have done these additional assignments, or taking this additional test to get a “B.” But you didn’t do that. Now, you’ve got a week left, maybe you want to try to do that. But don’t ask me, I laid out the contract, I laid out the terms, I laid out the specs at the beginning of the semester, and you make your choices. This whole thing makes students feel a lot more responsible for their grades. They made the choice. And we respect that.

Rebecca: So I think a question that would come up for many faculty is you’ve laid out that certain assignments need to be accomplished to get even a “C…”

Linda: Yes.

Rebecca: …and what the specifications are. What happens when a student struggles to meet those specs?

Linda: Oh gee. Well, now when you say struggle, you can struggle before the assignment, or you can be disappointed and angry after the assignment. So if you’re talking about struggling before and say, “Professor, I don’t understand these two specs. I don’t even know if I can do it. Could you give me some more guidance?” No problem, and then because the stakes are higher, you get credit or no credit whatsoever… doesn’t count towards your bundle… doesn’t count towards whatever grade you’re doing. So you’d better figure this out. Now, afterwards, don’t meet the specs. Okay. And obviously, you didn’t come to see me in advance, did you or otherwise, we would have hashed this out. Anyway… so you didn’t pass? So you missed these two specs. All right, well gee, that’s most unfortunate. But there’s this system called tokens… or I call them tokens, but you can call them… I don’t know, you can call them pigs in a blanket for all I care, it doesn’t matter. They’re like get out of jail free cards. I’ve seen them called hail Mary cards. In a geography course, they were called globes, but anyway, they are opportunities to either redo an assignment that didn’t meet the specs, or to get a 24-hour extension. And there might be some other things that you make up along the way if absence counts against you in your course, you can get out of absence. But let’s say it’s for redoing this assignment, which is really important early in the semester, because students are not going to believe that you’re going to grade them pass/fail, because this is all new for them. And they go, “Well, how can this be…” and “There must be some sort of partial credit or something…” They won’t believe you. So there will be a number of students who will need this token or get out of jail free card to redo the assignment. It’s up to you, as a faculty member, as to how many get out of jail free cards or tokens that they might have. Three is a nice number. I like it, but some people give five. Some people give two. But I think it behooves you to have some reasonable number of tokens with specs grading, because that takes the pressure off of students; it allows them to screw up at least a few times. So this is something that students can get second chances out of this because the stakes are higher than typical. But the thing is, when you’re grading these assignments, all you’ve got to do is check the specs that weren’t met, and say, “This is why your assignment failed.” Now, if you want to give additional feedback, by all means, I’m not going to stop you. But I want you to have a life. That’s the only thing. And you can give positive feedback as well. And then you know what, you’ve passed something, then you get positive feedback from your instructor that you figure, “Wow, the instructor gave me this positive feedback out of the kindness of her heart, because she really cares about me and my success… told me how I could maybe do better in the future.” First of all, I’m going to read this, because this is meaningful feedback. This is not justification for having taken off points, because that’s what most of our feedback is about: why you didn’t get full credit. And now this is actual substantive learning feedback. So, it decouples feedback from evaluation in a way, but students get the chance to do it over and now that you have their attention, that let’s say they didn’t pass a certain assignment, that they can come to you and say, “Okay, how should I do this better?” Oh, okay. Isn’t this lovely? So the worst that will happen is you’ll have a conversation with a student about how to improve his or her work, that’s the worst that will happen. Isn’t that wonderful? [LAUGHTER] And sometimes students know, they just got lazy.

Rebecca: How would you suggest structuring specification grading for a class that’s more project based or even collaborative work?

Linda: Oh, sure. Whether it’s projects, or papers or whatever, again, it’s a matter of specs. Oftentimes, with our projects, that we might allow our projects different media, what we have to do is lay out specs for each media. Or say, if you can think of a different media, you can do a film, you can do a play, you can write pamphlets, you can write a paper if you want to, or you can do any of, let’s say, a half a dozen things. And let’s say if you want to do something else that’s not on the list, come talk to me, and we’ll work it out, we’ll work out some specs for it, too. So what you have to do is you have to lay out specs, but oftentimes the specs that you have to lay out, and students love creative assignments, is you have to lay out, let’s say, length, and so you might say, “Okay, if you’re going to go with a film or video, I want 20-mintues of that. Going to go with pamphlets? I want at least four pamphlets, I want each pamphlet to have at least 250-words in it. If you’re going to go with writing a play and performing a play for your peers, I mean, wow. But I do want it to be a half an hour long. And I want it to involve everybody in the group, everybody’s acting. And if you’re going to write a paper, I want it to be at least, let’s say, 2000-words long… or maybe that’s too long, whatever… and you lay it out, and I want it to follow this kind of an outline.” You can lay out specs for different kinds of media, and let students run with the project. Now a design project, what you might allow students to choose is exactly, let’s say, in architecture, what kind of a building they’re going to design. Let’s say, “Okay, you can design a residence, single family, a separate dwelling, but I want it to be at least four bedrooms, I want to be two stories, this and that. And you could do an office building too. But I want this to be at least two stories, and I want it to be so many thousands of square feet.” And then you let them run with it. And you know what, we know from our history, and from a lot of publications, that students run with this sort of thing. So I’ll give a little guidance and off they go, it’s a beautiful thing. Now in terms of projects having to do with group work, I don’t know that that specs grading will make group work smoother. [LAUGHTER] And I’ve talking about out-of-class group work. If you want to play marriage counselor, you just go right on ahead. But you don’t have to. I mean, there are some people who say, “Hey, you guys work it out on your own, I don’t want to hear about it and you can fire freeloaders. And if you are doing all the work, you can resign from your group and any other group will be happy to pick you up. But all this has to be done by the end of the fifth week,” whatever. So I don’t know that we’re going to get rid of those problems. [LAUGHTER]

REBBECCA: I think my question was more about the high-stakes nature or the long-term nature of projects.

Linda: Yes.

Rebecca: So would you recommend doing scaffolded specs or something like that to help make sure they’re on track?

Linda: Well, yes, what you need to do, as with any major project, you need to divide it up into smaller tasks that students can be at accountable for along the way, and maybe even get feedback for. Now, you can give feedback… your peers… people in other groups… can give feedback. Because after all, we’re doing criteria referenced grading, we’re not grading on the curve. So there are different ways for students to get feedback. But in any case, this divides a massive task into little pieces. So maybe starting out with, “Okay, by the end of the third week, we want your group to have a literature review put together.” And we put specs for the literature review. “And we want so many references. And we want so many of them in the last five-years,” And what have you. So we can even put the specs, I even specify specs for the format that we want: APA format, or ASA format, or Chicago, whatever it is that we want. That’s something that we can put in the specs if we want to, and that way students will actually proofread. Now I think we ought to be somewhat tolerant with respect to format anyway, we ought to allow three little errors in format. Because you know what, when we submit a paper to a journal, they’re tolerant of us, right? [LAUGHTER] I mean, you can’t be sloppy, but if we put a period instead of a comma, hey, we are forgiven. And we’ll catch it later on the proofs. So we need to allow some errors on something like that. But anyway, yeah, we divide up a big task into pieces. And then students get feedback on the pieces. And that’s also a good way to find out early about who are your freeloaders, who’s not doing the work early, so they can get rid of the freeloaders if they need to. Here’s another thing that you can do with specs grading, when you do bundles, you know, the project, the big project, you can make that big project required for an A, period. And then you will only have groups with students who are going for As. So it depends on the size of your class, of course. You might wind up with only, let’s say, three groups who are going for As. Or you know, the number of students that will fall into three groups. Well, guess what, you’re going to get some really fine projects, you’re not going to have to worry about it, and they might not be freeloaders. But anyway, in the examples in my book that a lot of faculty members and I recommended as well, that group projects be reserved for getting an A, for the A bundle. And other assignments be in the B bundle and C and the D bundle for that matter, which might be very minimal. But hey, if you want to a D, far be it for me to stop you. So, anyway, that’s another way to solve the problem, you just put those projects in just for the students are going for an A.

John: You’ve mentioned bundling a few times in terms of grades, could you talk a little bit about that and how you can go from the specs grading to the course grades that are assigned in the course.

Linda: Okay, there are two ways and the first way I’m going to just mention and not recommend. But you can keep your point system, and you might have some haggling over grades. But in any case, you can keep the point system and say, “Okay, my course is 100 points and if you get 91 points or more, you’ll get an A.” And okay, fine, but the bundling system allows students to choose their grade. So what you do is, you set up clusters of assignments and tests that students have to pass at that B level to get credit for their bundle. So you set those up. And the easiest way to do this, so you got a D bundle, is pretty minimal. And you can tell students, “Look for your D, here are the only outcomes that you will be able to achieve if you go for the D.” Some students say, “Hey, I don’t care, not in my major.” Fine. So it’s minimal work, maybe it involved just passing tests at let’s say, a 70% level, and maybe doing a couple of little written assignments along the way. But, they all have to meet whatever the specs might be. Whatever specs you set out for those assignments… And C bundle, “Okay, you got to do everything that you do in that D bundle, plus there are extra assignments as well.” And so maybe what you are having students do, and I’m just making this up as an example, is the students have to turn in 15 out of the 18 reading assignments that you have, they have to turn in a typed up outline on a particular reading. This way, at least your C students are going to do the readings, right? Because at least most of the readings, almost all the readings, and you’d give your teeth for almost all your students to do almost all the readings, right? But that’s all they’ve got to do. Maybe for a B you have to do all the requirements for a C, maybe even more of those reading notes, or something different. Perhaps keep learning journal, keep a journal on how you are learning, what you’re having trouble learning, the different strategies you are trying to learn the material, and it’s going to be collected. But you really have to specify what questions you want students to answer. So that will be collected maybe four times during the semester, and you might collect some every week or so and look at a few of them. But again, you’re just looking for the answers to the questions. That’s what you have to do for a B. You’re going to get a good handle on those readings and you’re going to find out how to best learn this material. For the A, you got to do everything for the B, but you have to do some sort of a group project where you’re going to learn, let’s say application, analysis, evaluation, what have you, whatever higher thinking levels that you have in there. At the C level, that’s pretty low. But good lord, at least they’re understanding the readings, at least they’re reading the readings and presumably understanding it. And you know, you can spot check some of them to see that they are understanding and if they’re not, you can help them with that. And as soon as they’re learning how to learn for that B, but students are learning how to analyze the material, even higher-level learning outcomes for that A. Now if your student learning outcomes are dictated by a professional accrediting agency, then you’ve got to put all of those outcomes in what’s required for a C, or maybe your institution considers D passing, some institutions don’t. Whatever you consider passing, that’s where all of those outcomes have to go. B students and the A students, they will achieve even more outcomes. If you’ve got an outside accrediting agency telling you what your students have to be able to do, then guess what? All the students that pass the course have to be able to do these things. And you’ve got to set out the specs accordingly. There are some accrediting agencies that layout good outcomes with active verbs. And other accrediting agencies, they give you crummy outcomes where you’ve got to essentially rewrite them so you can assess them. But I’m not going to get into that. [LAUGHTER] Did I answer your question, I hope?

Rebecca: Yeah, I think a lot of your examples focus on a more traditional class and not necessarily a lab or a studio kind of class where there’s a lot of class work that happens inside of class. What might a bundle look like in a situation like that? Where a lot of the learning activities are happening in person rather than for homework. So, for example, I teach studio classes that meet for six hours a week, and they presumably do about three hours outside of class. Versus the reverse, which many people have and labs are similar.

Linda: What area are you in?

Rebecca: I teach graphic design.

Linda: Okay.

Rebecca: But it would be similar to something that would happen in the sciences too and having to do lab.

Linda: Yeah, to an extent, it depends on whether it’s a decent lab or not. [LAUGHTER] Some labs are a waste of everybody’s time. But anyway, what you have to do there is, you’ve got projects, right? …throughout the semester, different things that students are doing. but how many projects do you have in a semester?

Rebecca: Usually, it’s like four big projects, or three.

Linda: Okay, well, now this sounds very, very radical. But what if for a D, you only had to do one project? …and I mean, at that B-level that you set to pass. What if for a C, as a student, I only have to do two. Now, for the first two, because I presume this is somewhat cumulative. For a B, I only have to do three projects. But an A, of course, I have to do all four, if there are four projects. Now, what does this mean? Oh, by the way, do you have tests?

Rebecca: No.

Linda: Just curious. Okay, it means that assuming your D student does passable work, that first project, you can say bye bye after that first project is done. If the person wants to sit in… you can’t, you know… it’s a free country, right? Well, not really. But they can sit in if you want them to, that’s fine. For a C student again, you can say bye bye after the second project, assuming they do it at that passable level. And by the end of the semester, you have only you the most motivated and committed students that you have to be concerned with. And you can give them some very challenging work. Now, I don’t know how you feel about that. But that’s the way you could do that. Now, you could also bundle it differently. You can expand the number of projects that you have as well and simply make them shorter if that’s what you would be required. But I don’t know how you react to that.

Rebecca:I think my accrediting agency would react to that in that they have to put a certain number of hours in. [LAUGHTER]

Linda: Okay, okay, so your accrediting agency is time focused.

Rebecca: It’s part of it. Because presumably with more practice, you get better.

Linda: Well, presumably. [LAIUGHTER] …practice with feedback, you get better. But yeah, if we are committed to this hours business, that makes things very unruly, everybody’s got to put in the same amount of time…

Rebecca: In class anyway.

Linda: Oh, in class, okay. That’s very unruly, with respects to specs grading because specs grading isn’t about time. I mean, you can recommend that such and such assignment should take you at least three hours for it to be passable. But other than that, that’s your recommendation for a certain given assignment. But yeah, that makes it very unruly when your accrediting agency says, “Okay, they’ve got to spend so many hours in class and not just looking at their phone.”

Rebecca: Yeah, I mean, we do critiques and things like that. We provide a lot of feedback. But yeah.

Linda: Yeah. Okay. You do critiques?

Rebecca: Yeah lots of critiques.

Linda: Yeah, like art classes and things like that?

Rebecca: Yeah.

Linda: Okay. Well, that’s nice. But now, you could hold your B and C students to doing critiques on all the assignments, just so they can put in their hours. But they don’t necessarily have to do the assignments, at least they can see models. But again, I don’t know if your accrediting agency would be happy with that either.

John: This may not exactly be specs grading, but could you give, say, different levels of activity in each of the assignments? If you have four-projects, you could have one bundle in that project that would give a C, another that would give a B, and another that would give an A, changing the scope of the projects and it would be specs grading within that scope.

Linda: Yes, that has been done and I’m not sure about what kind of projects you’re doing. But yes, you could do that. And you could set different specs for each level. So D students, lets say, have to do a D-level project, which is not nearly as time consuming and not nearly as high a level cognitively, it doesn’t demand that much, doesn’t demand much time, it doesn’t demand that much thinking. And so you could do that and that way students would put in their time, and as long as you can set out the specs for those levels. Yes, absolutely you could do it that way.

Rebecca: Yeah.

Linda: But yeah, accrediting agencies can really get in the way of flexibility. Rebecca, would your accrediting agency go along with that?

Rebecca: With scope, probably. Yeah.

Linda: Yeah. Okay, good. I want to tell you about a course that does this, it’s in computer science, they have students writing programs, and there are six projects that students have to do, they do these individually. If you are going for a C, you get some pretty easy projects to do, pretty easy problems to solve. If you’re going for a B, well you get sort of intermediate-level problem, you don’t do the C one. Because if can do the B ones, you can do the C ones. Now for an A you get some much more sophisticated ones. But still, for every level, you only get six. So there is that, that as long as you can designate level of difficulty, or for that matter, breadth of knowledge, if you can designate that… of course, usually people can do that… that you can make levels out that you can make different bundles out of that. This is another little interesting take, in this particular course, if you are late handing in your projects, that you’ve got to do two more projects at that level. So in other words, you are penalized for lateness by having to do more work. Now that really hits students where it hurts. And if you’re like super late, you got to do yet two more. So any way that keeps students on their toes. [LAUGHTER] Anyway, so in this particular course students are rarely late. So yes, there are ways that you can do that just in terms of level of complexity or level of cognitive operations demanded.

John: How do you explain this to students, if you’re going to introduce a specs grading system?

Linda: First of all, you’ve got to sell specs grading, because they’ll look at you like, “Huh?,” which is fine. And you should explain this, but you can tell them, “You know, I’m going to hold you to higher expectations.” And we know that holding students to higher expectations feeds into student success, student learning, but also student success in general. You could explain to them the concept of andragogy. In other words, that pedagogy really has to do with little kids. Andragogy has to do with adults. Saying, “I’m going to treat you like an adult.” And students like that kind of thing, you might be choosing the kind of project you’re doing. It’s easy to build choices into this. “This will be a safe but a challenging environment because you know exactly what you’re supposed to do. I’m going to give you the descriptions of what you have to do for the assignment to pass in advance and in detail. So you almost can’t screw this up unless you don’t read the directions or pay attention to the directions.” So you want to emphasize the choice and control over their grade that they’re going to have. You want to tell them about tokens, we’ve got some wiggle room, if you will. Now these tokens, by the way, are not physical, you make them physical students will develop a black market, [LAUGHTER] believe me, but virtual and you keep track, or you can ask for an extension, but you don’t ask me for an extension. You’re late, I just take a token away. That’s all. We don’t have to talk about it. I don’t care why you’re late, this is just the way it’s going to be. Just like your boss won’t care why you are late or why you are absent, it counts as like a holiday. Now we’re going to tie what your grades is to what you are learning. If they look at the syllabus at all, everybody’s going to know what you have learned in this course. So that’s how you sell it to students. Now students will need to be reminded about how the system works, because this will strike them. “This is so weird.” Even if they initially like the idea, they still need to be reminded a few times because it is really strange. According to my research, students way prefer this system to traditional grading. And one of the reasons why they do is because we give better direction, we tell them what we want, we give them the formula, we give them the template. or we give them tons and tons of freedom to meet the specs. Especially with students when we talk about length or whatever, length means depth to the students. It doesn’t mean that necessarily to us. Students really like this because they feel way more secure in it. And they do like the element of choices that are built in, that’s motivating for them. And they do learn more. And if they are more motivated, they’re likely to do more of the work at a higher level, they are more likely to excel. And the A students are going to do A work anyway, because they don’t know any better. All they know is A work. And there’s a sense in which they want us to love them. I mean, really, so they’ve got this strong sense of loyalty towards us and they really want us to respect them. And they’re used to feeding off of that they’ve been doing it all their lives. Don’t worry about them getting sloppy, they’ll never get sloppy on you. [LAUGHTER]

John: If someone wanted to transition to specs grading, how should they get started?

Linda: What you might want to do if you’re switching, transitioning to specs grading, you want to look at your assignments, and for that matter, look at your tests, like what you have in your tests. You want to look to see what can be transferred into pass/fail. You also want to look at your tests, if your tests are very objective, and you’re relying on a test bank, remember, it’s so much easier for you to grade with the specs that you can start assigning more written work or design work on tests, what have you. But definitely higher-order thinking types of questions and you will have the time to grade them, because you’re going to lay out the specs and without giving away the farm, you will tell your students the specs of your essay questions in advance, and they will study accordingly. But anyway, you want to look at your outcomes and then you want to identify the cognitive level of your assignments and your test questions which again, you might want to change. You want to be able to group your assignments and tests by that cognitive level so you can develop bundles. It’s kind of a radical way to do it, but it makes your life so much easier once you have those bundles. Because all you’ve got to do is say, “Okay, let’s see. So I had four assignments in this bundle, this student passed them all, hey B. It’s so much easier, you just have to be sure that you set out the deadlines for the different assignments in the bundles. You don’t want everything being submitted to you in the last week or two. So you still have to have your deadlines along the way. Just warning you about that. I want to refer people to my book for examples of courses, and not just my courses either, that have been specs graded. And they cover a range of over a dozen different disciplines. So they can see all kinds of examples.

John: That’s a great resource for those who are considering moving to specs grading.

Linda: Yes.

Rebecca: And it also sounded to me like a couple of cautionary tales that you have about transitioning are about time and making sure that you spread things out…

Linda: Yes.

Rebecca: …but also being very clear in what the specs are, and then following through with whatever you said.

Linda: Yep, yep. And if you didn’t lay out enough detail, well, there’s always next semester. [LAUGHTER] But you can’t change those specs in midstream. “Oh, I wish I thought of this and then start grading students.” No, no, no, no, no. But again, we live and learn, right? We’ll get better the second time.

John: We always end with the question. What are you doing next?

Linda: Oh, what am I doing next? Well, what I’ve been doing is I’ve been doing a lot of traveling, doing keynotes and workshops… oftentimes on self-regulated learning, sometimes on specs grading, and sometimes on any number of different topics having to do with either teaching or academic writing, or something like that. So I’m going to be taking a little bit of a break from that until the end of, or late, July anyway and then things start up again. But that’s kind of nice. I don’t mind that because this keeps me semi-retired and that’s the way I want to be. A semi-retirement is Nirvana, just letting y’all know. [LAUGHTER] So, that’s what’s next for me, that and to finish my tea.

Rebecca: Sounds like the key to semi-retirement is the semi not retired part. [LAUGHTER]

Linda: Yes, it is, it is. Your brain is still working and you still got your thumb in the pie so to speak. And it feels good because you’re not under that same 60 hour a week pressure that you otherwise have with a regular job and then doing these other extra things on the side and it doesn’t work real well.

Rebecca: Thank you so much for joining us.

Linda: Well, thank you for this opportunity. I normally say… autograph books and things like that… happy teaching, but I want to wish everybody happy grading. [LAUGHTER] An odd phrase right? Happy grading. [LAUGHTER] What an odd phrase, right? Happy grading!

John: Thank you.

Rebecca: Thanks, Linda.

Linda: Bye bye.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

92. Diverse Classrooms

The student population in most colleges and universities is becoming increasingly diverse during a time when much public discourse is characterized by growing political polarization and divisiveness. In this episode, Melina Ivanchikova and Mathew Lawrence Ouellett join us to discuss a MOOC that is being developed at Cornell University to help faculty nurture a productive learning environment for all of our students.

Mathew is the founding Executive Director at Cornell University Center for Teaching Innovation. Melina is the Associate Director of Inclusive Teaching in the center.

Show Notes

Transcript

>

John: The student population in most colleges and universities is becoming increasingly diverse during a time when much public discourse is characterized by growing political polarization and divisiveness. In this episode, we discuss a MOOC that is being developed to help faculty nurture a productive learning environment for all of our students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Melina Ivanchikova and Mathew Lawrence Ouellett. Mathew is the founding Executive Director at Cornell University Center for Teaching Innovation. Melina Ivanchikova is the Associate Director of Inclusive Teaching in the center. Welcome.

John: Welcome.

Melina: Thank you. It’s nice to be here.

Mathew: Thanks. Delighted to be here with both of you.

John: Our teas today are…

Mathew: I’m drinking Sea Buckthorn and Siberian Blueberry from Mongolia.

Rebecca: Wow, yummy.

John: That’s impressive.

Melina: And I decided to go the rebel route and I am drinking coffee.

Rebecca: That is a true rebel.

Melina: I apologize to all of your listeners who might be dismayed to hear that there’s a coffee drinker here in the afternoon.

Rebecca: Again, yeah… [LAUGHTER]

John: About half or more of our guests are drinking coffee or something else.

Rebecca: I have my nice boring English afternoon tea again.

John: And I have ginger peach black tea.

Mathew: Black tea’ s always appropriate. [LAUGHTER]

Rebecca: Can’t go wrong. So we invited you here today to discuss the teaching and learning in the diverse classroom course that you’ve been developing at Cornell. Can you tell us a little bit about the origin of the project?

Mathew: Sure, when Melina and I were introduced I guess, when we became colleagues back when I first got here, we were looking for a project that could play up to the strengths of the merger of our units. So part of being the founding director is two units came together. And I’ll spare you all of that, other than to say it was a great opportunity. So one thing was finding a project that had some heft for our newly formed unit. But second, and perhaps the primary part of this origin story was the inaugural address by President Martha Pollack, who was newly installed as President. In fact, the first thing I did when I got to Cornell, the first public thing I attended, was her inauguration. And in the context of her remarks that afternoon, she talked at length about the importance of creating an inclusive learning environment for all students. And I thought, well, I know just how to do that. And now we’ve got this fantastic staff. We have the skills and the expert knowledge that we can actually do something that would benefit our campus, but also might be something with a usefulness for people out on other campuses that might not have the same opportunities or resources.

Melina: And I’ll add to that to say a little bit about the context in which the course has emerged, which is that Cornell, probably like many other campuses across the US, was rocked by several events that happened both on campus and off campus. Moments of slurs being used in public… events that were very demoralizing and just strained the learning climate for students here. So, within that context, we’re also thinking about how to support our faculty and teachers in the classroom to be able to reach out to students and warm up the learning environment.

Mathew: Yeah. I would want to add, though, that this course is not in response to those. This isn’t a reaction to these sort of community and campus incidences. Mostly it’s to prove the point that at Cornell we’re as vulnerable to them as every institution in America. There’s really very little inoculation against it. And so what we thought is that if we could do something that had utility for our faculty that appeal to them and help them, that it might also appeal and be of use to faculty at other schools and colleges as well.

John: I saw a little bit of that at a presentation at a conference a few weeks ago, and I was really impressed. Could you tell us a little bit about how the course is structured?

Melina: Sure, we’re using a framework that has five different dimensions to it. And it’s the way that the course is organized. So we begin by asking instructors to reflect on themselves: “Who are you as an instructor?” And then who are students? How do you get to know who your students are? How do you help them get to know each other? What do you know about the students at your institution in general? And then how do you teach? What are the teaching strategies that you use? What is your pedagogy and part of that is talking about what you can do to prepare in advance for a hot moment that might arise, as well as what to do when there is a hot moment that arises. And then what is your curriculum? Both from the perspective of the content of what you’re teaching, but also how your discipline looks at the world, how has your discipline wrestled with diversity and inclusion at the broader disciplinary level. And then ending with really thinking about the learning environment and thinking about action planning, what are some changes that you can make to your course? And then what we’ve been seeing in those is that people think beyond the course level from changes small to broader and more systemic.

Mathew: So just to tag on to that, people have been thinking about their ongoing learning… things that they can do to continue to advance their own development, things that they can do at the course level, interventions that they might make at the departmental level. And that’s pretty exciting when they want to go out and talk to their colleagues. And then, third is thinking at the college and or the institutional level changes that they’d like to see happen in terms of the larger climate. They have actually been really ambitious and pretty exciting.

Rebecca: Can you talk a little bit about the timeline of the course?

Mathew: Yeah we, like everybody in higher-ed, are always looking for that sweet spot. And anyone who works with faculty or as a faculty member knows that there are about five or six weeks in the dead center of the semester where we might have half a chance of getting your attention. That’s it. That’s the sweet spot. And so the whole intentionality around the course being four weeks long was so that we could load it right in the middle of this semester, not right at the opening of the start of the launch of the semester, but also ending before the Thanksgiving holidays. Knowing that once people return to campus, faculty and students alike are all on the downhill slope and at that point it’s all about wrapping the semester up.

John: How many times have you offered it now at Cornell?

Melina: We’ve offered it twice, we just wrapped the second run of the course. And and I’ll just add to what Mat said earlier that we estimate that it takes people about 10 or 15 hours to get through the course. It’s asynchronous, and we release modules each week.

Mathew: And I should add too, just for transparency, we let people take as long as they want. So even though the course officially runs for four weeks, we can get tons of requests for extensions, and we’re happy to grant them. I mean, it’s just like teaching a group of undergraduates… we understand, mostly we want people to feel like they can complete the experience.

Melina: Yes, and we should say that the version that we’ve run on the Cornell campus is going to be transformed into a MOOC, a Massive Open Online Course, that’s set to run in November this year. So that will be open to anybody.

John: And you’re running that on edX.

Melina: That’s correct.

John: And there is a sign up form on your website and we’ll include a link to that in the show notes so people can be notified to join that when it’s available. I’ve already added my name to the list. Rebecca and I have talked about and we’d like to run a cohort here, through that as well.

Rebecca: Yeah, that’d be great. Can you talk a little bit about how faculty have responded in the last couple of cohorts that you’ve had?

Mathew: Sure. Well, I’m really gratified to say overall, we’ve had a very positive response and the only negative has come when people have run out of time when they said “You know, I’m just crazy busy and I wish I had more time to do a deeper dive.” So in terms of regrets, that’s one end of the continuum. But we also are, I think, assessing the utility of the course… of the usefulness of it… by people’s expressions of learning outcomes. So we do a pre-post with… this is just only for the on-campus cohort. But we’ve had fantastic responses along a whole range of outcomes, some we hadn’t expected, and others we had hoped for. Do you want to give some examples?

Melina: Sure. One thing I wanted to say that was interesting is that we also offer face-to-face opportunities. And we were wondering, were we going to get the same folks who come to those coming into the course? But instead, we’ve seen quite a range. One of the things that surprised me is that we asked people how many years they had been teaching. And so that range goes from zero years to 20 to 25, even 30 years of teaching and all along the continuum and quite a large percentage of people who have been teaching for more than 10 years. So that inspired me just thinking about how many people are committed to lifelong learning and willing to think about what’s happened in my classroom, my demographics have shifted, what is all this buzz around diversity? We’re getting folks who are really curious and willing to think and learn together. And so the response among faculty has been very inspiring because the core of the courses are these fantastic videos where instead of giving lectures through the videos, we’ve asked people to tell their stories about their lived experiences and their teaching practices. And we have faculty, staff, and student voices in the course…

Mathew: graduate students

Melina: …graduate students…

Mathew: and undergraduates

Melina: …and these testimonials, people they’re just… you have a visceral experience as you’re watching and listening to those. And so over and over, we heard the comment of faculty saying things like, “Well, I knew my students were people. But now after I’ve seen all these different points-of-view, I got to hear really personal things about them that I normally wouldn’t ask my own students. I have a much deeper sense of the challenges that they’re facing.”

Mathew: And the reverse is true, too. We’ve had graduate students say to us, “I had no idea my faculty member had anywhere near that sort of experience.” So, referring to a video where two of our colleagues talk about being first-generation college students, and having come from very poor backgrounds, or very poor working class backgrounds, and it was a revelation to our undergraduates that there might actually be faculty here who’d come from a similar kind of lived experience. The other thing that’s just been, I think, really a good metric for success is that people have often talked about wanting to go back and talk to their colleagues. And I think that, as Melina is talking about the nature of the videos, is that there’s so few opportunities to talk about this aspect of one’s teaching. You might, for example, sit on a curriculum committee or you might get into conversations about grading or end-of-semester evaluations, but rarely do you get invited into a more authentic, deeper, personal link between who you are as a human being… fully… holistically… and what you bring to the classroom. So I think the videos do a fantastic job and I want to put a little bit of a pitch in here. Melina facilitated all of those videos and I think she just did a fantastic job in getting people to relax and warm up and feel comfortable telling their story. It’s really powerful.

Melina: Thank you. The other core piece of the course is reflection. So throughout the course, there’s moments where we prompt participants to think about their own lived experience or their own socialization. And it becomes a very personal contemplative process. So I think that’s also one of the things that I’m seeing among the faculty participation is that yes, they’re active on the discussion board, but they’re also just really active and looking at the pages and reading the material. And it’s nice that you can track all of that information in online courses. You can really see how people are interacting.

John: How have faculty responded? Has it been growing? Does there seem to be a lot of interest? And I seem to remember something about there being a fair amount of administrative support there too.

Mathew: I’m really happy to report from the first time we offered it to the second time there’s definitely what I would call an upward trend line. We have far more people register in the spring. So that was a huge sigh of relief from Melina and I because of course, you know, if word on the street was negative, no one would have signed up. So we were immediately gratified that we probably have a 25% jump in registrations. And interestingly enough, we’ve had a number of department chairs who have been genuinely engaged as participants. We’ve had some Associate Deans… and I’m very proud of this fact, our president and provost both worked through the course themselves, because they wanted to be able to talk about it in a first-hand way. And it’s hard to express my gratitude to them for setting the tone as our senior academic leadership cohort to really send the message that this is something we all want to pay attention to. And I think we’ve had also the other group that can particularly be challenging in faculty development work to get to get engaged with this, senior post-tenure folks. And as Melina mentioned, we have a number of people who are full professors who’ve been teaching for quite a while, who said, “Yeah, I’m going to swing back around and take this course.” And both semesters we’ve done almost exactly a third, a third, a third. Graduate students and post-docs. Tenure line or laddered faculty and a full range within that from pre-tenure to post-tenure. And then about a third academic administrative staff who have teaching us some component of their job:, folks from academic advising, the Learning Services Center, other sorts of student activities related positions. But it’s made for an extremely interesting conversation. And I think everyone would say that they’ve benefited from that.

Melina: Yeah, one of the things that we made available as an option was for self-selected groups to take it as a cohort. So this is something that we were also hoping that when the MOOC comes out that some faculty development centers might offer a cohort experience for their own campus. And so those groups have been able to have leaders emerge from their own group and they have their own face-to-face sessions where they discuss the content of the course and take it just one step further.

Mathew: So we’ve had two experiences of that, that I think maybe would be interesting. I’ll share them. One is we teach an introduction to teaching in higher-ed course for graduate students, doctoral students, and post-doctoral students and they participated as a cohort. And that’s a natural affiliation. And just as you’d expect, they loved it, they got a lot out of it, it was enormously interesting for us to have them in the course. The other group that’s been equally interesting have been the department chairs who have been coming to it for a variety of different reasons. But the one I want to highlight is the idea that as you hire new faculty into the department… thinking about their orientation and onboarding, both to the department, but also to the institution. And that’s been a really interesting goal. And I thought, really, if I can say, this is a kind of a selfless goal, people really are thinking about the community writ large, and how to help people accelerate their integration into the values and the priorities of our institution. That was not something Melina and I had anticipated. We thought, sure, this might at some point contribute to new faculty development. But we really didn’t think of it as an orientation for department chairs in which they could then begin to think about their approach to teaching and learning and a way to communicate that with their new colleagues.

Rebecca: That sounds really interesting. Can you also talk a little bit about some of the specific ways that, through reflection, you’ve seen faculty talk about how they have changed their teaching or the impact that the class is actually having on their own classroom?

Mathew: Sure. Melina loves this question. Yeah.

Melina: So we did some interviews to explore…

Mathew: … just that…

Melina: … just to ask that question. So we have a testimonial video, which we can show you later. There’s a couple of stories that really stood out in my mind. One was a woman who went back to her guest speakers list. This was out of the Business College and realized that all of her guest speakers were white men. And she thought, “Wow, I can’t believe this happened to me. I thought that I was aware of this issue, but I really need to actually have a systematic way of looking at my curriculum so that I make sure that I have a diverse offering. I can try harder. There certainly are some women business leaders I can reach out to.” So that was one and another comment was somebody saying, “I do so much work in the community around advocacy for women’s issues, but I never bring that part of myself into the classroom, because I just don’t know how to do it. But now I’m thinking that it’s actually important to show this side of myself and I want to be able to share that a little bit more with my students.” Those are kind of my two favorite but…

Mathew: … there’s there’s a third one I love. One of our colleagues who’s a full professor here, talks about how she flunked out of college initially, and probably wouldn’t have finished except that another faculty member of hers reached out to her… and really encouraging and supportive of her and helping her figure out a way to finance her way back into school and to complete the program. And I think that’s sort of visceral level of authentic crisis, that undergraduates can often feel like they’re in that alone or that no one else has had that experience before them, or just that they’re in it alone. And so I think her willingness to sort of frame that, she used the course and the reflection exercises to frame that out as her story. And then she actually, this spring, shared it with her students. She had, I think, 12 or 15 people show up in office hours literally crying their eyes out in gratitude that she had shared that story because the amount of stress that they were feeling and isolation they had been feeling and that no one else in the community had put themselves out in a way that resonated that deeply for them. So I thought that was a moment where, of course, we’re not advocating that everybody just stand up and start babbling. But I think in a thoughtful way, she picked the right time and the right place, and the right amount of self-disclosure, and it had a genuine, immediate impact on her students. She teaches a large lecture undergraduate section, and as we all know, that can feel pretty anonymous to begin with. So I think that was just really lovely.

Melina: So one of the questions that comes up for folks is when and how much information to share about themselves and their backgrounds and identities. So she felt like, “Oh my students aren’t going to care about this part of me.” But midway through the semester, she noticed that some students seemed to be having trouble in class. So that was when she strategically shared this personal story and then had folks coming in and just thanking her for being open about herself and sharing.

Mathew: It was really a beautiful moment. So one of the outcomes, one of the ways I think we know the course of success is when we hear these kinds of stories back… because most of our colleagues, I would say, 99.9% of our colleagues have a good heart. They want to do the right thing. They want to connect with their students, but they just don’t know how to do it in a nuanced and appropriate kind of way. So this colleague is an excellent example of someone who was willing and ready… just needed a strategy to shape it in a way that was appropriate to the academic environment and to her role as a senior faculty member. So, I think one of the things Melina and I have been surprised about is the amount of willingness coupled with the amount of trepidation. There’s just a lot of self-consciousness on people’s part about wading into these issues because as we know, faculty are deeply socialized to not get out of their realm of expertise, you know, “stay in your lane,” as they say. And so we’ve heard over and over and over again, “I’m not trained as a therapist. I’m not trained as a diversity expert.” Well, welcome to the world. Most of us are not trained therapists or trained diversity experts, and so the exercises and the content of the course is really meant to build a sense of efficacy, just a way to get started. So we’re very clear with participants that this is not meant to be an activity that’s an end in and of itself. It’s meant to be a bridge onto further deeper relationships and experiences.

Rebecca: Can you talk about some other strategies in addition to self-disclosure that are revealed in the course that might get people itching to take the course once it becomes a MOOC?

Mathew: Well, one aspect of the course that I love is we focus a lot on active learning and student centered pedagogical strategies. That’s not the same as focusing on social justice and diversity issues, but it’s a predicate for it. It’s a super helpful way to get started. So we have just loaded the course with all sorts of very practical pedagogical strategies that act to warm up the learning environment by making it more active learning and more student centered. And we’ve tried to keep these things sort of discrete enough that you could peel off one or two of them. So we’re trying to break down this idea that either you go in and you do everything and all of a sudden you’re our diversity expert, or you don’t do anything. And by trying to give people options of two, or three, or four, or five different things that they might consider doing even in just one class session, it doesn’t mean you have to reframe your entire semester long course. But what our experience has been is that the response from students is so overwhelmingly positive when you move in that direction, that there’s a lot of internal motivation to keep moving in that direction to keep layering in active learning strategies. A lot of these are pulled from the PCAST report in 2012. And for a lot of our STEM colleagues, it’s helpful or there’s utility in being able to suggest the pedagogical strategy and then link it immediately to the research that supports its efficacy. And that’s been helpful on our campus.

Melina: Another thing that’s persuasive is hearing it directly from the students. So instead of having this giant checklist of “here’s all the little pedagogical tricks, tips, and tricks,” we try to be pretty thoughtful and reflective so it doesn’t become advice giving or something like that. But in the interviews, we did ask students to answer the question, you know, “Do you have an example of a time where you really felt a sense of belonging that was created or facilitated by a faculty member in your time here at Cornell?” And so the feedback we got from faculty talking about those stories was things like, “Oh, now I really understand.” Like, for example, we had a young, gay Asian male student who took a course where a faculty member just acknowledged that don’t expect to see any references to gay relationships in this literature, because this was a time where that was just severely censured. And so he just felt so glad to have it be acknowledged that it was an absence. So that’s something you might not think of, but you hear a student talk about it, and then you start to slowly get a picture. You hear lots of little stories like this, of a black student talking about what it feels like to be at a primarily white institution, and what has made a difference to ameliorate the stress that comes with that… hearing it from students and often the strategies that go with them are incredibly practical. Like break the ice, offer a genuine opportunity for students to get to know you as a person, have office hours that are kind and open, be really clear and transparent about how you’re grading. Some of the strategies are super practical and you wouldn’t even think of them as diversity strategies necessarily, but they do reach students well.

Rebecca: We had a similar experience with a cohort of faculty that I’m working with related to accessibility. And we met with some students who take advantage of some disability resources we have available on campus. And so we met with some of those students and talked about their experiences in their classrooms and what has made them feel welcome and not. And we had some very same positive reactions like, “Oh, I didn’t realize that a discussion class could be more tricky for you if you’re taking notes and things because you might not always know what the clear takeaways are if we don’t go back and summarize what was it that we just talked about.” So sometimes it’s just really small, easy things that a faculty member could do. We just don’t necessarily think about it. So I think those student responses are just so powerful and really helpful.

Mathew: I totally agree. Another example that we’ve gotten very positive responses to is that when there’s been a national or regional or a city-wide or a campus-wide incident that’s happened that we know has resonance for our students, we have sent out some strategies for faculty to use in the classroom, beginning with just acknowledging that it was rough. This was rough to experience this, whatever that is, fill in the blank and letting students at that point know, you just acknowledge that this happened. And you don’t have to go any further than that. Just acknowledging, “Over the weekend such and such happened in downtown or it happened on campus and I want to acknowledge that and ask you to be sure to take care of yourselves… reach out to your friends… your family… reach out to services on campus, and here’s a short list of services that you might take advantage of.” But just that aspect of acknowledging it, students find profoundly helpful. So if you’re not making, as Melina’s example was so eloquent about, taking it out of invisibility, and making it real and bringing it into the classroom environment. Because one of the things that we know is that students care most about how their faculty interact with them. So in the college experience, we know there are two key predictors of undergraduate success. One is meaningful relationships with their faculty. The second is meaningful relationships with peers. And so even though the student affairs folks and the residence hall folks are wonderful people, and they do a fantastic job. If they’re not hearing acknowledgement from their faculty, if these issues aren’t coming up in class, then there’s a huge gap for that… they really feel the absence intensely. So we in the course try to give participants strategies depending upon their level of comfort. So I always say, “You don’t have to go one step further other than say, “Wow, rough weekend, be sure you take care of yourself.” And then move right into your content.” But just that moment, those two or three minutes of acknowledging the moment and acknowledging students are real people and they have significant feelings about these incidents can make a huge impact on their experience of the environment. All the way to the other end of the continuum where we have a wonderful colleague who will literally throw out the curriculum for the day, put people into individual writing exercises, and then into dyads and then into small groups and into a large group to process what the implications are for whatever happened for them individually, and for us as an academic community. It’s a continuum in what we try to reassure people… as anywhere along there is useful. Anything is better than simply ignoring it, and starting with where you feel ready.

Melina: Yeah, so one of the outcomes we’ve heard from faculty is them saying, “Well, you know, I sort of got the message from the senior administration that I should acknowledge but I wasn’t fully convinced. But once I took the course, I realized, Wow, it really does matter to them. They really do care about this, it really does make a difference. And now I have to figure out how to do it.”

John: Bringing that in through student voices, I think is a really effective way of doing that. And I was very impressed with the sample videos that you showed at that conference a few weeks ago.

Rebecca: I think the time and space that you give faculty to reflect on those moments is really important. Just in the conversation that we’re having, I was thinking back to moments as I was a student when things like that had happened. And there was one moment that sticks out in my mind that I don’t remember any other faculty handling an incident. I was a student during 9/11 and I remember one faculty member in particular did that throughout the curriculum thing. I was in a creative degree so the conversation was, “Hey, it’s really hard to make when you’re scared and things are going on, and you’re not sure what’s going on in the world. Sometimes it can be difficult to make, but sometimes it can be therapeutic to make.” But we talked through what that means is a professional when things like that happen in the world. And that stuck with me forever since then. I think it can be really powerful, whether big or small or a big amount of time or not. And I think taking the time as a faculty member to remember some of those moments that you had as a student is also really powerful.

Mathew: I love your story. And it’s one of the learning outcome goals for the course which is that you do not need to be an expert. You don’t have to have an answer. You just have to hold the conversation and facilitate a moment of reflection and connectivity. And I think in faculty lives, there’s such a drive towards being an expert and delivering an expert’s answer, or solving the problem that I think one of the big takeaways from the course is that with this sort of engagement, you really just have to be present and be authentically yourself. And that in and of itself is the work.

John: One of the issues that many underrepresented groups have to deal with is stereotype threat. Are there any particular strategies that are addressed through the course to help faculty reduce that?

Mathew: We do explicitly address both stereotype threat and also other sort of key concepts that I’ll come back to in a moment. But in particular, with stereotype threat, some of the ways that that can get triggered is unconscious and unintentional. Where you, for example, ask someone to answer on behalf of what you perceive of their community to be. And so some of the discussion guidelines that we give people and some of the resource materials that are a part of the course go explicitly in setting up environments where you can anticipate and ameliorate stereotype threat from the very beginning. And part of that is making really public your perception around mindset. And this is one of the most popular strategies, but also really effective… to make it clear that you believe that intelligence isn’t inherited, and it’s not static, that we get better at things by practice and by application. For example, we often say, “We wouldn’t have accepted you as the university if we didn’t believe you have the acumen. But having acumen is not the same as having all of the prior preparation that some of your peers might have had. And so figuring out what you need in terms of strategies and learning how to learn, those are things that you can achieve, that we would expect that you would need to work at them.” So even being at Cornell University was extremely interesting. We have a very well prepared undergraduate student body in many respects, just pretty spectacular people already. But a proportion of, a group of them, have come through high school just sailing through. They never really had to develop really coherent strategies for learning because they were just always ahead of the curve. They get here their first semester, their first prelim or mid-semester exam and it’s often quite shocking. And I think for many of them very destabilizing. For example, the first year I worked here, the daughter of a good friend of mine was a first-year undergraduate student as well. She got an 80 on her first exam and literally collapsed. I mean, she literally thought she wasn’t cut out for college. She shouldn’t be here. This was too big a reach for her. She was never going to be successful. And I was still trying to wrap my brain around, “How is an 80 failing?” But this is a kid who never in her life had ever seen the 80s. She lives in the 90s or the hundreds. She’s never seen the 80s before, but all of a sudden the level of competition across the institution is at such a level. And I think that’s true in many institutional settings from community colleges right up through university. And so helping students learn some concrete strategies for, at sort of at a meta-level, learning about themselves as learners is another way to ameliorate that. So we have a lot of strategies like that in the course too.

Melina: Yeah, and I’ll add to that even when we don’t say this is how to ameliorate stereotype threat ABCD, a lot of the strategies are doing exactly that. And we’ve just put them in the course where it makes the most sense to have them. So at the beginning of the course, we talk about things things you might consider as you’re establishing your learning community within your classroom, including how to help students get to know each other. One of my favorite all time icebreaker exercises is to invite people to tell the stories of their name… like the origin of your name story. When we think about bringing the whole person into the class… just allows people to share some cultural information because our names are encoded with all sorts of cultural information, whether you’re married or not, whether you’ve changed your name, immigration patterns, history of oppression… are also encoded in names. We also have a very high percentage of international students on campus so that enriches the name stories as well, because you get different naming traditions. Names tend to mean different things across different cultures. So over time, you also get a bigger picture of how the world works based on people’s name stories. So that’s just a little example of that. We had another faculty member who sort of shares how he uses an identity pie activity to share a little bit about his own identity. So not just a single identity axis. So that also helps to ameliorate stereotype threat because you prompt someone to anchor themselves in the complexity of their identities and then you’re not just a Latin-X student in the classroom, or a person speaking with an accent that sounds different from most, or a person with a disability. You’re just much more than that. And I think that’s probably one of the strongest features of the course. Because it’s sort of something that comes out throughout every aspect of the course… is just people are more complex. Here’s ways to welcome that in.

Mathew: Yeah, social identities pie is a great example of what we try to do in this course, both giving people an opportunity to reflect on their own growth and development, but then to have an exercise that they can peel off and use with their own undergraduates. So that we would expect that that would be useful to you personally, but also it would be a fantastic tool to carry away and use in the classroom. You know, of course, depending upon your subject and your specialization. And so through the whole course, we try to develop what I would consider sort of heuristics or models that help you individually, but also, I think could be really useful for you as a teacher and instructor in helping your students grapple with these issues as well.

John: So modeling, in the course, how courses can be delivered to address these issues effectively.

Mathew: Yeah, that’s exactly our goals

Rebecca: How incredibly meta. [LAUGHTER]

Mathew:But that’s some of the fun of it, I think. And we try to be really transparent about that in the course. So we have what I would call annotations all along in the course. “Here’s something we’re going to ask you to do that we also think would be useful to carry over into a classroom as well.” And some of the discussion questions are really about, “What was this like for you? And do you think this would work for your students as well?”

John: I’m going to throw in a reference to a past podcast we had. You mentioned how building a growth mindset can be really effective. We did an interview last year, I believe it was, with Angela Bauer at High Point University who uses growth mindset messages, weekly in classes, and it’s been found to have a significant effect on reducing performance gaps in the classes there… effectively eliminating them.

Mathew: It’s amazing what a few well chosen messages can do. And as Molina mentioned, it’s a great way to prime students, but it also makes transparent what your values are. So one of the exercises in the course that we asked our participants to do is to craft a multicultural or a diversity and inclusion statement. You can call it whatever you want. But just to put out there for students to read in the syllabus. Here’s what I think an inclusive classroom looks like. And these are the attributes of it. And these are the behaviors associated with it. And this is why I think it’s important in the context of the course but also in the context of the discipline. And it’s remarkable how effective that is. If you do nothing else, but that to strike out and make your own values transparent to your students, it can be pretty amazing.

Rebecca: So when can we start taking this class?

Mathew: Oh… the fall… we would be delighted to have you participate. And also we really hope to stay in touch with people who do take it and use it as a learning experience for a faculty learning community on their campuses. To be quite honest, that’s been one of my number one goals all along, of course, has been to serve my own institutions community here at Cornell. That’s our number one priority. But we think there’s relevancy. We think what’s going on here is pretty common. And in fact, a lot of campuses and a lot of faculty are likely starting at similar places. And so our hope is that you can take it yourself, but also grab it and bring in a bunch of colleagues at your own institution and have a shared experience, primarily because we think that you will be able to tailor this to your institutional context. I think it’s really important to make it personal and make it authentically linked to your legacy, your history, your current demographics, whatever the initiatives are on campus. We hope that this will be situated within a more robust conversation at the campus level.

John: When I was seeing the initial presentation on it, I texted Rebecca about this and said, we should run a cohort on this in the fall. We’re very excited about the possibility.

Rebecca: Yeah, definitely.

Mathew: One thing I would just want to add is that we’re going to design the MOOC so that people can take it individually, as well as as a cohort. And I want to reassure people that we’re deeply aware of how constrained faculty are for time, it’s just really tough to carve stuff out. Even if your heart is there and your intentions are gold, it can be really challenging. So we’re really going to try to send the message that it’d be ideal if you could do this within the context of a group, but you could also just grab and go. You could jump in and hopefully it’ll be a benefit to you individually as well.

John: We’ll share links to information on that in the show notes.

Mathew: One thing I would say is that I think people have found it a lot less scary than they thought it would be. It’s very important to know that we don’t have a subtext or a secret agenda of hunting for the racist. That’s not our goal. It’s not how we facilitate the course or how we facilitate the MOOC either. And so Molina and I were laughing about the fact that a lot of people have had prior experiences with diversity related training or professional development or workshops. And we were laughing because I’ve heard this since the 90s from people saying, I took a consciousness raising workshop in the 70s. It was horrible, and I hated it and I’m never going back. Or these opportunities come to people as mandated top down HR related expectations. So you have to take this course and sign it before you can get your contract. And we’re the antithesis of that. This is strictly voluntary. It’s strictly collegial. And it’s meant to be an opportunity, as you were saying, to get meta… to just step back from the doing and have a chance to think about resources that are useful in shaping our thinking, which in turn will shape our behaviors. And for most of our colleagues in the faculty, I just want to underscore it’s not that there’s a lack of willingness. There’s just time to get the resources and have some focused time to think these things through and apply them in a tailored bespoke manner to their own context and discipline and courses. And I think that’s what the course really offers. It sort of gives you this lovely little bubble of a garden in which to sit and reflect and think in ways that you don’t typically have in the course of a day.

Melina: You know, one of the things that we’re seeing in our survey data is that people’s sense of responsibility around this issue increases… goes from “The university should do this, but I don’t have to do” this to going to “Oh, yes, this is about me and what I do.” There’s just a much higher level of awareness and excitement about being a part of it.

Rebecca: …probably speaks a lot to the idea that reflection is a very valuable teaching tool.

Mathew: Yes, and one that as instructors, we know this, we know this, but it’s easier said than done a lot of times.

Rebecca: I’m really curious about… behind you under window. There’s a tomato.

Mathew: Yeah.

Rebecca: …it looks like a tomato.

Mathew: It is a tomato. Thank you. [LAUGHTER] I’m going to tell my husband who’s an artist who doesn’t think I can draw that you recognize it as a tomato. So, thank you. It’s the pomodoro technique.

John: That’s what we were wondering, actually. I think Rebecca and I both had that thought.

Mathew: I cherish when I can get literally five minutes in a row to complete a thought. And so I’ve taken to taping over the class and my door with a tomato to signal my colleagues. I’m here. I’ll be available in a moment, but I’m just trying to get one thing done.

Rebecca: So you’re human then.

Mathew: Yeah. Oh, yeah. Oh my gosh, yes, yeah.

John: So since you’ve created the course, could you tell us a little bit about your background in the area and your experiences related to the course?

Mathew: One thing I love, which is completely accidental… is that Melina and I are both from New Mexico. And that has absolutely nothing to do with anything except it’s extraordinarily rare to meet another person from New Mexico. So I just love that… that’s just as sort of a weird thing we have in common. She actually grew up there. But I was born there, but didn’t really live there in my childhood, but you lived there. The other thing that we share in common is we both have traveled a lot internationally our entire lives. Melina and I have both been, what I would call third-culture kids where we’re American by citizenship, but also culturally, it’s much more complicated than that. And I’ll let Melina tell her part of that story. But I think that’s been really important in our growth and development and of our approach to these issues. So my father was a pilot in the Air Force. He was a fighter pilot in the Air Force for his career, and we moved a lot and we moved all over Western Europe and all over the eastern seaboard of the United States. So in my own lived experience, I’ve had a lot of opportunity to both be an insider and an outsider. And that has, I know, shaped my approach to this work as sort of a specialization level. I have a doctorate from University of Massachusetts Amherst, in multicultural organization development. So it’s my research area, as well as sort of my lived experience. And I’ve been out as a gay man for a really long time… since probably high school… early high school and growing up in a military community and also State Department community, my dad was a military attache, I think that really shaped me… sort of that fitting in, but not fitting in, that a lot of times it’s called code switching where you have to sort of adopt a certain set of behaviors or certain narrative form to fit in whether that’s your home base or not.

Melina: … What about being a white man… [LAUGHTER]

Mathew: Oh, yeah. Yeah… John and I have this in common… we’re both greying a little bit or at least I’m greying and so I walk into the classroom and I get an enormous amount of privilege, a benefit of the doubt. People automatically assume I belong at the front of the classroom. I’ve never been mistaken for our grad students, even as a grad student… people always thought I was faculty. But because I teach in social work, my specialization areas and my practice was in social work. And so I taught at Smith College in the School of Social Work for about 10 years. And always, whenever I do this work, I have to lead with “What’s a white guy know about diversity? And who am I to be at the front of the classroom?” And so I have, of course, as you’d imagine a pretty comprehensive response to that. But mostly, I like to lead with the idea that this is everybody’s work and that white men have a role in this as deep and as important as women of color. It’s just two ends of the continuum. But if white guys aren’t involved, and we’re not taking it seriously, particularly with a privilege that comes from being an academic, than I think we perpetuate misogyny, and patriarchy, and racism in deep ways. So I think I can see when I do that when I start right off with, “Okay, I know the first question on your mind is, ‘What’s a white guy know?’” I can see the visceral level of relief in the room because it was on everybody’s mind and until we address that I know we can’t get on to the work of the course or the session or whatever. So it’s pretty fun.

Melina: So a little bit about me. I’m an Associate Director of Inclusive Teaching here at the Center, which is a new position… a new role since last July. And before that, I was focused on supporting global and intercultural learning at Cornell. And my interest in this particular area has been sort of bubbling and growing throughout my entire life as Matt alluded to. I grew up bilingual and bicultural, Argentinian-American and spent part of my childhood living in Uruguay, where my mom and her family still live. And doing that kind of cultural code switching of realizing I was an American at I think age 10… having these moments of self awareness that sort of continue to grow. And I still continue to have the moments where I realized “Oh, I had a blind spot in relation to not really understanding this particular other way of being in the world.” So and I’m a poet by training, which I think has honed my observation skills. And I’m a former faculty member, I used to teach English at a community college in Massachusetts where I was specifically hired as a bilingual bicultural faculty member to do quite a lot of teacher training and faculty development, actually, around that particular identity category. So I also had to contend with the complexity of being a white identified Latina woman and what that means and seeing my Latin-x students eyes get really big and be like, “Wow, I didn’t even know there were white Latin-x people.” When they didn’t believe I could speak Spanish until I would speak Spanish to them. And that would sort of challenging the assumptions of who we are and I love the discomfort that comes from being in the soup that is the complexity of identity and learning from how people’s experiences of being misread or mislabeled or misunderstood inform us about how to do better in terms of building inclusive communities. So the work at Cornell… there’s a lot of work to be done… but it’s also an exciting moment because there’s a lot of people on deck thinking about this. So the response we’ve seen from the faculty and then the President… also being able to speak about this is incredibly inspiring. And then also going out to other campuses and meeting you in New Paltz and seeing other people are hungry for these conversations too, and students have a lot of place to think about their identity formation. And faculty, they’re not often necessarily asked to unless there’s suddenly an occurrence or an opportunity or an invitation. So I like being able to offer those moments of invitation to think about this together.

John: We’re glad that you do. It’s a very nice resource.

Rebecca: Yeah, we’re definitely excited to explore it with our colleagues here.

So we always wrap up by asking: what’s next? [LAUGHTER]

Mathew: Well, now that we’re concluding the second iteration of the on-campus course, the next is to actually write the MOOC. And we’re also going to write a Course Guide. So for folks like yourselves who might host or facilitate a learning group there, this is a genuine invitation to feedback. We think that we’re going to have a really fine course… it’s going to be worthwhile… but we also always know there’s room for improvement and so we’re hoping that this will be a sort of a virtuous loop of feedback from participants. And the course from the fall to the spring changed a lot… we learned a lot… and I expect that the same will be true of the MOOC as well.

John:That’s something we all should do with our courses, which is, again, a nice practice to share.

Rebecca: Oh look, reflection comes back again.

Mathew: Absolutely. [LAUGHTER] Absolutely.

Rebecca: Well, thank you so much for joining us today.

Melina: Thank you

John: Thank you. We’re looking forward to continuing the conversation through the MOOC this fall.

Rebecca: Yeah, definitely.

Mathew: Absolutely. It’d be really fun in another year, assuming that we get it written and published, and that you get a chance to convene a cohort… it’d be really fun to come back and do it again and talk about what was it like, from your perspective, your experience on the ground? That would be really, really solid.

Melina: We can interview you for your own podcast.

John: Yeah,that would be a nice twist…

Rebecca: That would be fun.

Mathew: That would be fun, yeah.

John: We did have someone do that. It caught us by surprise because we weren’t ready for that.

Rebecca: Yeah.

John: But fortunately, we have the ability to edit. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

89. Teaching About Race

Class discussions of race and racism can be difficult for all participants. In this episode, Dr. Cyndi Kernahan joins us to discuss ways of building a classroom climate in which these issues may be productively explored.

Cyndi is a psychology professor and Assistant Dean for Teaching and Learning at the University of Wisconsin – River Falls. She’s the author of Teaching about Race and Racism in the College Class: Notes from a White Professor, which will be available from West Virginia University Press in Fall 2019. The book will be part of the Teaching and Learning in Higher Education Series edited by James Lang.

Show Notes

Transcript

John: Class discussions of race and racism can be difficult for all participants. In this episode, we discuss ways of building a classroom climate in which these issues may be productively explored.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Cyndi Kernahan, a psychology professor and Assistant Dean for Teaching and Learning at the University of Wisconsin – River Falls. She’s also the author of Teaching about Race and Racism in the College Class: Notes from a White Professor, which will be available from West Virginia University Press in Fall 2019. The book will be part of the Teaching and Learning in Higher Education Series edited by James Lang. Welcome, Cyndi.

Cyndi: Thanks.

John: Welcome. Our teas today are:

Cyndi: I actually I just have water although I am a big tea drinker usually.

Rebecca: I’m drinking golden-tipped English Breakfast tea.

John: That’s a new one.

Rebecca: I know I’m branching out! [LAUGHTER]

John: And I have blueberry green tea today.

Rebecca: We’ve invited you here today to discuss your forthcoming book, Teaching about Race and Racism in the College Class: Notes from a White Professor. Can you tell us a little bit about the book?

Cyndi: Yeah, the book is essentially my answer to a question to my earlier self. So when I started teaching about the psychology of racism about 20 years ago, when I first started here, I felt desperately in need of help, because I’d always wanted to teach about the psychology of racism but it was much more difficult than I anticipated, as most teaching often is. And I was very young and new and I wanted a guidebook and there really wasn’t one. And so I kind of have had it in my mind for a long time and about five years ago, I started thinking seriously about how to do it. So the book is meant to be sort of a guidebook. It’s got both my own experiences, but also a lot of evidence in it. I’m a social psychologist, so there’s a lot of evidence from my field that I think is very easily translatable to the classroom in terms of how to learn and how to think about these issues because they’re hard to teach. It’s hard to teach about racism, I think. There’s a lot of difficulty in it. There’s a lot of evidence and also just sort of my overall philosophy about how we can teach it in compassionate but very honest ways. And so that’s my overall thinking…making sure that you tell the truth but that you tell the truth in a way that doesn’t alienate your students and keeps them engaged, which I think is kind of a can be a difficult line to walk. So, that’s kind of what it’s about. It covers a lot of different things, student resistance, creating a good climate, how to take care of yourself as an instructor when you teach this sort of stuff. But, those are some of the basic ideas.

John: A few years ago, with the election of Obama, there was some people who claimed that we had moved to a post-racial society. I think evidence since then has shown that that hasn’t quite been the case.

Cyndi: Yeah.

John: And I think the book is particularly well timed because these issues are in the forefront with the news all the time. How do you begin to address issues of race in your classes?

Cyndi: First of all, I think that idea of the post-race thing is really interesting, and I see it a lot in students. I mean, I’m just finishing up teaching this class now. We’re in our last week of classes now, it’s finals next week. And when most of my students, most of whom are white, came to the class a lot of them just have this colorblind idea, which is similar to the idea of post-race, like we’re done…sorted that out in the 60s, it’s all good. And it’s obviously not…and so they believe that we’re in this equal playing field, which we’re not really in obviously. And so that’s kind of a starting point. I talk about that in the book, this colorblind ideology that most Americans share. The first order of business is sort of getting through that. And so there’s, I think, two main things that most students and most people (especially white people) need to understand. One is that colorblindness isn’t really possible, even though we think it should be the norm, it’s not really the ideal. So that’s one thing, but then also that there is this larger structure of what people in my field would call institutional racism or structural racism. And that’s the piece that I think most white people, most students don’t really get: that racism is not as people said, individual acts of meanness, it’s also these bigger things that affect us that we don’t think about. That’s usually where I start. We talked about what race is and what it’s not, what institutional racism is, and what it’s not. So I think that mostly answers your question as to where we start.

John: And that feeling of colorblindness is that more unique to white students, perhaps than students of color?

Cyndi: I think it’s more unique, but it’s not exclusive to white students. Students of color can often struggle with that understanding of institutional racism, and structural and cultural racism, as well. If you look at attitude surveys, it’s not unusual for people of color to say that they don’t necessarily see it in institutional or structural terms. Or you’ll see surveys, they’ll ask, “What’s more important? Individual behavior or institutional laws and policies?” And almost all Americans with the exception of really recent immigrants and Native Americans, I think, say that the individual behavior is more important. And as a social psychologist, I would say, actually they’re both important. But as far as what impacts your life more, it’s those big, broad institutional, cultural stuff. So I think white students are more likely, but not only.

John: How do you make students more aware of those issues? How can you help get them past that notion of color blindness?

Cyndi: One answer is a lot of evidence, but it’s how you deliver that evidence. My usual way to try to get these things across is to combine a lot of statistical evidence, a lot of broad evidence, with stories and examples that are representative. So I try really hard in my content, like I don’t just cover a bunch of psychology experiments, and I don’t just cover statistics. I try to have that together with individual stories of people’s experiences. And I also think discussion is really key. So I don’t lecture in this course, really much at all. I’ll do some mini-lectures. But, that’s never the main thing that I’m doing because I think it’s really important for them to read, and then come to class and process all that stuff. Because the number one thing that happens, again going back to the misconceptions they come in with, is that they realize that there’s all this stuff that they didn’t know. So we cover a lot of history, for example, and there’s all this history of how we got to the racial categories that we have now that they’re just like, “No one told me this.” And they need to hear other students say that too. That’s part of also creating that climate is like, “Oh, I’m not weird or stupid for not knowing this. All these other people didn’t know it either.” And we talked a lot about, “It makes sense that you wouldn’t know because we don’t really teach it in our K-12 system very well for most students.” So, I think it’s a bunch of things. I think it’s what the content looks like, it’s how the class is structured, it’s how the evidence is presented. I think all those things matter.

Rebecca: Many faculty members avoid talking about race, especially in classes that are not about race specifically.

Cyndi: Yeah.

Rebecca: So can you address maybe why faculty do that, and how to help faculty overcome that fear?

Cyndi: I think fear is the main reason. And there’s different types of fear. One is: I don’t want to be the bad guy and I don’t want to be confrontational, which is understandable. Many years ago, I was talking to a friend of mine who taught in our English department, and she was teaching something called ethnic film and literature. And at the time, I was coordinating ethnic studies, and I really wanted her to teach that class again so that I could get it back into the rotation. And she just told me…we were at a party and she said, “I’m not teaching that anymore.” I said, “Why?” And she said, “Because I have to fight with them about whether or not racism is a real thing. And I don’t want to do that anymore.” She didn’t want to put up with the resistance, essentially. And she didn’t want to have to be what she felt like was the bad guy to deal with that resistance. And so I think that’s a big piece of it. If I try to tell students the truth about this stuff, they’re just going to resist and then I’m going to have to deal with that discomfort. And that’s a real fear, particularly for instructors of color. I mean, they’re all these national examples, right? …of people getting called out by their universities for basically just telling the truth in their classes and trying to teach institutional racism. The most famous example was Shannon Gibney over at Minneapolis Community and Technical College near where I live, and she was officially reprimanded by her University, basically for teaching what I teach, because she was getting pushback from white students, essentially. So I think that’s part of it. Also, it creates a lot of dissonance in students which was related to the resistance, so knowing how to deal with that dissonance can help students feel okay about themselves even as they recognize that they hold a lot of these beliefs and they haven’t really been very critical about it. So I think that all those things, all those types of fears play into why you don’t see people covering it.

Rebecca: How do you suggest maybe faculty get over that or feel prepared for that resistance or can actually deal with that in the classroom and not feel shocked or distressed or overwhelmed.

Cyndi: The big key, I think, is being prepared and feeling like you know how to talk about it. I’ve also heard a lot of instructors say, “I don’t feel like I have enough knowledge.” For white instructors, they feel like they don’t have the right or the credibility to talk about it. So that’s sort of an issue, and for instructors of color, there’s a whole other set of things. If you look at the research for them, there’s just a ton of microaggressions that they often have to deal with. They’re also seen as not being credible purveyors of this information. So that’s an issue. So I think just a couple things: one being as prepared as possible. So knowing your subject really well…being clear that when you teach the class, you’re gonna be clear with the students like, this is the evidence we’re going to use. So you’re not coming at it, like it’s all people’s personal experiences or opinions. That I think is where it gets especially hard. But if you know that, you’re going to come back to this scholarly base of evidence that usually makes most of us more comfortable, because that’s how we roll. …and people had this misunderstanding of teaching about race: “Well, it’s all just opinion.” I once had a student say, “How could you possibly give us a test on this? I mean, it’s just all people’s opinions.” I was like, “No, not quite. [LAUGHTER] There’s definitely evidence there. There’s psychology, there’s sociology, and there’s history and we’re going to use all of those things.” So I think that part of it is knowing that you have this common base of information. And also knowing what to expect…how to deal with the resistance, knowing what the resistance looks like, I have a whole chapter on what resistance looks like and how it manifests in white students versus students of color and how to think about it and how to deal with it. So I think that can be helpful too.

Rebecca: Can you talk a little bit about maybe one or two ways that we tend to see resistance and a couple of strategies that we might be able to use to overcome that?

Cyndi: Yeah, I think, in general, what you see with the research is that white students tend to be more resistant than students of color in general. They tend to be more vocal about it than students of color. One sort of broad distinction you sometimes see is that students of color are more likely to leave if they’re the minority in the classroom. So, if you have mostly white students (which is a lot of what I deal with, and maybe I have a few students of color), you might get more passive resistance on the part of the students of color, they sort of withdraw a little bit more, because they don’t want to be the one student saying things in the class and then being really looked at or targeted in that way. And for white students, they just tend to be more comfortable speaking up about this, particularly in the primarily white environments. So I think the ways to get around resistance are: 1. always sort of coming back to the evidence that you’re using. And if you set the table at the beginning of the class that can usually work so you can come back to: “Okay, here’s what we’re going to focus on.” So for example, I’ve had students say in class, when we talk about wealth disparities or something, they might bring up a particular person that they know who doesn’t fit that. And so what I found is useful as I don’t fight with them about whether or not their story is true. I don’t know the wealth of this individual family of color. So if I tell you that the wealth disparity is like 10 times… white families have on average 10 times the wealth that black families have…and they say, “No, no, there’s this one family I know. And they’re, they’re really, really rich…,” you know, as sort of an anecdotal thing. That’s a way you see resistance manifest, right? And so instead of arguing back and forth about whether or not that family is rich, which is useless, you can just undercut that and talk about the general wealth disparity. So, that’s one way to do it. That’s obviously pretty mild resistance. If you have consistently students questioning evidence, which doesn’t happen as much as you might think, bt it can happen, but it can happen, then I sometimes will take that out of class. For students of color, the way I try to work with the resistance you often see there, which is that sort of passive withdrawal because again, they don’t want to be targeted within a predominantly white classroom, one technique I use is to send an email really early in the semester that says something along the lines of, “We’re going to be talking about racism all semester. Your classmates may look to you to be a representative, just know that I know that you don’t need to do that. That’s not your job. And I’m not going to expect that of you, because I want to try to make those students especially feel as safe as possible. And I also recognize too, I reinforce to them: “I’m a scholarly expert on this. But this is your lived experience and I recognize that. Not saying that one is better than the other, but just I see you and I see that your experience is going to be different than the white students in the class. And so that’s the way I try to short circuit that withdrawal from class as much as possible. And most students, at least so far, have appreciated that…and also just acknowledging way up front that this is going to be new to you. You’re going to be uncomfortable, I put it in the syllabus: “You’re going to feel uncomfortable, this is not comfortable stuff to talk about, and so just know that it’s coming.” So those are a few things.

John: Do you recommend having the class come up with rules of engagement or discussion on these issues.

Cyndi: I definitely do. I mean, I have some that I think are important, but I let them drive that discussion. And then I add them in if I feel like maybe they’ve missed them, or something. And I usually have them. Actually, I think I learned this from an earlier version of your podcasts, someone was on talking about having the students working groups to develop their rules of engagement. And so I did a little bit of that on the first day this semester, let them talk about it in small groups before we talked about it in large groups. And then I just take a picture of what those agreed upon discussion groups are and I post them up on Canvas, so that that way they can come back to them. Yeah, we have a whole discussion about discussion: about what it should look like in our class, how we want to engage. One big question we always talk about is do you want to raise your hands or just talk? How do you want to handle somebody upset? They always think it’s going to be more contentious than it actually is, at least so far in my experience. I haven’t had a whole bunch of anger or confrontation. So that’s been so far that’s worked well.

Rebecca: What are some of the consequences of not dealing with race specifically in classes or subject matter that doesn’t directly indicate that race is going to be a part of the conversation? So we often hear this from faculty in math or science, or I would even say in my area of design where it’s not obvious that race might be something that should be discussed. What are the consequences of completely avoiding it?

Cyndi: Well, obviously it marginalizes it. So, it turns it into something that only some people can do. Only some people can cover these topics, only some people are allowed to on some level. And it’s interesting you ask that because this has come up on my campus in the last couple of years in a couple of different ways. Because we have, I don’t know if you all have it, a diversity requirement, but we do. So we have one course, American Cultural Diversity, that students are supposed to take. And there’s been sort of a…fight is a strong word, but…argument over who should be teaching those courses. And I don’t think just anyone should be teaching them. But I do think people can develop an expertise within their own field. So for example, I have a colleague (and we have a very big agriculture college on my campus), and she teaches soil science and crop production and all that sort of thing. So that’s an area where you think, where’s the race going to be that but it’s deeply in it because she’s teaching these future farmers and folks working in that industry. And there’s so many immigrants that work in farming. So she developed and she developed the expertise on this. And she developed a class on immigration…migrant farm workers, essentially. She and I talked about that course many times when she was in the process of developing and starting to teach it. She teaches it regularly. And I think the advantage of that is that, for the students who are majoring in that college, that’s going to feel very relevant for them. And so again, I think people have to develop some expertise to do it, and then they need some tools for how to teach it. But we all need tools for how to teach it because it’s hard. So, I think it’s important because it allows people to be more motivated to see how it relates to their specific field. I also have a colleague in math, who has talked a lot about trying to develop some courses around race and whiteness and math. You may recall, there was a math professor, I think, in Illinois, who was writing about white privilege and math and got a lot of haters online, essentially. Basically a really good scholarly analysis about the way white privilege works in math. And that stuff is really useful for students. It helps them connect in ways that are relevant to them.

John: You mentioned using narrative. While evidence is useful, it doesn’t always reach all of the students. I’ve been teaching about discrimination in my labor economics classes for decades. And it’s remarkable how resistant people are to facts. Because they rely on the sort of narratives: “My third cousin twice removed is this exceptional case. And it means all this evidence is wrong.” But you mentioned using narrative perhaps in a constructive way to help students understand that. Could you give us some examples of that?

Cyndi: Yeah, sure. Like the example that pops to mind right away was a really powerful piece that I used this semester on black maternal death rates, you may know that they’re wildly disparate, right? There’s a huge health disparities when it comes to race and healthcare. And so there was a really nice piece that was actually in the New York Times Magazine. That was this really good combination of a personal story of one woman and her experience. She lost a baby and then she was pregnant again and dealing with that… and there have been all these famous cases. Serena Williams most famously talked about her complications during birth and not being believed by doctors. So this piece was really great because they both had her story but then woven into it, it was a really well written piece. It had all the statistics, the differences…And the students…I gave it to them over a two-day period, because it was pretty long, and we discussed it for two days. And I just used it as this larger example of implicit bias in the healthcare industry. And these larger statistics on the disparities, the wealth gap…it illustrated a bunch of things that we had already covered. And the students loved it. It was hard for them to read, but they were very engaged with it. And some of the questions I got were excellent. And I’ll just say, too, I think one important thing is that when you come into a discussion, it’s really important to have a base to discuss with. And so, every day they have to read and post questions. I don’t give like reading quizzes, but they have to post questions so that I know that they read or at least mostly read, and I grade them. Because if you don’t do that, then your discussion is not good at all. If people don’t have a base of evidence to work from, you’re not going to get anywhere. And so those types of materials…I’m always looking for them…where they have that combo of the broad statistics and also the individual stories…and yeah, I know economics, it’s really tough to get those pieces. It’s the same for psychology can be really hard to find, but they’re useful.

John: Well, certainly in immigration and discrimination in labor markets, there’s a lot of examples out there and lots of good content.

Cyndi: Yeah, there definitely is. You just got to always be on the lookout.

Rebecca: So, what happens when it’s a conversation where the discussion point wasn’t going to be race, but then it becomes race. There isn’t a piece that you’re going to discuss ahead of time. But it pops up in conversation, it needs to be addressed. Do you have any strategies for handling those more impromptu situations that occur?

Cyndi: Well, for me, I think I always go back to then, either being honest that I don’t know enough to comment on it. What that question makes me think of are when students will bring up current examples, either that I don’t know or that I don’t know enough about. And so I will always try if I can to find some relevant psychological data or evidence or sociological data or evidence that I can bring it back to. And there have been times when if current events are happening, and I feel like I don’t know enough, I’ll just say, I don’t think we know enough. So if we have time, maybe we all spend a few minutes on our phones trying to find some information and you can do that right in class sometimes. But I always see myself as sort of a guide in terms of helping them sort through what the larger patterns are in terms of that evidence. And I think if you create a decent enough climate where they trust you, that usually can work. But yeah, the impromptu stuff can be tough, particularly if it’s not your area. I think that’s where, again, being prepared is hard. And I think you can just say, if this isn’t your area, like, “Let me go and find some more. Let me go find some stuff out and then we’ll come back to it.”

John: On dealing with things in the moment, how would you recommend people respond if someone makes a comment that somewhat racist without realizing the impact of that?

Cyndi: A really good example of that happened actually in a colleague’s classroom on campus this semester, and he called me after it. I forget what the term was, but a student had used a term in class that he found offensive and he was pretty sure other students found offensive and he didn’t really deal with it in the moment. And so he called me and asked what I thought. And I think one of the strongest ways to deal with that is, oftentimes, students don’t realize or people don’t realize that a turn might be offensive, or it might seem offensive. And so a lot of times, and what I advised him (and he said, it worked pretty well) is to go into those conversations, discussing it in terms of kind of a growth mindset idea. You know, here’s this term that was used…it’s offensive for some folks, but some folks might not understand why and then maybe talking about why that is. The word “colored” gets used a lot… “colored people.” And it makes sense in some ways that white students now who are very young wouldn’t know that that was an offensive term in the 50s and 60s. And so they use that term because they hear people say, people of color and then so colored people seems like a normal permutation of that, right? But it’s really not. It has this very unique history and so you could talk about how “Here’s this history that you may not have understood. And here’s this term that people didn’t use to describe themselves, it was used about them. And so that’s part of what makes it offensive.” And it’s normal that language changes and it evolves. And there’s plenty of examples you can bring up around that, like we talked in my classes about the word queer and the way that shifted over time…and language evolves…and so just sort of accepting that you’re going to make mistakes, you’re probably going to say things that might be offensive, but what’s the mindset that you bring to that? Do you bring the mindset that it’s normal, and you’ll figure it out, and you have to make your classroom safe for that too. So that it’s not like people are being called out and told that they’re saying the wrong word. We talked a lot in class about the difference between willful ignorance, like, “I know it’s wrong, but I’m gonna say it anyway” and just ignorance…like just really not knowing and coming out that and I give examples of my own, like times I screwed up…things I’ve said that were wrong, as a way to help them see that you’re never finished. I’ve been doing this for like 20 years, and I still make mistakes. I think that helps.

John: When you’re setting the classroom discussion rules, would that be a good time to bring that up?

Cyndi: Yeah, I think so. I think that you could, you could talk about expecting people to make mistakes. In the rule setting phase, you can talk about not expecting perfection, and how people will make mistakes and that’s all right, and ways to sort of come back from that, and gently talk about it rather than calling people out.

Rebecca: How do you handle microaggressions or other behaviors that might happen in class, that aren’t just like a word or whatever, but it’s something that’s happening or you see a pattern of behavior with a particular student. And maybe it’s something that you feel like you need to handle one on one. How do you usually handle those kinds of conversations?

Cyndi: Dealing with those. It’s usually much better one on one, because again, just like anybody, if you if you were to call someone out in class, then you’re likely to just get defensiveness and nobody’s going to be able to hear it. So what I’ve done in the past has been to talk to students one on one rather than to frame it as “You’re a bad person for doing this,” it’s like, “This is what I’m seeing. This is the pattern that I’m seeing. This is how I think it could be perceived” …and then just listening to what you hear. And you have to have a fair amount of trust with a student to be able to do that. But in general, I think whenever it comes to talking about someone’s racist behavior, it’s always better to focus on the behavior rather than the person. This is why when you hear national conversations about is that person a racist, I always want to throw my radio or my phone or whatever because it’s so frustrating to hear it framed in that way. “Is someone a racist?” is not a useful question. And I never quite sure what that means, because the goalposts always move, in terms of like how we think about what that word means. So, instead, focusing on the behavior, this is the pattern I’m seeing this is a problem. And I think if you’re in a moment where (this would be less with students, but more with colleagues) where you’re seeing this happen, and it’s directed at a person who has a lot less power in that moment… so like, a person of color, for example…you could step in and say, “This is what I’m seeing, and this is how I think about it.” So you’re not putting it on the person who was maybe the target of it. But you are saying in that moment, I see this and I see that this is a problem. That can be harder to do and less with students, I think more with colleagues. But, in general, it’s just sort of noting that it’s happening and being honest about it without necessarily saying you’re a bad person for doing this.

John: Last fall, we had a reading group addressing some of these issues. And one of the issues that came up in a lot of discussions is how to address these issues with colleagues, particularly those who are evaluating you for retention, promotion, and similar issues…

Rebecca: or hiring…

Cyndi: That is so hard when someone is in a position of power. Because if you’re the job candidate, there’s just no way that you’re going to be able, in that moment, to be able to do that.

John: What if you’re a junior faculty member on, for example, a recruitment committee and you observe comments or behavior that seems to be biased in some way. What would you suggest to a faculty member in that position?

Cyndi: I think you could go back to the sort of something called micro resistance. And there’s been a little bit written about this. In terms of how to deal with it, again, not making it about the person but just saying like, this is what I’m hearing, this is what I’m seeing. This is how I feel about it. And so you make it more about yourself. In extreme situations, and I’ve certainly been in them and seen them, you could go to other people that you trust on the committee and say, “This is what I’m seeing. This is what I’m hearing.” This is slightly different, but I had a slightly different but I had a student come to me last week and say that she’s in another course. And she’s hearing this from an instructor. And so then I was able to go to that department chair and say, what’s happening? So, I think using your mentors, using your colleagues, if you’re in that lower-power position; and if you’re in a higher-power position in those same spaces, try not to make the target responsible for that. If you’re a man, and you’re seeing sexism, it’s useful to just call that out. And again, not calling the person out, but just saying, “This is what I’m seeing. This is the pattern.” We talked about this actually, there’s an interesting anecdote in my class this last week. We were talking about this micro resistance thing and one of the students is a softball player and she’s on the softball team. According to her, there’s one black softball player on the team and everybody else is white. And, according to my student, whenever racist things will come up, like, they’ll all look at the black student to ask her “Is this okay?” And we talked in class about like, maybe that’s not fair to put that on the student of color. This white student feels like she really wants to be an ally, like she really wants to be an advocate. So we talked about, well, maybe you just say what you think about it, rather than asking her “Is it okay?” or going to her afterwards and saying, “Do you feel okay about this?” Because what is she going to say in that moment? I mean, she’s in the minority…the black student is, and so I think that can be a useful way to think about it too, because a lot of times we want the person who’s lower in power to like, excuse it and make it okay. And that’s really not fair. And I think it happens just because people don’t think about the power dynamics at all. They just don’t think about it. It doesn’t occur to them. And so trying to be more intentional about what is the power in this situation and trying to be more fair.

John: One of the issues if there’s a small number of minorities in a class, one potential issue might be stereotype threat. What are the consequences of that? And how can we address that perhaps by making it a more supportive environment?

Cyndi: Yeah, stereotype threat is really interesting. I know a lot about this. Actually, I’ve given lots of workshops on this, in addition to like implicit bias and stuff, too. And it’s a real problem. The consequences are…they’re sort of short term and long term. So the short-term consequences of stereotype threat is that you have students who underperform. So in a test situation or on a writing assignment, where you have a student who is feeling stereotype threat as a result of race or gender or social class. And so then it just create that extra layer of anxiety and stress, essentially. And it’s not always apparent. And you don’t necessarily know that that’s what you’re experiencing. But we know from the neuroscience research that, you just have less working memory in those moments because of stereotype threat. And so the short-term threat is that you underperform. The long-term consequence is that students disengage from the area altogether. So this is why we hear.…I’ve heard it so many times from my female advisees…“I’m not a math person. I’m not a science person. And I think it happens in art as well. I don’t know about design specifically…

Rebecca: um hmm.

Cyndi: …but you’ll get like, I’m not an art person. I’m not creative.

Rebecca: I can’t draw.

CUNDI: I can’t draw, yeah, that’s it. That one’s, I think, less about race, maybe a little bit more about gender, but it’s a very similar thing of like, “I don’t feel like I can do this. I’m not creative.” And so I’m just going to withdraw from it altogether. And so you see what Claude Steele calls dis-identification. So I’m just going to dis-identify with that field. It’s just not my thing. I’m going to go get my self esteem somewhere else. And obviously, that has serious consequences if the thing you’re dis-identifying with is school altogether. And so that’s why we see this underperformance over time with students of color and with women in math and science. The ways to get around that…there are a few. There’s a whole set of interventions that social psychologists have developed that can be really powerful. I guess I would send listeners to the mindset network web page. I don’t know if y’all have ever seen that. It’s mindsetscholarsnetwork.org. But it’s a bunch of social psychologists who have gotten together to create these really pretty low-cost interventions around increasing belonging…using values affirmations… Utility value is another one…growth mindset. There’s a bunch of them and there’s a little tweaks that you can do in your classes to help that. The other big intervention, and you can sort of call that active pedagogy. So there’s really good research that the more active your class is, that’s going to be good for everybody. But, it’s especially good for your students of color your first-generation students and your students who are women in math and science and engineering courses, where they’re more likely to feel that thread. That as an intervention itself is really great. There’s a ton of really nice discussions of that, and studies of that that you can find in terms of active pedagogy being an inclusive pedagogy. Because, in general, you want students to have a sense of belonging and you want them to feel included, and that’s going to help to undercut that, because really all stereotype threat is about is about a lack of trust. So everybody thinks it’s a lack of confidence in the student. It’s not. They don’t trust the environment to be fair, and so that’s why they disengage and they pull back. And so you want to you want to do everything you can to keep that trust.

John: …and they build more of a sense of community with their fellow students.

Cyndi: Absolutely.

John: I’m going to our conference in a few weeks, and one of the activities there is something called “sip and paint.” A friend of mine tried to convince me to do that. And my reaction was “No, the last time I painted I think I was seven years old.” [LAUGHTER] So, there’s a gender issue perhaps with the artwork thing.

Cyndi: Creativity. Yeah.

John: You mentioned implicit bias. My labor classes are online and one of the things I do is I have them take some of the Implicit Association tests, and then discuss them. And they tend to be pretty comfortable discussing many of them, but they tend to be much less comfortable discussing race.

Cyndi: Oh yeah.

John: But one of the things that led to some really good discussions are the associations between gender and careers.

Cyndi: Yeah.

John: And a lot of female students remark on how surprised they are that they associate women with home activities and men with careers. But, one of the things I note from the students who tend to perhaps have the more resistant attitudes towards facts in general, from other discussions, is that they tend to question the tests themselves and say, it’s clearly set up to demonstrate a bias when that bias really doesn’t exist. And those students are really hard to reach and we can keep giving them facts. But I’ve never been completely successful in getting through that barrier, at least in any one course. Any suggestions?

Cyndi: It’s really tough. As a social psychologist, I feel pretty comfortable talking about the Implicit Association test, but it is really hard to describe well, so that’s one problem with it, because you try to explain “No, no, like 25 years of research…” When I still had paper versions of the literature, I gotta bring in my big giant folder and I just sort of slap it on the desk and be like, “They’ve been studying this since 1995. But, like you said, the facts don’t always help. One thing I think that helps with them understanding implicit associations, is to depersonalize i… and I have some great podcast and book suggestions and article suggestions on how to help them understand what implicit associations are. But really, it’s not about them as a bad person. And that is one way I found to get at it. There’s a phrase that gets used by Mahzarin Banaji, who was one of the test co-creators and she talks about implicit associations as the thumbprint of the culture, which is really accurate, you know. So it’s not you’re a bad person, you have implicit bias… like, we all have it and it’s the thumbprint of the culture. You’ve been learning since you were a baby, what’s associated? what’s good and what’s bad? I mean, it really is that crude. It is your brain saying, “This group is bad. This group is good” over and over and over again, you get those messages. So if you can de-personalize it, I think that can help a lot. I have found that using the podcasts that I have on it, and some of the more newsy articles and they cite the researchers, that can be really helpful, too. But yeah, it’s they want to criticize the test all day long. I’ve gotten to where I don’t have them take the test until after they have a decent grounding in the science because they’re very resistant to the idea. They think the test just sucks.

John: At least those who have their preconceptions not confirmed in the way they’d like them to.

Cyndi: Yeah, because again, they think this means I’m a bad person. They think it’s the racism test. There’s a King of the Hill episode. I don’t know if y’all have ever seen that show, but I used to love that show. And there’s an episode where Hank has to go take the racism test, because he’s worried that his dog is racist or something. I can’t remember the full thing of the story. But, that episode is one of my favorites because it’s like, “Okay, let’s see if he’s racist.” But, that’s not the way it works, folks. I’ll have students sometimes say like, “We should just have all cops and all teachers and all judges take this test. And then we’d know who to hire…” and I’m like, “There’d be nobody left. There wouldn’t be enough people left to do all these jobs.” And I think if you talk about it in that way, it can make it so that it’s not a moral failing, which is, I think, why they’re so resistant.

Rebecca: I’ve done something as a follow up to doing some of the tests in my classes where I had students look at their portfolio of design work, and just see who was represented in the materials that they made. And what they usually do is discover that either it’s a lot of people that are just like them, or that it’s white and young….

Cyndi: Yeah.

Rebecca: …which some of the people in the class may not fit that particular group, but that’s what they’ve still represented. And that helps a lot, because we talk about, “Well, it’s easier to design for a group of people that you’re around all the time, perhaps”

Cyndi: Yeah.

Rebecca: Or, You know, know what, like this particular population, maybe preferences, if that’s a group that you’re a member of. And that sometimes helps too because it kind of breaks down some of the total ownership or blaming a student for something. It becomes more of that cultural identity piece.

Cyndi: Yeah, you can ask them, like, “Who’s most of your friends? Who’s in your environment?” I have them write journal entries all semester. I don’t say “Go find the racism and tell me about it” I just say like, ”Just tell me what you observe in terms of both race and gender.” Just like “What do you see? Who’s doing what jobs? Who’s in what space?” and that helps them too to start to see the stuff that they just sort of take for granted, because it’s the water that we all swim in. We’re all very segregated. And so I think it’s good for students to recognize that and then how that plays itself out in who you select to design, for example, and who comes into your consciousness. So again, thumbprint of the culture rather than moral failing…bad person.

Rebecca: I also do an activity in my capstone class where I ask students like, who are there five designers that inspire them, and then I end up with a pretty small list when we aggregate all of them together. And then I say, I’m going to ask this question again later in the semester, and I expect these lists to be really different. [LAUGHTER]

Cyndi: Yeah, that’s good…makes them explicitly think about it.

John: We always end with the question, what are you doing next?

Cyndi: I want to write more about these issues. What I really want to do is run some workshops for faculty. I’ve done a couple. Most of my workshops have been on stereotype threat and
implicit bias. So I would like to run more workshops on this topic in particular, like how do you teach about racism rather than teaching inclusively. That’s fun to talk about too. But how do you how do you teach about race and racism? I would love to do more of that. And I would also like to write more about these issues. Because I think it’s hard to do. And so I would like to just have more conversation. I’m also hoping eventually to maybe write a different book about inclusive pedagogy. We’ll see. I’m not sure. it’s a ways off.

John: And when is your book coming out?

Cyndi: It’s supposed to be November, I believe, November or December.

Rebecca: Well, I know that will probably have a line of people now that really want to make sure they get their hands on your book, because…

Cyndi: I hope so.

Rebecca: …there’s a lot of books that deal with these issues conceptually, but not in a practical way.

Cyndi: I could not find a lot on teaching about it. Like I said, I wanted the guide that I wish I had for myself 20 years ago, but I there’s just there’s not a ton. There’s a lot of good chapters on it in some edited books, but there wasn’t a lot that had sort of an overarching idea. So that’s what I wanted to try to do.

John: We were looking for that just last year. So, we will have it on pre-order very soon.

Rebecca: Yeah, Definitely.

Cyndi: Cool. Thanks.

Rebecca: Well, thank you so much for joining us. This was really great.

Cyndi: Yeah, thanks so much for asking.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

86. Attention Matters

Our smartphones, smart watches, and other mobile devices provide us with a growing number of convenient distractions that can interfere with our productivity and learning. In this episode, Dr. Michelle Miller joins us to discuss one approach to help students better understand how to focus their attention.

Michelle is the Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and the President’s Distinguished Teaching Fellow at Northern Arizona University. Her research interests include memory, attention, and student success in the early college career. She co-curated the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.

Show Notes

Transcript

John: Our smartphones, smart watches, and other mobile devices provide us with a growing number of convenient distractions that can interfere with our productivity and learning. In this episode, we examine one approach to help students better understand how to focus their attention.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Michelle Miller. Michelle is the Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and the President’s Distinguished Teaching Fellow at Northern Arizona University. Her research interests include memory, attention, and student success in the early college career. She co-curated the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She’s the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications. Welcome back, Michelle.

Michelle: Hi, it’s so great to be here.

John: We’re happy to talk to you again. Our teas today are:

Michelle: Well, it is still technically morning here in Arizona where I’m speaking from so I am going with home-brewed coffee today with a whole lot of sugar and a little bit of cream.

Rebecca: I have an Orange Cylon tea today.

John: Cylon? Weren’t they on Battlestar Galactica?

Rebecca: How do you say it?

John: I believe it’s Ceylon, and I have Ginger Peach Black tea.

Rebecca: So we invited you here to talk about your Attention Matters project, Michelle. Could you tell us a little bit about the project?

Michelle: This is one of my favorite projects and really the most unique one that I’ve worked on, really going back to as long as I’ve been at Northern Arizona University. Attention Matters came about like this. After I wrapped up writing Minds Online: Teaching Effectively with Technology, I was really just still so engaged in this interest in disseminating cognitive psychology and cognitive science, and pulling out those key principles that we can all use in our lives and can make human wellbeing better and have all these great applications, and especially applications in teaching and learning environments. So you can probably guess that it has to do with attention and attentional processes in the mind and in the brain and how those play out in some different situations, especially with teaching and learning. Basically, the idea is that we do need to raise student awareness—and really raise all of our awareness—about how things like digital distraction—the smartphones, the alerts, the temptations that are always there when we’re online—all those can impact teaching and learning and things like memory as well. I had been really interested in finding ways to get that stuff out to students. So there were some people at my institution (at NAU), who knew that I was interested in that and that I really liked talking to students really directly about what they knew about attention, how things like attention and memory really work, addressing some myths and misconceptions that people tend to have about their own attention. So, I was interested in getting that out to students and faculty who knew that I did that would come to me and say, “Hey, can you come and talk to my students about this?” And I had kind of put together this PowerPoint presentation, and really interactive stuff where I could pull down some video clips and interactive demonstrations off the web and we could actually really get a discussion rolling about this. And so that was my idea. I was kind of going around with my little traveling show from class to class to get this out to students. And it’s obviously not terribly scalable for me to do that. I mean, how many students can I really reach that way? But I still really like doing it. So I was talking about some of these ideas with colleagues at one of our teaching day events at NAU—a lot of institutions have days like these and they’re just wonderful. They’re days when projects like this really get started—and I was talking to one of my most dynamic and really respected colleagues that I have at NAU, and that’s John Doherty. He is a brilliant instructional designer, who now works with the library at our institution but has held these roles for years at NAU. And he said, “You know, have you thought about putting this online? Have you thought about finding a way to make this an accessible online resource so that instead of you having to take your show out to different classes every semester, they can drop in and complete some of these?” and we started talking about what would that look like? How would we do it? Let’s put this together. We had no funding, there was no official initiative behind this. We just jumped in and decided to do this and he looped in some other really generous colleagues, who contributed from our e-Learning Center, and we put this together. I guess we’ll be talking some more about exactly how it’s configured, but we created this project and we’ve kept it going to this day. So this exists as an online resource at Northern Arizona University. I’ve also shared it with dozens of other institutions around the country and a few around the world, and at least one has turned it into their own module that works in their learning management system. It did win an Effective Practice Award from the Online Learning Consortium in 2015—and we were really, really proud of that—and I’ve written about it in a couple of other articles and publications, including one that came out a few years ago in Inside Higher Ed that got some nice feedback. So that’s how it started.

John: So how long does it take for students to complete this module?

Michelle: Well, it’s a module that self-enrolls so they don’t have to pay, there’s no extensive signup time or anything like that. And it does take about one to two hours for students to work through what’s an online sequence of different activities, discussions that they posted, and other types of reflections and self-assessment. So it is something that works quite well as an extra credit resource and that’s exactly how it’s perpetuated in its current form at NAU. So, faculty can voluntarily opt to incorporate it as an extra credit activity in an existing class anywhere in the curriculum. It tends to be particularly popular with our STEM teachers who have these large science and foundational science and mathematics courses, but anybody can use it. And yeah, then students put in a few hours, and they get this experience that they’re really not going to get anywhere else in that class.

Rebecca: What do students find the most surprising about this module?

Michelle: Well, let’s see. You’re picking up on a theme that we really did try to work into the design of the module itself—that element of surprise—that we typically know a lot less about how our attention works than we think. We take in a lot less than we assume or believe that we are. And so time and again, that’s exactly what we asked students. “What surprised you about this demonstration?” Students are surprised at how much can get past them when they are looking at, say, one aspect of a visual scene, but not paying attention to the whole thing. And you’re going to say that I’m going to be kind of weaseling out and talking around some of these issues. Some of these demonstrations do depend on the element of surprise. So I don’t want to talk too much about exactly what students are looking for in these different displays. Some of our more key demonstrations in this module have to do with some phenomena that most people, even if they don’t know the technical term, they’ve seen something about it before. So one is the change blindness effect, and this has to do with the fact that when we are paying attention to one aspect of a visual scene—like we’re looking at what somebody’s doing with their hands, or we’re looking at their car and not something else in the background of a street scene—we remember very little from moment to moment. And that’s why if somebody is, say, momentarily distracted or they just are really focused on one part of it, they can miss these huge visual changes. And then with multimedia, we can rewind and show you, “Oh, hey, here’s what you missed.” …and, it’s just shocking. And it’s something that is very robust in fact. You don’t have to have these perfectly controlled lab conditions, it happens pretty frequently, very reliable, so this makes a very good demonstration of attention and how attention works. We take a lot of these change blindness effects and those are very surprising. So students time and again will say, “I was surprised that I missed X, Y, and Z out of this thing that I saw.” And when they rewound it, “I just couldn’t believe that it got past me.”

Rebecca: Seems like this is a technique that pickpockets use pretty frequently. [LAUGHTER]

Michelle: Yes it is. [LAUGHTER] …and I was actually pickpocketed when I was abroad last year too. It’s something that can happen in a real scene, it happens visually. We are really subject to attention. And actually, if you’d like to, for people who would like to read a lot more—would like to do a deeper dive on this—there’s actually some really fascinating work on how stage magicians use this as well. So this is something that stage magicians have really known about in a different kind of arena for many, many years and is a very key aspect of what they do. So if you’re familiar with the concept of misdirection, it ties into that. So this is something very close to my heart because a former colleague from NAU named Anthony Barnhart is himself a very skilled stage magician and he is also a PhD who works in visual attention, so a lot of his work plays those out. So exactly, it’s something that we can all really get excited about. There’s these neat connections in so many different areas that are very, very relatable. So this is part of the fun of bringing attention and attention research out into this area to our students.

John: I want to put in a plug for another podcast actually, that came out on April 25 from the Teaching in Higher Ed podcast. There was a discussion on that, where Bonni Stachowiak was talking about something she used to demonstrate—so it won’t be one of the demonstrations you used—where she played some audio to students of a recording where there were three separate conversations going on, and she asked them to interpret it, and they couldn’t recall much of anything. But then she divided them up and she has some of them to focus on one conversation, others to focus on the other, and others to focus on the third, and then they were able to reproduce pretty much everything. But there was that issue of being able to focus on particular things and that focused attention, I think, is an important component to that. It’s not visual imagery, but it’s still, I think, the same sort of processing.

Michelle: Yes, absolutely. I mean, this is the real fun of teaching something like an attention module—which I get to do since I’m a psychology professor—because there are these neat interactive demonstrations. So those are the types of things that besides just the multimedia, and these neat demonstrations and things like change blindness that are out there on the web, I was able to kind of dive into my repertoire of those demonstrations of trying to focus when you’re distracted and how different competing inputs can take over and that affects your memory of which is interpretation, all these things I was able to pull out of my teaching repertoire and think about how could these be put online into this really massive course that students can come into, and can you make that work? And using those to—instead of trying to teach theoretical concepts about attention—to raise students awareness of an investment of that idea of being good stewards of their own attentional resources. To realize that yeah, things like learning by osmosis don’t really happen… that we need focused attention in order to learn, and that we aren’t always aware in the moment of just how our focus is being divided. And once we have lost focus on something, it’s very, very hard—if not impossible—to recapture that and get that information back. So establishing those ideas, getting students aware of and invested in them and getting us thinking about, “Alright, so how does this relate to very practical questions like you’ve got a cell phone and you’re in a class of 200 students, and you’ve got five friends texting you, what do you do? Or what if you’re sitting next to that neighbor in this large class, and they’ve got their laptop going, and it’s distracting? Or even low tech stuff like conversations.” That’s an issue that’s been around forever. So getting students kind of thinking ahead about that, instead of just kind of reactively—or worse depending on the teacher to tell them what to do in this situation. So those are some of the ways in which I was trying to weave together the teaching of some basic aspects of attention and cognition with, “Alright, you’re the student here. How are you going to handle this in your day-to-day life?”

Rebecca: How have your students who have completed the module change their behavior or used this information in a practical way?

Michelle: Well, just with so many of our teaching interventions that we do and our student initiatives that we do, I go to say it is very, very hard—not impossible—but that’s going to be a much harder and more long-term process of determining what changes in terms of actual behavior, the choices that students make. And I want to be real upfront here that in putting together and running this module, while we have done some basic empirical work on things like attitude change and knowledge change. I don’t know. I don’t know if students are more likely to turn off their cell phone and put it in their backpack, but I will say that what students say and do in the module itself is quite encouraging, quite eye opening. I mean, we asked them—and we could talk a little bit more about the design in maybe a minute—but we asked them at the end to tell us, “Alright, what is your plan going forward for managing your own attention? What’s your plan for having you manage your technology instead of your technology managing you?” And I am always surprised at the sophistication and the commitment that they expressed in these things, how they really personalized these concepts. So they will say, “From now on, I really am going to put my phone in airplane mode, or I’ll use the Do Not Disturb functions so that if that there’s a real emergency, my mom and dad need to talk to me, here’s what I’m going to do,” or a lot of what we exchange ideas on towards the end of the module when we say: “What’s your plan?” are, what’s kind of ironically, technology to manage your technology. So there are a range of other things that students can do—apps that are out there—that will do things like give an auto bounce back message when you’re in class, they’ll shut down certain problematic sites—that’s a favorite of mine because I definitely wouldn’t have any of this writing or anything to get this work done if I didn’t have a few checks on my own internet usage while I’m using my laptop—so you can make great concrete plans to do that as well. So those are some of the things that students say about their future behaviors assumption of this class. But yeah, if we look at something—I mean, I do hope perhaps in the Fall when I have a new student coming in—to look at things like longer term differences in say GPA, but those global measures, it can be very hard to discern the influence of something like that. But I do think that’s kind of the next step with this project.

John: In terms of the motivation for this, one of the things you’ve done is—I believe—you’ve looked at the relationship between the counterproductive belief scale and multitasking behavior in a convenience sample that you had worked with. Could you tell us a little bit about what you were looking at there and what you found?

Michelle: Yeah, and you know—just to give a sense of how those measures fit into all this—the part of what I’ve really enjoyed about this project as well has been the fact that it’s an opportunity to gather some data, both those qualitative impressions from students—which are so incredible to test and speaking in their own voices, their own experiences with digital distraction, that’s really neat—but we also have these very brief quantitative measures that we developed as part of the project and built in originally just as part of the assessment, but they have really become part of my research as well. So the counterproductive belief survey is a short 20 question set of items that breaks down in three big groups or subscales and they tap into what do people know and believe about their own attention and memory. So things like “quizzing yourself is a good way to remember information.” And, if we remember back to the podcast that we did together on retrieval practice, that’s such a bedrock idea about how attention to memory works, and it’s primarily an area of memory—but it relates to attention in that we kind of can contrast that with another item that says, “Oh, rereading things or skimming things in your notes is a good way to learn.” So that effortful, attentive processing is really important for memory and other things like really passes. Just seeing something go by doesn’t help you remember. So that’s the sort of thing that is tapped by that scale. So we look at what do they know about attention and memory? What do they know about attention itself like how limited it is, and so forth. And there’s actually another piece of it too that I’ve termed self-exceptionalism. The idea that, “Well, you know, other people might have a hard time texting and driving, but I’m really good, so I can do that,” or “I’m better than most people at dividing attention across different areas.” Oh, and I should say as well that this scale too was really inspired by some concepts in an absolutely fantastic book by Christopher Chabris and Dan Simon—two attention researchers—called The Invisible Gorilla: How Our Intuitions Deceive Us. So I went into that book and kind of pulled some of their core concepts and turned that into this counterproductive belief survey. So that’s what we can survey people on, the kind of, “What do you know?” Now the multitasking behaviors inventory was a self-report survey that looked at how often people said they multitasked with certain kinds of media or online types of things and how they did that in different day-to-day settings. So we could kind of cross those in a way so that we could ask, “How often do you do email that’s not relevant to the task at hand? And how often do you do this in classes if you take classes? How often do you do this at work? How often do you do this in social areas?” And then we can go through those same situations with other things like casual gaming or social media, or things like that. So we can also query people and get that sense of what they said about how frequently they found themselves doing this in day-to-day life. And in this survey that we ran with worldwide convenience online sample, we did find that there was a relationship between what people believed about attention and memory and how often they said they did these multitasking behaviors, which just establishing that yeah, beliefs relate to behaviors, at least self-reported behaviors. I think that was an important step. And we found that in particular, that correlation—that relationship—was strongest for the part of the counterproductive belief scale, specifically relating to attention. And that makes sense. So yeah, if I understand attention a little bit better, if I know “Oh, hey, this is limited and I’m not even going to know that I’m distracted until it’s too late,” then hey, I’m less likely to say, “Yeah, I sit in class and I do email because what could it possibly matter?” Right? So that’s something that we kind of uncovered and revealed as part of this project.

John: You mentioned the development process. Could you tell us a little bit about how this project was developed and how it’s evolved?

Michelle: In the design process for this—since it was just a project we put together spontaneously out of interest—we had a lot of opportunity to just go about it in a completely fresh new way. It wasn’t like being handed a course redesign project where new classes developed where you kind of have a rut that you’re already in. So with the design process—so I work really, really closely with John Doherty, who again, put in so much of the intellectual power behind this and made so many of the key design decisions and helped me with that. So we worked together to really talk about, “Okay, here’s sort of what we want. Here’s our philosophy,…” and then he—and again, with input from other instructional design colleagues—said, “Okay, how about this particular resource? How about this particular way of setting this up in the learning management system?” and they were just things that I would never have thought to come up with. So, with the philosophy—or grounding principles of what we wanted—we wanted to show don’t tell, we did not want it to be just this flat website that was like, “Okay, here’s a list of 20 things that I’m telling you not to do,” that we wanted students to have this experience of exploring this on their own, and have these surprising and interactive demonstrations. We did want to have those very selective, curated set of principles about attention that we wanted to convey and we didn’t want it to be academic—this is not a psychology course, not the way it would be taught in the psychology class—but it was very, very important to me that everything that we convey to students be grounded in science. And, if you just Google attention and attention demonstrations, you’re going to get lots of fun stuff off of the web, but this is an area that I already knew there’s a tremendous amount of misinformation, so everything had to be top notch in terms of what we were telling students. And we also wanted it to be something that had an open resource feel to it, something that we could kind of take apart and share with other people that wouldn’t be this one specific package that you had to either, Heaven forbid, purchase or that would be very hard to convey. We wanted it to be something that could be used in a lot of ways by a lot of people and adapted. So that’s what we wanted to do. And our design philosophy kind of converged too on this being a little bit like a MOOC—a Massive Open Online Course—so it’s got that flavor as well. Because it’s not part of my teaching load or anybody else’s, it has to be something that this does not result in a lot of papers or exams for me to grade. That’s just not feasible. It has to be something that can be run with very light interactions from me, so that’s what we were going for. So with that, it became this really creative, open-ended process because they would come up with research and say, “Hey, here’s this video I found,” I would say, “Well, no, it doesn’t really tie to this or it kind of conveys this in a way that I don’t like,” or “Oh my gosh, that’s wonderful.” Just for an example, John and his colleagues came up with one of the videos that I will talk about a little bit. It was put together by a driving safety initiative in Belgium, so it was actually subtitled, and it’s this crazy thing called the Text and Driving Test where it’s sort of like an online prank, where a student taking a driving test is set up to believe that now we’re asking everybody to text while they drive around the drivers’ training course. And havoc ensues, and the students are yelling and saying, “There’s no way this is safe, nobody can do this,” and that is something that I would never have thought to put in there, but that is something that students can definitely relate to since many of them have only recently learned to drive. And they know about the dangers of texting and driving, which—while it’s not learning specifically—it does kind of tie back to that idea that when we’re distracted, we’re oftentimes the last to know and the consequences are disastrous. So that’s how this process played out.

Rebecca: Can you talk a little bit about how the faculty have responded? You mentioned that a lot of faculty are using it as extra credit assignments and obviously, if it’s got distribution beyond just your institution, people are obviously engaged with this, but can you give us a little more specifics about what faculty are seeing the student benefit of doing this in their classes?

Michelle: Here to, this is something where really systematic impressions is a project for the future. But there’s absolutely a “voting with your feet” component to this. My fellow faculty have been so willing to incorporate this into their classes, especially those who are teaching with STEM classes, as I said, those really tend to be pretty popular. They are really, really careful about where they put their class time. Their classes are so sequenced and there’s so much in them, that they’re not going to sacrifice a moment or a point, if they don’t believe that this is something that they really want to get across to their students. And, I think this is reflected in some of the numbers which I just drew up as of today since we have students coming through this module all the time at NAU. We have had 4,481 unique users since the inception of this module and about 3,100 have gone all the way through and completed it, which for a MOOC, that’s a pretty good completion rate. So faculty’s willingness to really incorporate this as a core part of what they’re offering students, I think, speaks very highly—and again, I’m grateful to my colleagues. Because this sort of thing would not work if faculty don’t really endorse it, and not just with, “Hey, you can do this, but I will give you credit for doing this.” Another part of this too, though, I think—and especially for those people who might want to try to replicate something like this at their institution—is really needing to look at it from that faculty side as well of how can we make this a really kind of painless and low-investment proposition for faculty. So, you know, being able to really assure that the students are getting something where the science has been vetted, it’s going to pay off for the time invested, but also they don’t need to grade or support it, and it’s not they don’t care about this issue, it’s just really, they’re as stretched as they can be. So what students actually get at the end is a certificate of completion. So it’s something that our learning management system makes it very, very easy for them to just send right to their professors so the professor to mark them down for extra credit. So streamlining that last piece of that I think has been very helpful too for getting that positive faculty reaction.

John: If faculty would like to use this, or to use this approach, how can they go about doing that?

Michelle: Well, let’s see. I think first of all, I really encourage faculty to get a grounding in some of the basic issues and it’s not something, even as a cognitive psychologist, I’m not going to say “Oh, go out there, make a deep dive into the journal articles on this.” There are now some really high quality resources that talk about student distraction in a reasoned and evidence-based way which is so important. I would encourage them to consult—first off—Jim Lang’s excellent teaching distracted students series that has come out in the Chronicle of Higher Education. And he’s just been a really great voice for this, for giving a balanced reasoned take on the issues. So, I think getting familiar with that. Now, really playing around too with like what we did, to say what would be the best way to engage students in this, with that engagement idea of being really key here. And we’ve had such success with this MOOC approach, this online module approach at NAU. If they’d like to do something like that, of course, I say, “Get in touch with your local e-Learning Center and instructional designers to look at that.” There’s just no way that I could have put together something like this without that kind of input to set up the design in the way that we did and to come up with all these innovations. So, seeing whether there are some productive collaborations that can be done there, or perhaps with the library leadership and staff as well. So, having an idea of what does your initiative look like, what approach you’re going to use, how are you going to get it out there to students, is it going to meet with the faculty as we do, or something else? Now, if they would like to specifically take advantage of what we’ve developed as far as Attention Matters, contacting me directly by email or any of my other modalities is the way to go. Now what we have is not a slick commercial type of module that you can just pop open. But we have all of our materials unbundled. Everything from the self-assessments that students can take, the different surveys that we developed if they want to use those, the very short conversational introductions and summaries that I wrote—you’re welcome to those as well—and we can help you to an extent to put that together or give you what you need to create something like this in your own learning management system. And so we’ve had some colleagues who’ve been really willing to do that and to put their own spin on this and make it work for their own students and their own environment, so that’s something that you can do as well. We have a chapter out in a publication that’s free online—it was put out for the Society of Teaching Psychology—that talks a little bit more in depth about the different pieces and some things we found as far as attitude and belief change, so that I think is a good general appeal reading that you can do. And we’ll have, hopefully, some more publications coming out on this. And of course, they can always come to my website or talk to me if you’d like to do a more deep dive, put together a workshop for faculty, or anything else like that on this issue of distraction for students and how to build students skills in this area. I really come to believe that this is a metacognitive skill for the 21st century. There’s so much discussion and debate about what sort of policy should we have, what do we tell students to do or not to do? And I just think we need to come at it in a different way of looking at this as something that students need to master for themselves, to understand for themselves, and really make their own plan for how this is going to be a part of their life, kind of managing their own distractions and what they should know about their own minds. So I’m always really, really happy to share more and to talk more about this because it is just such an issue that I care so much about.

John: And as one of the campuses where Michelle has given workshops, we’d also encourage you to contact her about giving a workshop on these or other materials.

Rebecca: It’s also a topic that, like many others, like learning how to learn and other things that we’ve talked about with you, Michelle—and also other episodes of the podcast—students don’t innately know these things. I think sometimes the assumption is that we know about attention, but there’s lots to learn about it. So you got to meet people where they’re at and remember that that’s not necessarily something that they know about and maybe be willing to spend some time on these issues because in the long run, it might help them engage in whatever subject matter you’re hoping to get them engaged in.

Michelle: Absolutely.

John: And they’ll be more productive in the rest of their lives because these distractions are not going to be going away anytime in the near future.

Michelle: Absolutely, absolutely. I think we’re all completely on the same page as far as those ideas are concerned.

John: We always end the podcast with a question—as you know—what are you doing next?

Michelle: One of the big things, my big project for the summer, is to continue actually revising and getting out there an article that does go into more depth about the impact of the project itself. And I’m spending a lot of this upcoming summer pushing a lot of different kinds of learning related research projects ahead. I’ve got quite a few things in the cooker and I love them, but that’s how the summer is going to go. I am continuing in particular to work on some neat projects having to do with virtual reality for education with the immersive VR lab we have at NAU, so that’s all part of this kind of big complex of interest that I have around the mind, how we take in information, and teaching and learning. I’ll be continuing to work with faculty doing some speaking and some workshops, and I will be tackling my own long, growing, and rather intimidating reading list at this point. So much good work is coming out now, so many books, so many articles. So, I’m really looking forward to continuing to get through those this summer.

Rebecca: Well, thank you so much, Michelle. It’s always a pleasure to talk to you and I think faculty will engage with your materials but also just think about attention as a topic that they might want to tackle with our students.

John: It’s always a pleasure talking to you.

Michelle: Likewise, always a pleasure.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

85. Small-Group Discussions

Small-group discussion activities provide all students, even the quiet ones, with an opportunity to actively engage with course material. In this episode, Dr. Dakin Burdick joins us to explore a variety of small-group discussion activities that can be productively integrated into our classes. Dakin is the Director of the Institute for College Teaching at SUNY Cortland. He has been active in professional development for almost 20 years, and has served on the Board of Directors for both the Professional and Organizational Development Network in higher education (the POD network) and the New England Faculty Development Consortium, where he was a president for four years.

Show Notes

  • Burdick, Dakin (2019). Small Group Discussion Protocols
  • Joan Middendorf — Teaching Resource Center Director at Indiana University
  • Middendorf, J., & Shopkow, L. (2017). Overcoming Student Learning Bottlenecks: Decode the Critical Thinking of Your Discipline. Stylus Publishing.
  • IUPUI — Indiana University – Purdue University Indianapolis
  • Angelo, T. A., & Cross, K. P. (2012). Classroom Assessment Techniques. Jossey Bass Wiley.
  • 84. Barriers to Active LearningTea for Teaching podcast (with Lindsay Wheeler and Hannah Sturtevant)
  • Larry Michaelsen — Professor of Management at the University of Central Missouri, pioneer of Team-Based Learning
  • Michaelsen, L., Knight, A., & Fink, L. (2005). Team-Based Learning: A Transformative Use of Small Groups in College Teaching. Stylus Publishing.
  • Team-Based Learning Collaborative
  • Elliot Aronson — Inventor of the Jigsaw classroom technique
  • Aronson, E. (1978). The Jigsaw Classroom. Sage. Chicago.
  • ZoomiOS, Android
  • Eric Mazur — Balkanski Professor of Physics and Applied Physics and Area Chair of Applied Physics at Harvard University
  • Teaching Professor Conference
  • David Pace — Professor Emeritus of History at Indiana University, Bloomington.
  • Pace, D. (2017). The Decoding the Disciplines Paradigm: Seven Steps to Increased Student Learning. Indiana University Press.
  • Pace, D. and Middendorf, J. (2004). Decoding the Disciplines: Helping Students Learn Disciplinary Ways of Thinking. Jossey-Bass.

Transcript

John: Small-group discussion activities provide all students, even the quiet ones, with an opportunity to actively engage with course material. In this episode, we explore how a variety of small-group discussion activities can be productively integrated into our classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Dakin Burdick, the Director of the Institute for College Teaching at SUNY Cortland. Dakin has been active in professional development for almost 20 years, and has served on the Board of Directors for both the Professional and Organizational Development Network in Higher Education (the POD network) and the New England Faculty Development Consortium, where he was President for four years. Welcome.

Dakin: Hi. Good to be here.

Rebecca: Today’s teas are…

Dakin: Yes, I am drinking Sleepytime Vanilla today.

Rebecca: That sounds yummy.

John: And a great way to start the day. [LAUGHTER] And I have Ginger tea.

Rebecca: And I have something different today. I have Strawberry Grapefruit Xue Long Flavored Green tea.

John: Okay.

Dakin: Nice.

John: We’ve invited you here to talk about effective ways of engaging in small-group discussions. You’ve done quite a few workshops on that. Could you tell us is about your shift to small group discussion protocols in your own classes and how you get started with using small group discussions?

Dakin: Sure. My method of instruction was lecture primarily. I started off in history and I did a lot of lectures. Some of them were good—and I’m proud of those few—but I was teaching a class at Indiana University Purdue University in Indianapolis at Columbus and it was a U.S. history survey, first-year survey. It was once a week, three hours long, and some of the students had traveled for more than an hour to reach the class after a full day of work. Some of them came in with their dinners, and I knew that lecture class was not going to get me through that class. Nobody would survive it; not even me. [LAUGHTER] So I worked with the Teaching Resource Center Director at Indiana University, Joan Middendorf. I selected several small group discussion methods: jigsaw discussions, role playing. I modified the debate system to create a evidence-based debate protocol. I used the just-in-time teaching method from IUPUI, and the classroom assessment techniques of Angelo and Cross, which I kind of regard as my Bible. The combination of those worked really well. The students remained active throughout the class, we were often surprised to find the class was over, everybody was still energized, I still had a protocol or two to go. Everybody learned each other’s names because we did random groups, and the class as a whole was tremendously successful. I was really happy with the results and I’ve used it ever since.

John: Whole-class discussions are often used but what are the advantages of small-group discussions relative to a whole-group discussion?

Dakin: I actually advocate both of those. I advocate lecture, small-group, and whole-group discussions for different purposes. The large group for me is one where I would often find faculty having the usual suspects were the only ones talking. You had three or four students in a class of 20 to 30, no one else was talking, and the faculty member would usually come to my office and ask, “Well, how can I change this? How can I get more students involved?” and often, that’s where the diagnostics began. What I found was, first of all, they needed to have preparatory homework, the students needed to do it, it needed to be graded, and if it was graded and it’s frequent assessment in order to reduce faculty load, they had to grade this lightly, and place the effort onto the students and not onto themselves. And if they did that, usually things improved. The advantage of the small groups is that if they’ve done all that work, then the students wants to talk. If you get to a large-group conversation and you’re not talking, it’s pretty boring for the students that aren’t talking. For the faculty member, it sounds like it’s a really good conversation because they’re the center of the wagon wheel, they’re the center of the hub, and so they’re constantly talking with those three or four students, but they’re wondering what’s happening with the rest of them. And that’s a good thing to wonder about, frankly. If you use small groups, you have the advantage that more people are talking at the same time. So instead of having one person talk at a time, you can have six people in the room talking at a time, so there’s a lot more conversation taking place and hopefully, more change in learning, which is important. Students in small groups feel more free to talk, there’s less risk in a small group, they can gain confidence from that talking, and they’re more active in the classroom at the same time because there’s more people talking. It also gives them the chance to practice disciplinary skills that the faculty member has put into that assignment. So the assignment shouldn’t be about declarative knowledge or facts, it should be about how can you do something in the field? How can you emulate the skills of an expert? The other piece of this is that the small groups have been demonstrated to be effective. Students in the 1920s said they preferred this sort of discussion—at least large-group discussion at that point—and then by the 30s and 40s, there was research and social psychology showing that small groups were more effective in promoting change and student learning. And from there it went on and since the 50s and 60s, a lot of different types of protocols have been invented and developed. There’s just a lot of advantages.

John: Could you tell us a little bit more about how you ensure that students come prepared to actively participate in those small group discussions?

Dakin: The main thing is grade it. The rule is that students—according to the Carnegie unit—are supposed to study for two hours outside the class for every one hour in. Well, the Study Study at Indiana University showed they weren’t doing that. They were studying one hour outside of class for every hour in, and it was usually on Sundays. That means that we kind of have to take that knowledge and make use of it. We know they’re going to do their work on Sundays. So okay, the assignments going to be due Sunday night, 11 o’clock, 11:55. But you make sure that they work harder. You don’t feel guilty about putting more work on them because more work means more practice. All of them could use more work reading, all of them could use more work writing, so that’s what I have them do. And then I make sure they turn their assignments in on Sunday night, I grade them Monday morning—which is a principle of just-in-time teaching—it’s preparatory homework, I read that, and then I modify my class based on what the students bring to it. And so I can see—first of all—their weaknesses. I can see their misconceptions. I also can see their strengths. Occasionally they have real strengths they bring to the class that would be totally invisible if I hadn’t done this work. And my example for that is I had a class where I was teaching the My Lai Massacre—about Vietnam—and I had in the class two very strong students. One was a G.I. who had fought in Vietnam, and one was a First Lieutenant who had taught the rules of land warfare at Fort Benning for three years. Both of these guys were A, smart; B, aggressive; and C, constantly fighting with each other because it was enlisted versus officer. So they dominated the class very often. And when I got back their feedback, they told me their knowledge and I hadn’t seen that knowledge before. I did not know that these were their backgrounds. So I spent an extra two to three hours reading about the rules of land warfare, came to class prepared, and instead of a class that would be a trainwreck for me, what happened is they came in and the Lieutenant said, “Everybody knows the rules of land warfare. So they’re all guilty, and they are all responsible,” and the G.I. says, “No G.I. is going to read a 100-page field manual on the rules of land warfare.” And the Lieutenant says, “Well, there is no 100-page manual on the rules of land warfare,” and I said, “Well, actually, there’s three. [LAUGHTER] There’s the 1956, 1965 and the 1973 (revised on the basis of My Lai).” Okay, so that stops that conversation. Then I turned to the GI and I say, “Okay, but every G.I. has those little plastic helmet liners, right? …with the 10 rules of land warfare on them.” “Yeah.” “Okay. So we’re agreed; they knew the rules of land warfare. Some follow them, some didn’t. Now, let’s talk about why,” and at that point, the conversation became really useful. First of all, all the other students could participate, because they now had the background, and the two people that were real experts in the room could help us kind of determine why people followed or did not follow those rules. And again, if I had not done that preparation—just two extra hours—that class would have been ruination.

Rebecca: What are the kinds of questions that you have students respond to that maybe elicited some of the information that helped you? What are the keys to asking good questions for that preparatory work?

Dakin: I think the keys to that are knowing your subject. So, everybody that is a content master—every faculty member —has their own expertise, and it’s pretty impossible for me to name the prompts that they might use effectively. But they probably know them, they’ve probably seen them in their graduate work. They know that these are the elements that made up their dissertation exams, their qualifying exams, and they’re probably pretty smart about what are those major issues in their field that need to be discussed, and to be prepared, and the students need to be prepared. The big thing is making sure that we’re talking at a high level of cognition. So in Bloom’s taxonomy, talking about analysis, talking about evaluation, those are the levels you want to get at. And those are almost impossible to get at with multiple-choice questions or tests, so that means there has to be conversation, there has to be writing. Those are elements that are important.

John: Earlier you mentioned that you use random assignments for small-group activities. Do you do change the groups on a daily basis or do you have more persistent groups?

Dakin: Occasionally, if I’m doing long-term group work, I will do some sort of pre-test, find out what the strengths and skills of the students are, and then place them mindfully into those groups, so that they construct useful groups because they’re going to be in those for half a semester. I’ll do a swap halfway through, but that’s a long time to be in a single group. For the random groups, I definitely do that and I do that on a daily basis. The students originally complain about that, but they get used to it pretty quickly and they’re ready for it. And the advantage is that they get to meet everybody in the classroom. They get to be in a group with everybody else and that builds trust and it builds community. And that allows them, by halftime through the semester, they know everybody, they’re comfortable with everybody, they trust that other people have their best intentions at heart, and then the conversation just escalates from there because everybody’s now willing to talk.

Rebecca: In our previous episode that we just released last week, we discussed some of the issues that can come up when you’re using evidence-based practices for the first time. A lot of people know or buy into the idea of small-group discussions and might just go for it without necessarily having a good plan in place, and things might go awry. Can you talk a little bit about ways to be prepared for trying something new, the kinds of things that might go wrong, and how we might adjust ourselves a bit as faculty members as we’re trying new things?

Dakin: The thing that I find usually is that people just don’t give the new techniques a chance. It’s scary. Now there was a study back in the late 90s out at Brigham Young and they asked faculty two questions. They asked them first, “What do you think are the most effective teaching methods?” and then “What do you do?” and they were diametrically opposed. [LAUGHTER] And the reason was time management, people are very busy and the stuff that’s effective takes a lot of time to do—or they think so. So I view my job as an instructional designer when I’m helping them to reduce that amount of time and make sure that they can do that. So first, make it time manageable, so that you can do the task and you can feel comfortable. Secondly, trust the system. Trust the change you’ve made. You made this change for a reason, trust it. And third, trust your students. Your students want to succeed, they want to learn. Trust that and have them help you make this successful. Tell them what it is you’re doing in the classroom, why you’re making this change, why you think this is going to help them learn better, and then also use feedback from them to get it. So I typically will use something called a stop, start, continue—What do you want me to stop doing? What do you want me to start doing? What do you want me to continue doing?—and use that student feedback to then modify the class. So it’s kind of like a mid-semester evaluation, but I feel like doing it whenever I do… it is just fine.

John: Now earlier you mentioned that whole-group discussions have a place. In what sort of sequence might you use or in what combinations would you use small-group discussions and then whole-group activities?

Dakin: My process is basically four-part. One, preparatory homework. There has to be preparatory homework and it has to be graded—lightly graded—and it should be moderately challenging. Next, they come to class, there’s a brief lecture and the lecture introduces the material, frames the questions we’re going to talk about today, maybe corrects some of the errors that were made in that preparatory homework, also celebrate successes from that preparatory homework. Once that lecture is done, maybe 10, 15 minutes, then move them into small group work. Small group work can be anywhere from one to two minutes in a lecture hall to 40, 50 minutes—and you might do a whole session on the rest of that piece, maybe a debate or some large-scale exercise—usually though, about 10 to 15 minutes in small group. Then when you hear the sounds rising, that means they’re talking about things they enjoy, which means their social life, [LAUGHTER] and so it’s time to stop them. You’ll also see sometimes that there will be a student—maybe all the A students somehow got at the same table—and they’re done three to four minutes before everybody else. Well, the point of putting them into small groups is to build energy and confidence and you don’t want your A students to be bored. So if you have a group that’s done first, you appraise how much of everybody else got through, “Can I stop this now?”—usually you can—and you bring them back to the large-group discussion. And in that large-group discussion of 10 to 15 minutes, you do debriefing and you find out what they think they know, maybe use a classroom assessment techniques from Angelo and Cross, and you evaluate and you build feedback that you can use later. And then once you’ve got that, then you move back to the lecture, and you clean up the misconceptions, you explain and reframe the next issue, and then it’s just a cycle. So it’s lecture, small-group discussion, large-group discussion, and continually like that.

Rebecca: What are some strategies that you use in small groups to make sure that everyone participates or is engaged and stays on task?

Dakin: First, make sure they’ve done the homework. Secondly, randomize so that I’ve got some good students and some poorer students in the same groups, so that we have people that can interact—also, so that people can learn about each other. To keep them on task and walk the room: first of all, be engaged with them. Listen to what they’re saying and if it’s on task, you just congratulate them and move on, if it’s off task, okay, now start working with them—and there’s going to be one group that’s off task, certainly. Other pieces are… that you might encounter a small group where there’s a number of dominant individuals. So there’s a couple of people that are really assertive, and they’re talking all the time, and they are just dominating the whole piece, and the other people aren’t getting heard. And so in that point, then you start introducing other discussion protocols that will allow more inclusivity: so things like expense account, talking stick, things where other people’s voices are valued. Another one would be Larry Michaelsen’s Team-Based Learning that also does that.

Rebecca: Can you talk through each of those, for those that aren’t familiar with each of those?

Dakin: Sure, let’s start with talking stick. Talking stick is very simple. You have an object—usually a pencil or something—and one person gets that stick, and is able to talk for one minute without being interrupted, or any comments from anybody else. And then you pass it to the next person, for one minute they get to talk, and it goes around the room that way. And then once it’s gone around once, everybody can talk at once and kind of work out what it was that they heard said, but everybody’s voice is listened to and heard during that time. That’s a rather formal way. Another less formal way is expense account, which is maybe you give them three—or however many pennies you want—three tokens. And they pass those tokens in each time they talk. So the assertive ones are going to spend their pennies very quickly. [LAUGHTER] And the less assertives are going to then have a chance to spend their pennies. And when everybody has spent their pennies, you all get your pennies back and now you can start again. But again, that’s a way to give people a chance to speak. But people can choose when they want to speak, rather than having this turn where it’s coming around. And it’s very set. Larry Michaelsen’s Team-Based Learning is much more complex. Larry started this in a lecture hall. And so he has basically an IRAT and GRAT. And the IRAT is an Individual Readiness Assessment Test and that GRAT is a Group Readiness Assessment Test. So he has them take an IRAT first, and as an individual give their answers, turn that in—that’s a grade—and then he has them do the GRAT. So as a group, they now turn in their group grade. And he also uses the scratch off cards, the if/at cards, that sort of thing. Initially, the assertive ones—again—are giving the answers. But as they discover that they don’t have the right answers all the time, then the quieter ones in the group suddenly become more important to the group because their grade is dependent on this. So they’ll start asking, “What did you get for this? You seem to get A’s all the time. Can you please help us?” And so that’s his method of doing that inclusivity.

John: And in that approach though, I believe he recommends persistent teams over the course of the semester, so that they develop that sort of team dynamic.

Dakin: Yeah, very much so.

John: Are there any other small group activities that you like to use?

Dakin: I have a lot of protocols that I’ve gathered over the years—probably got 40, 50 protocols—and the ones that I select are the ones that are low risk. So I’ve kind of classed them as low risk, medium risk, and high risk in terms of how much risk does the student feel when they’re in the classroom doing these… and I like low-risk things which are usually small group where they’re by themselves and they’re talking, and it’s not in front of the faculty member, and they are not having to answer to the whole class in front of a large group. So, some of the pieces I like are jigsaw—which comes out of Elliot Aronson’s work in the 70s—the idea that you break up an assignment into five pieces, and each of the students in a group will do one of those five pieces, and then they will talk in class and share out what they’ve learned from each of those five pieces. So it’s a great way to synthesize a lot of data that maybe you don’t want all students doing. So when I was teaching my Middle Eastern history class, each student was responsible for a different country and they had to do a lot of reading on that country. But if I had all the students do all that reading, it would have been far too much. So instead, I can have these various countries sit at a table and then have a conversation, and the student representing Israel can talk about Israel’s point of view, and the student doing Jordan can talk about Jordan’s point of view, that sort of thing. So that’s one method I love and I even do a double jigsaw. But I only do a double jigsaw maybe twice a semester and they’re at moments where there’s so much content, that there’s absolutely no way we can cover it. And the best example of that is U.S. history survey, first day, which is the dawn of time to 1492, which I think is horribly disrespectful to everybody that was in North America before 1492. So we do a double jigsaw, which is where you have a jigsaw that creates experts at each table and then those experts then are now experts in five different topics. And those people then go off to create super jigsaws. And that works well, but it takes a lot of time. The other one I love is role playing. Role playing… just because it’s my age… I grew up with role playing, but I’ve done a lot of different styles of role play. The one that I think I use the most in first-year history is Articles of Confederation. Everybody takes a representative to the Confederation and talks about what it was that person was like, and why they voted the way they did, and what were their goals. And then we skip ahead to the Constitutional Convention and we talk about who’s still there, who’s not there, why are they not there? If they’re still there, do they still have the same opinions? Are they still voting the same way? Why are the results different at the Constitutional Convention as opposed to with the Articles of Confederation? So that’s a good one. Other classics are the Oregon Trail… everybody loves the Oregon Trail. And unlike the computer game, what you learn is not many people died on the Oregon Trail. People who died most were the Native Americans who are along the trail, everybody else pretty much made where they were going, but that had to do with who those people were. The other one we did was Cuban Missile Crisis. Did the Cuban Missile Crisis and role played the various operatives in the Cuban Missile Crisis. And then next week we talked about Watergate and again looked at those same operatives and where they were now in the Watergate plumbers. So that was also useful …those kind of things. There’s lots of different ways to use that though. I’ve seen people use that with theorists. So in psychology, different theorists are represented by the students and they argue their different theories and try to figure out how these things go together. Role playing is obviously one I love.

Rebecca: Can you talk a little bit about how you set the role play up? You mentioned what some of the topics are and when you use it, but can you talk a little bit about the logistics of setting that up and how you have students prepare for that?

Dakin: And those vary a lot. When we’re doing the Articles of Confederation, I just have a list of representatives and I asked them to choose one and then we go from there. So my prep on that is zero, they are the ones responsible for that prep. On some of the others, there’s a lot of prep. With Oregon Trail I worked out, I took a K-12 game that had been done for Oregon Trail, and then I made it much more complex and they had to purchase their gear so I had a full list of gear, I had a list of where they were going to stop, how they were going to stop, and what the mortality rates were. So I basically created this whole game around it and then they played through that. With the Watergate and Cuban Missile Crisis, it was kind of halfway between there. I made cards with each of the people they would role play and on the back—like a Clue card—it tells you who this person is and what their role is and then I gave those to them. And then from there, they again generated most of the data.

John: You mentioned you have these organized by levels of risk. How would you recommend using the different levels? Would it make sense to start with low-risk activities, and then as more trust is built, build the higher ones, or would that be affected by the level of the class that you’re teaching, whether it’s introductory or more advanced?

Dakin: Yeah, it definitely depends upon the purpose of the class. I tend to teach introductory classes so I build a lot of trust, I use a lot of low-risk pieces, and I’ll move to maybe medium risk by the middle of the semester—or maybe I won’t ever use medium risk, it’ll all be low risk—because I’m trying to get them to get used to college and figure out what that’s involved in. If I am teaching a class that’s kind of a gateway or portal class that’s going to lead on and it’s supposed to cull out people, it’s supposed to find out who the best people are, then maybe it makes sense to start doing some of those high-risk pieces, but I probably wouldn’t do that until at least the third year. Build a lot of confidence, a lot of trust, and there’s a lot of learning that has to take place before that, before you get to that point. Traditionally of course, if you look at law school or medical school, they have a lot a lot of high-risk protocols, because there’s a lot at stake and people have to do well. And I remember when I started working with the med school that I read academic medicine, and one of the articles was, “We should abuse our students less.” [LAUGHTER] Not, “We shouldn’t abuse our students,” but, “We should abuse them less.” So, that kind of gave you a sense of what we were dealing with.

Rebecca: Can you talk a little bit about some of the small-group techniques that you use in online environments?

Dakin: Sure. Online environments—actually, the small groups—I usually leave that up to the groups themselves. So if there are groups of students that I’m having work on particular topics, I’ll have those students organize, say, a synchronous conference where they get together on Zoom, and they talk about these things. So they get to pick whatever tool they want—I’ll recommend Zoom because it’s free and you can have up to 40 minutes free, and you can record it. It’s very simple—but they will do that work by themselves. If we’re dealing with, say, a discussion forum, what I’ll generally deal with is ways that students can interact kind of more of at a large-group level, because there’s really no need for a small group when you’re on the discussion forum. But what I do want to do is get rid of the old “post one, reply two” because after you’ve taken two or three online courses, you’re pretty bored with that protocol. And so giving them different ways to think about it and moving the jigsaw into it, moving a debate into it, moving role playing into it, those are all really useful.

Rebecca: Can you pick one of those more complex ways of using a discussion board and talk through how you set that up or organize that?

Dakin: Sure. I think the main thing I do is really—it’s not so much about the organization of the board when I’m doing it—but building student activity. I do a big sales job in terms of talking about what is the value you get from an online course. Now, if it’s just teacher to student, I think that’s a really limited amount of value because there’s a lot of good books out, you can read, you can train, you can look at YouTube, there’s all sorts of great ways to learn. But a real value from an online course for me is who is in that class with you and finding out what their strengths are and what they can bring to it, and that’s where a large part of the education comes. So I don’t use this “post one, reply two” but I do want them to make sure that they are responding weekly to their colleagues, but at a level they feel is appropriate. So don’t say something if you think it’s totally pointless. But if you have a comment and you feel it’s worthwhile, say it, because we need to hear it. That’s the largest part of this. In terms of the organization, the only pieces I’ve done in terms of organization have been very slight. So, with an assignment, you turn in your first post Wednesday, and you turn in your final post on the piece, on Friday, Saturday, or Sunday. So the initial post is what your response is—out of a think-pair-share, this would be the think part—that you’re doing your initial writing. And then the second part of the week is simply the sharing part. So now you’re responding to those students. Now, if you’re in Canvas, you can do this with setting up the initial one as your due date, and the second piece as your until date, so you can do it within one assignment. Unfortunately with Blackboard, you can’t do that, you have to have two separate assignments. But that’s the only real difference.

John: What about larger classes? What techniques do you recommend there?

Dakin: In large classes, I often talk about Eric Mazur and his peer instruction, simply because I can send them to the videos he’s got on YouTube, and he’s got a lot of videos there. He’s got a lot of publications. So that’s great, I have a lot of resources I can send them to that they can start working on. But Eric’s technique is largely one that applies to an interactive lecture. It’s not really small-group work per se, it’s a way to maintain activity by the students and also makes sure that you’re getting feedback back on what they’re understanding. But since Eric uses multiple-choice questions, he’s really not getting beyond that understanding- or application-level question. So, the issue really with small-group work and large classes is really not about the size of the class, it’s about the furniture in the class. So, you can do small-group work with a very large group as long as you’ve got movable tables and chairs. So, I did this at the Teaching Professor Conference a few years ago. I had 110 people in the room and we did value line, and we did jigsaw, and we did all these different things, and it’s very easy to do as long as you’ve got the furniture that allows you to do it. The hard part about a “large” class is really it’s about the lecture hall and the furniture in it. So if you’ve got furniture that’s fixed, if you’ve got a table that’s fixed, if you’ve got chairs that are fixed, it’s hard to have more than two to three people working together at any one time because they can’t turn around—they can’t do anything else. Also, since you’re in a large lecture hall, there’s a lot of noise. So again, you don’t want to get more than three people because you won’t be able to hear the others. If in a lecture where students can turn around, then you can have a larger group of say four to six. So you have two to three in the front row, two or three in the back row, and they’re talking together in that small group. And I’ve seen small-group work in lecture halls with as many as 160 people, so I know it works. It does take some effort in terms of arranging it. Usually they don’t do random small groups every day—because that would be chaos—but they do long-term teamwork and the faculty member who did this was David Pace at Indiana University—he was very good at this, he’s the one who taught me how to do much of this—he does a pretest, he organizes the students, he puts them into these long-term teams, and then in those teams in the lecture hall, they have the seating arrangement where they’re sitting. And then when he wants to do small-group work, he’ll do his lecture, and he’ll do small-group work, then he’ll do a debrief, same sort of pattern.

Rebecca: What do you find your role is, as an instructor during small-group work? You want to put a lot of the onus on the students, but what’s your role during all of that and how does that scale up to a big class?

Dakin: My role is—as an instructor small-group work—is essentially challenging, adding to, and supporting. Making sure that they know they’re encouraged and they’re doing a great job and going around doing that sort of thing as I walk the room. A lot of the work I do is really the preparation. Making sure that those things are well thought out, that I have a lot of idea of which directions they can go, and to, after the class, make sure I’ve done my reflection: I’ve written down all the weird places they went so that I know that those are possibilities and I can be ready for those, or maybe I just work towards those. Maybe those were better ideas than the ones I came up with—which is actually one of the big advantages of small-group work because you are paying more attention to the students, the students have a bigger role in the class, and your life isn’t as boring. If I was doing the same lecture 20, 30 years later, I would be bored to tears. But as it is, since I’m using these, every semester is different because every group of students is different. So my life is constantly interesting. And it’s almost like doing improv, really, in a way. You have to be a little brave about it, you give them opportunities, but there’s a lot of trust, you trust the students are there to help you. And everything goes well, even with apathetic classes that when I’ve walked in, the class has just been dead, they don’t want to do anything. After a week or two of this, they start getting into it. And by the end of the semester, they’re the same as every other class, and it’s going very well. So it’s highly enjoyable. So I think that’s it, make the class fun, get them to trust and encourage them to do their best work.

John: And in large classes if you have TAs, you could have TAs going around and doing the same thing, just so that you get more of the room covered.

Dakin: That, or if you’ve got a tight space to deal with, you could also have a backchannel going, so people in the groups are reporting out and the TAs are looking at that backchannel through Twitter or something else and kind of getting those ideas and feeding those back to either the students or the instructor.

John: Earlier you mentioned that light grading be used. Could you elaborate on that a little bit?

Dakin: I think that’s the hardest thing for faculty to do is light grading. Faculty members really want to mark everything. If they see something wrong, they will mark it. And I must admit, myself, when I’m posting to Facebook and somebody writes something and spells it incorrectly, I have to respond. It’s annoying, but I have to do it, and it’s the same way with grading. People will try to grade everything and they will eat up their lives giving these huge responses back that the students really aren’t going to listen to. Nobody has time to make all those corrections. So the smartest guy I ever saw was Bob Ferrell, who was a professor of history. And Bob was highly published—he had 50 plus books—and he still had a line out the door of students that he talked to every day and that was highly admirable as far as I was concerned. And so I wanted to find out how he did this, and what he did is… I took a readings class with him and I handed in a paper a week, and we worked through that. And every paper, the first time he got it, he marked it up pretty heavily to show, “You need to work on your grammar and I’m watching you.” But after that, every week, it was three things. He’d mark… circle one, flip a couple pages, circle another, flip a couple pages, circle another, “There you go,” out the door, you’re done. And so for me it felt like, “Oh, I only have three things to change. This is great, I’m really close to getting that top grade.” And next week it would be another three. And next week it would be another three, and so on. So, he was doing light grading, he was giving me feedback—feedback that was useful to me—feedback that was moderately challenging. I didn’t feel at sea, I felt I could do it. Great. And so I would do it. And that’s the way I come to this. The way I implement it is, say if I’m in a freshman class, I will have the students writing say 1000 words response every week, which for a freshman class seems like a lot, but I want them to work and I want to hear their voice. I will tell them not to use any quotations, I want to hear their voice, I don’t want to hear somebody else’s. I want to hear them thinking, and if they don’t agree with the text, argue with it—that’s fine. If you don’t agree with me, argue. That’s what you should be doing. You’re trying to construct your ability to speak and write. So, when they do that, they then turn these pieces in, and I grade them but I grade them lightly, which means I’ve got now 40,000 word essays I’m supposed to be grading, that will take me about 40 minutes. I spend about a minute on each. I just kind of flip through it, I can tell if somebody’s done the reading or not, I can tell if there’s a major issue or not, and then I write down my responses but I don’t give them to the students. I just give the students grades. And when I get to the class, I’ll do a group grade. So at the beginning of the class, I will then do a couple things. One, I will celebrate some people, and I’ll talk about that in a minute, but I also make corrections. I’ll do grammatical corrections, will say “Here’s the five grammatical errors of the week,” and by about mid semester, I’m still showing some of those grammatical errors up on the board and the girl who’s done it says, “Oh my God, it’s me again!” So they get it and they’re trying to reduce them and that’s fine. The other thing is I talk about misconceptions. Say somebody has a misconception about a particular piece, I’ll say, “A couple people had a misconception about X.” Now it’s not a couple people, it’s Joe. It’s always Joe. Joe’s in the back room, Joe never does the reading, Joe’s having trouble. But Joe knows he’s not doing the reading and Joe knows he’s not getting a good grade, he doesn’t need to have his name called out in front of the class. So I say, “A couple people had this issue.” I talk about that and address it. And then the next part, the celebration. So, in order to make them feel better about what just happened, I then say “Now then, I wanted to talk about some of the great things that were done this week. So first of all, Jenny had this fantastic response, it was just so meaningful. I want to share it with you, because I think it’s really worthwhile listening to. And Bob said something that no one has ever said in this class before and so I think it’s important to address that.” And then maybe I talked about Jim, “Jim really did a very deep reading of the text, he brought up some serious issues that I did not bring up myself, and I think we need to explore those today. So that’s part of our discussion today, it will be based on what Jim has talked about.” So that’s the celebration, but every week, it’s a different three. It’s never the same three, it’s never always the A students. Over the course of semester, I find a way to celebrate each and every student in that class, including Joe. And Joe is hard. Joe’s really hard. So I’m always watching every week to see what Joe is talking about and if Joe says something good it’s like, “Yes, I can now celebrate Joe! Good. Check.” I’m celebrating. And that’s the way light grading works for me, it allows me to spend more time interacting with students, less time interacting with their work.

Rebecca: I’m sure we can all take advice on reducing grading, right? [LAUGHTER]

John: When we talk to faculty about using group discussions in class, one thing they often raise is a question of when students are teaching each other—in general with peer discussion or peer activities—there’s a concern that perhaps it may reinforce misinformation. How can you be sure that that doesn’t happen with small-group activities?

Dakin: Well, I don’t think you can be sure it doesn’t happen, but you can certainly set up a system to check for that and make sure it’s not happening or that if it’s happening, you’re correcting it. So the way to do that would be use some ungraded assessments, those classroom assessment techniques from Angelo and Cross. In the large group during the debriefing, some of those may come to light again, and you can then use lecture to correct those misconceptions once they become apparent. I agree that small-group work builds confidence, but it can really be like the blind leading the blind sometimes, especially if the students haven’t been doing the preparatory homework, and especially if the groups aren’t randomized.

John: And if you do that sort of pretesting, where you’re getting the initial feedback, you can tell what those misperceptions are, so that you can be prepared to address them during the class, which should help reduce that issue.

Dakin: Sometimes… yeah. I have to know which questions to ask, and often I don’t. So it’s that ungraded assessment where they toss back an answer that completely takes me by surprise. Oh, I am so surprised. Now I know what your misconception is. But I couldn’t have guessed at it.

Rebecca: I think that’s important to remember too, that [when] you’ve been teaching for a long time, the misperceptions that you might have come across five years ago are really different than the misperceptions that you might experience this year, because the experiences of our students change and the group of students change, and all of that influences prior knowledge and prior experience that influences how they might interpret material.

Dakin: That is so important. Over the 30 some odd years I’ve been teaching, my students have changed a lot, not only in their content knowledge and what they know and what they’ve experienced, but also how they think and how they behave. And again, that’s the strength of using small group work, because you get to see how they think and how they behave. And they’re not just sitting there in rows in front of you and you imagine that’s the same class you were teaching in 1987. [LAUGHTER]

Rebecca: So we always wrap up by asking, what’s next?

Dakin: I’m collecting all these protocols for my own use and also to help others, so it’d be great if people listening to this podcast could send me some new ideas, send me some more protocols, so I can share those out. The way to do that is to send them at teaching@cortland.edu. That’s our email address. In the meantime, we’ve got a new Institute for College Teaching down here, we finished up a faculty needs survey. We’ve got our advisory committee in place, and we’re just about to start setting up priorities for next year. So, there’s a lot happening, I just don’t know what it is yet.

John: Because you’ve just taken over that position fairly recently, right?

Dakin: Two months ago.

Rebecca: Oh, the surprises you might find, right? [LAUGHTER]

Dakin: I have been very pleasantly surprised so far. I have found a lot of really skilled and dedicated faculty, and I’ve just really been enjoying talking to them. I know I enjoy this because it’s a challenge, and I love a challenge. And they are so well-educated already. It makes me work very hard.

Rebecca: Which means you’ll never be bored, right?

Dakin: Exactly, and that that’s why it’s so important to me.

Rebecca: Well thank you so much for joining us today. It’s been really interesting, and I think will help faculty as they plan for their next teaching adventure.

John: Thank you and we will share some of the resources that you’ve provided in the show notes as well.

Dakin: Thank you. I’ve really enjoyed our conversation. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.

84. Barriers to Active Learning

Despite research demonstrating the efficacy of active learning approaches, observations of classroom instruction show limited use. In this episode, Lindsay Wheeler and Hannah Sturtevant join us to explore potential interventions to overcome the barriers to the adoption of effective teaching practices.

Lindsay is the Assistant Director of STEM education initiatives at the UVA Center for Teaching Excellence and an assistant professor. Lindsay’s background is in chemistry and she has a PhD in science education. Hannah’s a postdoctoral research associate at the center. Her PhD is in chemistry with an emphasis on chemical education.

Show Notes

  • Henderson, C., & Dancy, M. H. (2007). Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics-Physics Education Research, 3(2), 020102.
  • University of Virginia programs
  • Teach Better Podcast Episode 80
  • Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE Life Sciences Education, 12(4), 618–627.
  • POGIL- Process Oriented Guided Inquiry Learning
  • PODLive! Webinar
  • Meghan Bathgate — Postdoctoral associate at Yale University
  • Emily Walter — Assistant professor of Biology at California State University, Fresno

Transcript

Rebecca: Despite research demonstrating the efficacy of active learning approaches, observations of classroom instruction show limited use. In this episode, we explore potential interventions to overcome the barriers to the adoption of effective teaching practices.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guests today are Doctors Lindsay Wheeler and Hannah Sturtevant. Lindsay is the Assistant Director of STEM Education Initiatives at the UVA Center for Teaching Excellence and an assistant professor. Lindsay’s background is in chemistry and she has a PhD in science education. Hannah’s a postdoctoral research associate at the center. Her PhD is in chemistry with an emphasis in chemical education. Welcome, Lindsay and Hannah.

John: Welcome.

Hannah: Thank you.

Lindsay: Thank you.

John:   Our teas today are…

Hannah:  I have a lemon filled Earl Grey tea. [LAUGHTER]

Lindsay: I have my water.

Rebecca: I’m drinking English Afternoon.

John: And I have Blueberry Green tea. We’ve invited you here to discuss the study you’ve done on why STEM faculty are reluctant to try new teaching techniques. What prompted the study?

Lindsay: One of the big things that we try to focus on in our center is how we use local data to drive faculty development to help improve teaching and learning on our campus. As part of that back in 2016, 2017, we did a large-scale observation project where we observed over 200 STEM undergraduate courses. And we wanted to look for differences in the different instructional practices that our faculty were using based on whether they were engaging in our center or whether they were not. And this was sort of the beginning piece of driving everything that we’ve done since then, because we did see differences in their instructional practices between faculty who have and have not engaged in our center, but we didn’t see as much as we thought we would. And so we really wanted to further explore that and understand what things were hindering faculty from doing what they wanted, using evidence-based practices, particularly those that had gone through our Course Design Institute and had done other programs with us. And these are things that we heard anecdotally but we really wanted to better systematically measure this. That’s where Hannah comes in as a postdoctoral research associate and I’ll let her talk about what we did to further explore this idea of what the barriers were.

Hannah: So I came into the project when Lindsay was wanting to develop this barrier survey of some kind. And so I started by going through the literature and I found a lot of work that was of a qualitative nature that people had done in various fields, looking at barriers to implementing evidence-based practices and research-based practices. A lot of different terms are used so you have to know which ones to search depending on which field… in which journal… you’re in, so I got introduced to that, which was a bit of a challenge, but was able to kind of sort out all these different areas and find work that had been done both in DBER in specific fields and then more of the faculty development field. So I pulled on all of those different sources, but I did not find any survey instrument that was of a quantitative nature that delineated all of these different barriers that had been found in the qualitative papers. I found a couple surveys that had little sections of barriers and then I found a survey that looked at institutional climate, but I didn’t find any that delineated lots and lots of barriers that I’d seen in the qualitative work. So I drew on all that qualitative work to develop a survey instrument that we then piloted, so that’s kind of where all that came from.

Lindsay: And to add into that, there are benefits for doing interviews and qualitative work, but we wanted to really be able to find a way to quickly but systematically capture these barriers. Because as I mentioned, we are really interested in using that locally driven data and there’s only so many people in our center that can be able to do that work. That was part of the driving force behind developing the survey itself.

John: Just backing up a step, Hannah mentioned DBER. For our listeners, could you define that just so there is some clarity there?

Hannah: Yes, DBER is Discipline-Based Education Research. So I am a chemical educator, I’m a DBER researcher. Biology educators, astronomy educators, those are all DBER researchers.

John: What did you find in the survey?

Hannah: That survey instrument was not just barriers, but also some related ideas, so it included a section on teaching-research identity because that was something that came out of looking at the literature and seeing that this tension between teacher-researcher identity seemed to be something that might be a part of the barriers. So we added a section on that because we’d also not found any survey instruments that delineated those in a quantitative way. So moving on to the study. We piloted with 86 and that was a subset of the 150 instructors that were observed in the study that Lindsay mentioned earlier, that was kind of a rationale for the current research. So we were able to get 86 complete datasets out of that from the 150 that we sent it to. So first of all, we had 46 Likert scale questions— different statements about barriers that faculty participants could rate on a scale of one-to-five of, “This is not at all a barrier for me,” to “This is a barrier for me all the time.” And when we looked at the results of all of those Likert questions, the top five were number one—and that’s 65 percent said this was at least a moderate barrier for them so they rated that at three-out-of five at least—was lack of time. The second was tenure and promotion guidelines. The third was fixed seats or infrastructure constraints at 61 percent of faculty mentioning that. Number four was that students don’t come prepared at 59 percent. And then five was that too much prep time in particular was required to implement these evidence-based practices, that was at 50 percent of people mentioning. And we investigated those also qualitatively and the qualitative question that we asked—the open-ended question that we asked—was simply, “What barriers are most significant to you in your own teaching and why?” That question was a bit different. So we had all 46 of those Likert scale statements that faculty rated, but this one was getting at, “Okay, so now thinking about your work, what is the most significant barrier for you?” so it was a slightly different question than what we asked to the quantitative, and it produced some very interesting results. So what these 86 respondents said is, number one, aligned with the quantitative at lack of time, but that was only 57 percent that were saying that. Second was classroom space and lack of needed technology at 22 percent. Third was the lack of institutional support, so there’s a lot wrapped up into that question. And then number four was a variety of student-related issues and student resistance and not doing what they’re needing to do at 12 percent. And then finally, the lack of TA support and classes being too large coming in at nine and eight percent. So that gave us a greater understanding of what’s the number one issue for our particular faculty, as well as the overall landscape of all of these different Likert scale barriers. So that was interesting and drove what we were doing in our research. So one of the other results that came out of this work had to do with satisfaction and dissatisfaction with evidence-based practices. We asked the faculty who responded to the survey to go through a list of evidence-based practices and say which ones that they used. And looking at one of those practices—for instance—collaborative learning, we asked them if they were satisfied or dissatisfied with that practice—or both—and that was the practice that people were most dissatisfied with. And when we looked at that, and we compared it with their barriers results, we found descriptively that those faculty had higher barriers across all of the different barrier groupings on the survey. The ones that were dissatisfied with collaborative learning had higher barriers across all the different barrier groupings and we ended up grouping those into five. They had higher barriers across the board and we had been investigating, “What does that mean?” and as we’ve been expanding the study, wanting to get more data to really understand that and look into the policy responses on why they’re dissatisfied… things like that. But what came out of that was what Lindsay referred to, was the need to support faculty, not just before they implement an evidence-based practice, but when they’re implementing it. And we found this excellent study from Henderson and Dancy back in 2007. They did a qualitative study of physics faculty looking at supporting them and what they found is for those faculty that weren’t supported, once they came across these, what they called “situation barriers,” when they were implementing a practice, that made them stop using the practice. And so we think that our results really back up what Henderson and Dancy found and the need to support faculty once they start using a practice, helping them understand what barriers are going to be when they implement that practice and then supporting them throughout the time that they’re implementing. Because otherwise, if they’re not aware of the barriers that they’re going to face, then they may stop using that practice altogether. So that was one of the tentative results that came out of this pilot study was showing us… demonstrating the need to support those faculty.

Rebecca: I was also going to say that a lot of times faculty don’t give themselves a break. The first time you do something, you’re not perfect at it, just like our students, they’re not perfect at it the first time. You have to practice and do it over and over again to get good at it. So I think reminding faculty when they’re doing something new that will also happen for them, doesn’t hurt. [LAUGHTER]

Hannah: Exactly. There was a study that came out recently, it was over five years of implementation. And the first year went horribly, and they adjusted. It wasn’t until like the third implementation that things started to go much better, student resistance started to go down, and just recognizing the first time you implement, there will be a lot of barriers… there will be a lot of problems and that’s okay… to keep going, that this is a normal thing.

Lindsay: I think that’s part of, really, the importance of this. Other people are struggling too. Helping to normalize the fact that when you try something new in the classroom, and it doesn’t go well, it’s par for the course and that other faculty are going through that as well.

Rebecca: Those are some interesting results, but not entirely surprising. I think those are some similar things that we’ve heard and seen in other research. But interesting that it’s at your specific institution from your specific faculty, and that the qualitative and quantitative pieces somewhat align. So what have you been doing with that data?

Lindsay: We have a few different programs that we are working on refining, aligning, expanding to what we’ve found systematically in these surveys with our faculty. Some of these include our Ignite program. Our Ignite program is something that we’ve been running with new faculty for the last few years. This is a program meant to support faculty as they implement a newly redesigned course. So these new faculty go through a week-long Course Design Institute with us and then they spend the next semester whenever they implement their new course, either in Fall or Spring, they meet biweekly with one of our faculty developers, and anywhere from five to 10 other new faculty in a learning community and they build on some of the things that they’ve been learning about course design and implementation. So they’re really getting that support throughout the semester. And one of the things that came out of our barriers survey was that the other work that we’ve been doing—particularly around these observations—is that the implementation is really important and that we really need to support faculty through that. We have some studies that, particularly around Ignite and new faculty, that demonstrate how important this learning community is, not just for the implementation, but the success of students. And so now we are expanding our Ignite program to all faculty, not just new faculty at our institution. We’re doing that for the first time this Fall semester. So that’s one of the programs that we have refined based on some of the data that we’ve been finding.

John: I think one of the benefits of that is if one of the barriers is departmental culture, that prevents people from trying new techniques, bringing in more senior faculty might break that down.

Lindsay: Yes, and one of the places that we’re beginning to expand to as well are learning communities, particularly for mid-career faculty. Many of our Ignite faculty are now moving into being tenured and so they are now becoming leaders in their departments and how do we foster and continue to help support them around teaching and learning?

Rebecca: Does your Ignite program come with course releases or does it come with time?

Lindsay: That’s a good question. We do not have course release at our institution, but they do receive a $1500 professional development fund, which helps support them in being able to continue to develop, they may be able to go to conferences, they do get supported in that way. Another one of our programs that we are developing and as Hannah mentioned one of the barriers are around class size and TAs. And so we have developed over the last few years a program called Spark. Spark is intended to be a program to support teaching assistants in the STEM departments. And over the last three years, we have had over 250 TAs enroll in our one-credit teaching methods course where they actually learn about different pedagogical techniques, learning theory, and they’re able to apply that every week as they are TAs in lab courses, discussion sessions, and even in co-instructor type roles. And that has been a really important piece to help support transformation in the STEM departments because our TAs are really the primary point person in many of our first- year courses and so providing them the support has been really transformative. One of the third things that we are doing in the center is around curriculum redesign. So one of the things that we found in the study that I think you alluded to was the differences between departments and the importance of the departmental culture and departmental support in helping faculty be able to utilize and implement evidence-based practices. And so we are actually working with departments to think about not just individual courses, but what is the curriculum look like for an actual major? What do we want our students to be able to know, value, and do at the end of four years—or five years—within different departments? And so we’re really working to develop this. This is something that we’re doing this year and really working to refine our programming around curriculum development and redesign.

Rebecca: One of the themes of all three programs is curriculum development. What are some things specifically that you’ve implemented or changed consistently to help with some of the issues that you’ve identified?

Lindsay: As part of the redesign process, we don’t necessarily recommend a single type of redesign or curriculum. We really strive to use evidence-based practices, whether that’s at the course level or curriculum level, to allow faculty to think about what best aligns with what they want students to be getting out of the course, or the curriculum. For example, if one of their learning objectives has to do with being able to collaborate and communicate, we might recommend some sort of collaborative learning design as implemented in their course. If they’re more interested in students engaging with the community, that might look a little different in terms of the actual design of the course. So I don’t know if that answers your question, but we don’t necessarily recommend one particular approach.

Rebecca: If faculty are resistant to evidence-based practices, and you were already introducing faculty to evidence-based practices in these programs, is there a different way that you’re presenting this information now to faculty to get them to buy in more to these practices, especially considering time concerns and student resistance and that kind of thing?

Lindsay: Interestingly enough, there are a handful of faculty that I think are resistant to the idea of active learning. The way that we’ve set up at least our Course Design Institute is in such a way that we attend to motivation first, and so we really get very little resistance to the idea of active learning or evidence-based practices. They want to do it. Some of them do do it. They either feel like they can’t do it as much as they want to or they do it and they’re not satisfied with their practice. We really don’t run up into the barrier of, “I don’t believe in active learning,” with the exception of a handful of faculty.

John: And there’s probably not much you can do with those. But I would think working with entire departments might help reduce some of the resistance because when you have that sort of collaboration with the department, it becomes part of the department culture, I would think. How has that been working?

Lindsay: We have had a cohort of faculty within a department go through our Course Design Institute and then another program that paralleled Ignite that was specifically for STEM faculty. And this department really has transformed, so this is about five years ago that they went through as a cohort. The department itself, the culture there is focused on teaching and learning, they continue to engage with our center, we have a recently started SoTL Scholars Program, so Scholarship of Teaching and Learning. We had five of the faculty from that department actually go through this together this past year. They’ve started their own reading group. We’ve been looking at data from the department and we see that student failure rates are going down in their department, particularly for underrepresented students. So working with the departments I think are really, really important and we’re seeing the fruits of that.

John: Earlier Hannah mentioned something about looking at issues of identity in terms of teachers and scholars and so forth. And I would think that perhaps the work you’re doing with SoTL might help unify that. Could you tell us a little bit more about the results you found and how you’ve been addressing those?

Hannah: What we expected to find was that there would be a correlation between teaching and research identities and that if you were high in teaching, you might be lower in research. If you were high in research, you might be lower in teaching. And what we found was that there was no correlation, that you could have both. You could be both an excellent teacher and researcher, you can be really strong in both of those identities, or you could not be. It was all over the place. And part of that is the sample size, and we have since expanded and haven’t analyzed that data yet, but we’re looking into that more.

Lindsay: And to add on to that, so the way that we looked at identity was the idea of how connected you feel with that particular profession. So if you feel connected to the teaching community versus feeling connected to the research community. And we also had a third aspect to that, which was the work identity… so how connected do they feel to the university? What we found was that faculty who had a strong work identity—meaning that they felt connected to the institution—they felt that the department was less of a barrier for implementing evidence-based practices, and they didn’t perceive that they had barriers related to supports. So things like having TAs, classroom space, and things like that.

John: Going back a little bit, you mentioned that one of the barriers that some faculty mentioned was the size of their classes. How have you helped faculty get past that?

Lindsay: We’ve actually had conflicting results around that. So faculty perceived class size as being a barrier to implementing evidence-based practices. But when we look at the actual observations of those faculty teaching, we see that faculty who have engaged in our center use more evidence-based practices, even when controlling for class size. And so what we need to further investigate is how our center plays a role in reducing barriers for faculty. The sample size that we have with our survey results is much smaller, and we can’t really disaggregate. There is something interesting that has to do with class size, and we’re not exactly sure what it is. Whether it’s a perceived barrier or an actual barrier, we’re not quite sure. But I might guess it’s a perceived barrier because we do see more active learning even when classes are large. So faculty are able to do these things, but sometimes they may not think they can.

Rebecca: Or they might not know what practices work at a large scale, because there’s different ways to implement… and so the more we expose them…

Hannah: Exactly. Yeah, because they’re trying to use approaches that require a studio. You can’t do that with a 500-student lecture. So obviously, that particular evidence-based practice is not going to be useful in that case. You can bring in some of these perhaps smaller practices but that are still powerful to get students actively working and collaborating with one another. Think-pair-shares, things like that, that you can still do and then there’s all sorts of work—great work—that’s going on now talking about what you can do with large classes.

Lindsay: And those are the things that we talk about in our Course Design Institute. How do you design your course, knowing that you have particular limitations because of things like class size? Or maybe it’s a required entry-level course, or maybe it’s an upper-level course, or a graduate course. All of those things are really important in thinking about the design.

Rebecca: Or the chairs don’t move.

Lindsay: But we do talk with them about how to deal with that. So in the lecture hall that the seats are fixed and you want to do group work, we have recommended to faculty—if they have space—leave every third row empty, and that way you can actually access students and students can turn backwards to work with people behind that. So we definitely try to help them think about ways to go beyond what they think are perceived barriers.

Rebecca: How to hack your classroom 101.

Lindsay: How to hack your classroom, I like it.

John: And actually, let me put a plug in for one of the Teach Better podcast episodes, which came out in April on the importance of classroom design. We’ll include a link to that in the show notes. The research they were citing finds that active learning helps, but classroom design helps even controlling for the use of active learning. So some of that flexibility is useful. This has been implemented in STEM fields—I think many of those topics that you found would work in other disciplines. Has the teaching center more broadly started to roll out some of these techniques throughout the institution?

Lindsay: I’m going to answer this from a much more broad perspective, thinking about what we’re doing in terms of our programming and supporting faculty. And I think Hannah can talk about the more specific piece around what we’re doing to better gather data around faculty barriers beyond our STEM faculty. So one of the biggest things that I think I mentioned at the beginning that we really are striving to do is use our own local data in addition to the literature to really drive what we do. For us, this goes beyond just doing a needs assessment. This is really doing research around teaching and learning at our institution. One of the pieces of evidence that we found around our prior work is the ways that students engage with each other in class, and how the instructor sets up that group work in class is really important to student success. And so what we are now doing this past year, we are collecting data to better understand not just faculty perceptions of how they design group work, how they assign students to groups, what do they do to assess group work, but we’re also looking at the student perspective. We are actually following students that are working in groups over time, having them reflect on their practice, share audio files and share working documents, to better understand what’s going on in group work. All of that data now we’re using to develop a advanced collaborative institute for faculty that’s going to use not just the literature that’s already published around group work, but also locally derived data that’s both STEM and non-STEM faculty in classrooms. And it’s been interesting because we think about our disciplines being very distinct in terms of  “Oh well,  STEM classrooms are very special, and they need to do these particular things.” As we’ve interviewed faculty, the reasons why they use group work—regardless of their discipline—is very similar. They want students to develop professional skills. I think it’s really important to gather that data to understand this perspective so that when we develop these programs and supports for faculty, we can actually talk about what the faculty are saying and how we use that to improve. So that’s just one example of how we’re broadening this idea of data-driven faculty professional development.

Rebecca: How are you gathering that data about group work?

Lindsay: In our center, I am 50 percent research and assessment and so a lot of my work is around being able to assess our programs, but also be able to gather the data to drive programming. As we said in the beginning, my PhD is in science education. So this is my formal training, being able to do this type of work. So I actually have a group of three graduate students—as well as Hannah and another postdoc—that helps support the research and assessment and center. So for example, as part of that group work study, I had one graduate student who over the course of two weeks, interviewed 19 faculty and over 1000 minutes of interviews that had to be transcribed. I really have a committed group of graduate students and postdocs that help support this work, because they’re really interested in helping make the improvements as well. I don’t think if this was something that was very abstract and not related to helping improve instruction that we would have such buy-in from the people that are helping support this work. So we’re doing interviews with faculty, students are submitting reflections, audio files and documents. So those are the data sources we have right now. We also have syllabi and course documents that the faculty have developed that articulate how they are setting up these group work or group projects.

John: That’s a great resource, I think, for all teaching centers because most of us don’t do that, and it’s nice to see this sort of research. We often talk to faculty about the importance of doing SoTL research, how the classes are working, but teaching centers don’t always do quite as much assessment of how their programs work, and how things are working on their own campuses in this way. So it’s a nice example, I think.

Hannah: Right, and I can talk to the real specific research that we’re trying to do to expand from STEM into non-STEM fields to kind of get more of that research across the university going. So the survey that I developed that has the barriers, that has the identity, that has some qualitative background questions to try to understand where their beliefs come from, all of that. I have been working with STEM faculty and non-STEM faculty now, to expand into the humanities, the arts, the social sciences. And what we’ve been doing is working with humanities faculty at the Center and then I had a focus group this week with several scholars in those areas to talk about the language that we use in the survey. So what I quickly found when we were trying to expand the survey across the university, is that the language that you use is really important. Now STEM faculty, they are fine with the use of the term evidence-based practices. And discipline-based faculty and researchers, we want to see the evidence. We want to know if something works, we want to know that there was a rigorous study that backs up that particular practice, and once we see that, we’re ready to kind of go for it. But when you try to expand that wording into the humanities, that’s not so much a crucial thing for them, they’re wanting to see that things work. The type of research that they do is very different and when we use the term evidence-based practices, the way that they think about that is very different from STEM faculty. So we had to change the wording, we’re modifying the survey, how the questions are asked, the types of words that we use, the assumptions that we’re making. So that’s been my job the past few weeks and will continue because it’s been proven it can be quite challenging to make sure that we’re not alienating a lot of the people that are taking the survey to the point where they see certain words and are like, “This doesn’t apply to me, I don’t want to take the survey anymore.” So that’s been the challenge with this, expanding this from STEM, is the language can be a barrier to people taking the survey and then we don’t get the data that we need. I’ve been working to figure out, “How do we talk about this in a way that we can compare across all of these groups, but still get useful data and not alienate groups within those different departments.”

Rebecca: I think sharing a summary of that information would actually be useful for a lot of centers and researchers too because teaching and learning centers probably also suffer from their advertisements and stuff, perhaps alienating groups of people and not realizing it for the same reason, potentially.

Hannah: Definitely, definitely. And one of the humanities faculty members here at the center and I have been talking about that and may be coming out with a paper once we gather more data on this, on the language that we use. What is useful and what is not useful by discipline?

John: That’s something I wouldn’t have thought of, because we use a lot of evidence-based practices here all the time.

Hannah: Yeah, I didn’t think of it either, and so I was in for quite the shock when I started talking with humanities faculty.

Lindsay: And I think another thing to add in terms of how we’re broadening this work, one of the places that I’ve begun to explore is how do we set up infrastructure at our institution so that we can actually systematically gather data, connect data sources, and then help faculty use that individually to improve instruction. It doesn’t do anybody any good if we gather evidence or research—we do research on our own—and then we don’t do anything with it. And so, we’re developing as I mentioned, our SoTL Scholars program so we can help faculty learn how to do this research on their own. So we are developing a set of tools that we can use to, for example, go out and observe faculty teaching in their classrooms and then from that data, create some sort of visualization that can be used in a consultation. We have a consultation program—many institutions in our centers have consultation programs—but what we really want to begin to do is gather that data in a way that we can begin to represent it on some sort of timeline, where the faculty can see, “Okay, the first 15 minutes of class we did lecture, I asked a few questions here and there, students didn’t answer those questions,” or, “I answered them myself or I moved on too quickly,” and so really honing in on some of those small details that can really help them make tweaks and improvements to their own instruction. So we’re really working at that infrastructural-level now to think about how do we create these tools and set up databases so that we can gather data and share that with faculty.

Rebecca: A follow up question to the qualitative research that you did at the very beginning… What kind of observations you were making for that qualitative research and what you were focused on? What you were looking for specifically…

Lindsay: Good questions. So the original observation study that we did a few years ago, we ended up using COPUS, the Classroom Observation Protocol for Undergraduate STEM. If you’re not familiar with that, COPUS measures the presence or absence of various different types of student and instructor behaviors over two-minute time increments. I was able to train 35 undergrads on how to use COPUS reliably and we were able to gather… for each individual course we observed twice. And we were able to then calculate the percent of time the instructor spent lecturing, or spent doing quicker questions, group work, administrative tasks. And we were recently co-authors on a science publication where the COPUS data were then transformed into profiles and so we were able to then categorize these different classes as primarily lecture—which was greater than 80-percent lecture using COPUS—interactive lecture—which was lecture but it had some clicker questions or some other group work interspersed throughout— and then the third set of categories was around student-centered instruction, so it could be POGIL type classes—so Process Oriented Guided Inquiry Learning type classes—or primarily group work, working on worksheets, doing problem solving, or a variety of different group activities. And so of those, we had 239 classes that we observed. Of those, we were able to classify those classes into those three categories—lecture, interactive lecture, and student-centered—and then we took those classes and organized them based on the intervention that the faculty have gone through. So whether they’ve engaged in our Course Design Institute, whether they’ve done our Ignite program, and we actually had a fair amount of faculty that we observed that have never engaged in our center at all. And so that’s where we were beginning to see differences… that our Ignite faculty, we saw much more student-centered instruction than faculty who had never engaged our center. We also gathered grade data on those classes. Do you want to know about that?

Rebecca: Yeah.

John: Sure.

Lindsay: This is actually a paper that’s currently in review, but the grade data was the thing that was really interesting to me. What we ended up doing is we calculated a DFW rate. That’s D, F, and withdrawal. So basically, failure rate for students in those classes that we observed, those 239 classes. We also were able to calculate failure rates for underrepresented minority students. So those were black, African-American students, and Hispanic students combined together compared to white students in the class. And even when we gathered observations of 239 courses, when you started to look at the courses taught by faculty at the different types of interventions—so that was Ignite, Course Design Institute—and then when you broke it down even further by, “Let’s look at those courses taught by Ignite faculty that did active learning, or lecture, or interactive lecture,” the numbers got very small very quickly. But one of the most interesting pieces that we found descriptively was, when you looked at just courses that were categorized as having student-centered instruction—so active learning, group work, those types of things—the faculty that have gone through things like our Ignite program, and another program called Nucleus—which is similar for STEM faculty—the failure rates between white students and underrepresented students were nonexistent. When you looked at student-centered courses where the instructors had not gone through our Course Design Institute or gone through any of our communities, the failure rates for underrepresented minority students were four times that of white students. Now this is descriptive, this is not anything that’s inferential, but that was one of the driving forces for me that made me realize that we need to look more at group work and what was going on in group work because it’s suggested that when you implement group work or student-centered instruction in your courses and you’re not supported in doing so, you are doing a disservice to your students, and that seems to differentially impact underrepresented students more so than white students. And that was really disturbing to me that we saw those differences on average. This was not the max, this was a mean value. And so that was so important for us to further explore, and we would not have known not had we not done such a large-scale study, and had we not used our own data.

Rebecca: That’s really interesting.

Lindsay: Thank you.

John: You’re making a big difference there, clearly.

Lindsay: We are, and it’s so exciting.

Rebecca: Yeah, I think sometimes we don’t always realize those other kinds of impacts. Or that there could be a difference in the kind of impact that one makes. So I think that’s a really interesting initial discovery to explore, so I’m really interested to see what else you find out.

Lindsay: So we wouldn’t have been able to make those findings had we not been able to connect to institutional data, and so that’s another reason why this developing infrastructure is so, so important, that we’re not going to be able to find meaning if we’re not connecting all of the pieces.

Rebecca: I think one of the things that is really interesting is that you’ve been able to do such robust research at your own institution and have the support to do that. Even how you structured that and how you’ve gathered that would be of interest to many other centers, I think. Sometimes the details of how you arrange that and organize it and how one thing led to another can help other organizations do something similar.

Lindsay: Thank you. I will put a plug in. So in terms of helping other centers be able to do this type of systematic research assessment work, we had a PODLive! webinar on Friday, April 26. If you’re a POD member, you should be able to access this through their website to see what we talked about and what questions we ask ourselves as we go through the process of thinking through measuring impact.

Rebecca: Great. We will make sure we link to that in the show notes and let people know how to access that.

John: We always end with the question, what are you doing next? You’ve already described some things, but we’ll still ask anyway.

Lindsay: So if you can’t tell already, I’m really passionate about data… using data to help drive what we do to improve teaching and learning. And so the two sort of big things that are next for me are really trying to build the infrastructure so that we can liberate data and be able to use data meaningfully, respectfully, and purposefully to help improve instruction. And also being able to help empower our faculty to be able to do research on teaching and learning in their classrooms… so trying to expand our SoTL Scholars Program, and developing further supports in that area. So that’s what’s next for me.

Hannah: And for me, I am working on a couple projects related to the barriers work. So we talked earlier about the humanities expansion, so developing a survey instrument that can be given across departments. So I’m continuing to work on that, work on the language that we’re using, making it relevant to them. And then we’ve got a national study that we’re trying to work on. So we have implemented the pilot—which is what we talked about today, the results of that—and then we implemented a second one also at UVA, but much larger. And then we’re wanting to now expand this and do a national study because the real beauty of this instrument is that it’s not just for us at UVA, it is meant to be a tool for any university, any department to be able to use. And one of the findings that came out of our study was that the barriers are different by department. The barriers, the use of evidence-based practices differs by department… it’s not just the university being different from another university. It’s the department being different from another department at a different university. And so this tool allows any department, any university, to give this to their faculty and see contextually, what are the barriers for these faculty? Now you look across the board, time is usually the highest barrier, but what comes after that differs by department. If there’s particular issues with one department, one university with the teaching-research balance at a given university, all of that’s going to be different. And so the beauty of this instrument is let’s look at a variety of types of universities, types of departments, let’s try to understand what is useful, what are supports, what are barriers across different institutions, across different departments. Try to look for where are there trends and where are there not trends. Where is it just entirely dependent on a given context and where do we see maybe some trends in tenure-track faculty versus non-tenure-track faculty, general faculty, things like that. So we’re really hoping to dig into a much larger sample in the coming year and investigate this further, and I will say that there are a couple of other researchers who are also working on this. So this is an up-and-coming area of research that you’ve got Megan Bathgate at Yale, you’ve got Emily Walter at Cal State Fresno, they’re both doing studies along this idea of barriers and supports for faculty using evidence-based practices. So, I just wanted to put a plug in that we’re not the only researchers doing this. There’s a lot of great work that’s going on and I think this is an up-and-coming area to really help support moving higher education forward and transforming higher education, ultimately, by understanding how can we help our faculty implement more of these practices that we know are going to support our students better?

Rebecca: Great, sounds like a lot of exciting things coming down the road for us to take in soon.

Hannah: Definitely.

Lindsay: Yes.

John: Thank you for joining us. This was a really interesting discussion, and I think many of us will reflect on it in our teaching centers.

Rebecca: Yeah, thank you so much.

Lindsay: Well, thank you, appreciate it.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.