Category: evidence-based practices

395. UDL 3.0

Posted in accessibility, classroom community, design thinking, engagement, equity, evidence-based practices, higher education, inclusion, learning, pedagogy, teaching, universal design
May 28, 2025

The Universal Design for Learning (or UDL) framework is based on research on how students learn. In this episode, Tom Tobin, Lillian Nave, and Jennifer Pusateri join us to discuss the most recent evolution of the UDL guidelines.

Tom is a founding member of the Center for Teaching, Learning, and Mentoring at the University of Wisconsin, Madison, and the author of the forthcoming book, UDL at Scale: Adopting Universal Design for Learning across Higher Education, as well as Reach Everyone, Teach Everyone: Universal Design for Learning in Higher Education and several other works related to teaching and learning. Lillian is the Faculty and Educational Development Specialist for the Center for Excellence in Teaching and Learning for Student Success at Appalachian State University’s Hickory Campus, a senior lecturer in first-year seminar, and the host of the ThinkUDL podcast. Jennifer is the Senior Universal Design Consultant at The University of Kentucky and has served as the co-chair of the international UDL in Higher Education Network. She is a member of the CAST National Faculty and is the author of Transform Your Teaching with Universal Design for Learning: Six Steps to Jumpstart Your Practice. Tom, Lillian, and Jennifer often serve as keynote speakers on UDL and other issues.

Show Notes

Transcript

John: The Universal Design for Learning (or UDL) framework is based on research on how students learn. In this episode, we discuss the most recent evolution of the UDL guidelines.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Tom Tobin, Lillian Nave, and Jennifer Pusateri. Tom is a founding member of the Center for Teaching, Learning, and Mentoring at the University of Wisconsin, Madison, and the author of the forthcoming book, UDL at Scale: Adopting Universal Design for Learning across Higher Education, as well as Reach Everyone, Teach Everyone: Universal Design for Learning in Higher Education and several other works related to teaching and learning. Lillian is the Faculty and Educational Development Specialist for the Center for Excellence in Teaching and Learning for Student Success at Appalachian State University’s Hickory Campus, a senior lecturer in first-year seminar, and the host of the ThinkUDL podcast. Jennifer is the Senior Universal Design Consultant at The University of Kentucky and has served as the co-chair of the international UDL in Higher Education Network. She is a member of the CAST National Faculty and is the author of Transform Your Teaching with Universal Design for Learning: Six Steps to Jumpstart Your Practice. Tom, Lillian, and Jennifer often serve as keynote speakers on UDL and other issues. Welcome Jennifer, and welcome back, Tom and Lillian.

Tom: Thanks a bunch, glad to be here.

Lillian: Very glad to be, again, with you all.

Jennifer: Yeah, excited. My pleasure.

Rebecca: Today’s teas are:… Tom, are you drinking tea today?

Tom: I am, I am coming to you with some peak leaf cold brew iced tea when all you want is just the tea. [LAUGHTER]

Rebecca: Love it. Lillian?

Lillian: I would like you to notice I’m drinking in my Tea for Teaching podcast mug that John gifted me at a conference, and I brought my organic hibiscus or hibiscus, depending on if you’re from the south or not, my hibiscus tea bags.

Rebecca: And how about you, Jennifer?

Jennifer: So, I’m not a tea drinker, guys. I drink a coffee in the morning, and then the rest of the day I am team water. So I’ve got my emotional support water cup with me here today.

Rebecca: I do like to always point out that water is the foundation of tea. [LAUGHTER]

Jennifer: That is true.

John: And I am drinking a pure peppermint tea because it’s been a long day filled with caffeine up to this point.

Rebecca: And I have my last cup of London Strand. I’m at the end. It’s very sad.

John: You may have to head back there again.

Rebecca: I may.

John: So we’ve invited you here today to discuss the UDL 3.0 guidelines that were released on July 30, 2024. But before we discuss the changes that have occurred, could you describe what the UDL framework is and what remains the same?

Jennifer: I’ll give it a shot. So the framework of UDL is essentially a collection of, gosh, I think there’s over 1200 different studies that have kind of been boiled down into these nine categories, and those categories are organized in a grid format. So you can kind of look at it in lots of different ways, but essentially this framework, the UDL framework, tells you what research says works in teaching, and it tells you about that in nine different areas. So three of those have to do with engagement. Three have to do with representation, or how things are presented, and the other three have to do with action and expression, kind of how people are taking action with and showing what they’ve learned. So that’s how the framework of UDL works, and it’s been updated many times. There was an update that I remember in 2013. There was another one in 2018, but the move from the 2013 to the 2018 version, the words didn’t really change, but the order of some of the guidelines changed around a little bit. So this is the first time in quite a while, and this new version, the 3.0 version that came out here in the summer of ‘24 this is the first time we’ve seen some changes to the actual words inside of the guideline themselves. And there’s actually quite a lot that’s the same. I don’t think anything was removed, per se. Some things were combined. So they took two that might have been similar and combined them into one consideration, which is what we call the bullet points that you see if you’re looking at the framework of UDL. So many were left the same. And then we also had some brand new ones, and those are really scattered across the framework, and a lot of them are bringing into focus some other things we hadn’t really talked much about in UDL. So I’m excited to learn a little bit more about that from my colleagues today, but I always love to talk about the framework of UDL, so I’m here for it. And for a webinar that Lillian and Tom and I did last week, we put together a crosswalk document that will show you what things stayed the same, what things are new, and maybe some things where it’s been changed, but it’s close. So we have a whole document laid out for that, and we will be happy to put that in the show notes.

Rebecca: Always like a good resource around here, that’s for sure. So as Jennifer just noted, both the current and earlier versions of the UDL guidelines include things related to multiple means of engagement, representation, and action and expression, but there were some revisions to the language about how these are described. The earlier guidelines asked faculty to provide each of these, and the current guidelines remove the word “provide,” and have shifted the focus to learner agency. Can you talk a little bit about what motivated this change and what are some ways of building learner agency into course design?

Tom: Rebecca, this is an excellent question, because one of the reasons why we moved into a new version number with universal design for learning is that, in earlier versions, we assumed a command environment where the instructor had all the skill, the authority, the deference and the learners were just there to sort of bask in the glory of the knowledge of the instructor. And that’s not necessarily the case. I mean, if you’re teaching elementary school students, you act in loco parentis. At the same time, in higher education, we’re working with adult learners, and they are their own folks. They’re there for their own reasons, and we don’t have any legal obligation to treat them in a power dynamic. So when we think about the shift to learner agency, there was, for many years, a call among practitioners to say the designers and instructors are not the only people with power here, and the designers and instructors are not the only people who are engaged in the learning process. So it’s not, “Hey, we design it and then you lucky people get to experience it,” but we should be making space to help learners understand their own learning process and then build up into agency. So the goal is not to create expert learners, but now they’ve actually shifted that language into learner agency, or agentic learners. In colleges and universities, a lot of our learners already have some experience under their belts, so there is much less of a need for an expectation to establish a command environment, like I was saying a minute ago. Now, that actually lets us design better for the goal of agentic learning for Universal Design for Learning, since we are in the business of preparing our learners to join us as professionals in our fields. And if you want to take a look into one of the new guidelines, you’d watch for language in the 3.0 version that talks about co-creating, and we can apply that broadly where it doesn’t yet appear. So Jennifer talked earlier about there are three big principles of universal design for learning. So design multiple ways to get engaged, to represent information, and to take action and express yourself. Each of those three big principles is further divided into nine guidelines. And each of those nine guidelines has several considerations underneath it, specific practices that people can do. The new consideration 8.3, “foster collaboration, interdependence, and collective learning,” it explicitly talks about learners creating their own learning conditions. But one of the older considerations, 2.1, which is “clarify vocabulary, symbols, and language structures,” it doesn’t yet do that. So if you want, listeners, you can search for research on social construction of learning, with colleges and universities in your search terms, and you can kind of address that gap. Similarly in how we design is there’s an assumption in Universal Design for Learning that learners’ brains are still developing. The UDL came out of the K-12 environment, and if you think of maybe a mom and her grade-school-age son, they’re sitting at home, they’re both working on their homework, she’s in college, so she’s on her laptop; he’s in grade school, so he’s got a pencil and a piece of paper. and we need to support developing executive and emotional aspects of learners in K-12 environments. But for college, we kind of have to shift our focus from developing brains toward social-emotional elements, and we see some of this in some of the newer UDL considerations. So the research that’s cited for consideration 3.2, it’s one of the old ones, that’s “highlight and explore patterns, critical features, big ideas, and relationships,” right now, the research that is cited for that doesn’t contain any higher education examples, so we’ve got about 15 years worth of studies like Jen was talking about earlier. And to address this, we can kind of move beyond the simple neurobiology argument of UDL toward a more social constructivist mode of practice. So we’ll design options specifically so that we can help reduce cognitive load, dissonance, and environmentally interfering conditions for our learners. So watch out in the new 3.0 guidelines for language that talks about emotions, self-regulation, and review the research page of those considerations to see what sources were used to develop them, and you can supplement your search with keywords from those considerations, plus college, university or adult learning in your research databases. So there’s a lot to unpack in the new guidelines.

John: The new guidelines also emphasize issues related to individual identity and building on the strength of the diversity in our classes. Some faculty, though, especially in STEM fields, are somewhat resistant to that. They’ll say, “When I’m teaching students how to work with differential equations, identity doesn’t matter.” How can we convince faculty that perhaps they should at least consider the diversity of students in their class and build on that diversity?

Lillian: Well, I’ll take that one, John, and let’s not give, and you’re not doing this, give the STEM professors a bad name or anything. But I do hear this a lot in a lot of my conversations with faculty, and I think it’s just not really broadening enough… broadening our definition of what identity is. It’s really not just the things that we can see, but recognizing that all of the students in our classroom are different, and identity can take many forms, and that’s the reality that we have in our classrooms. More and more students are coming to college than have ever gone before from different backgrounds: We have first-generation students, English language learners, college prep students, and students who’ve had really no college prep, so neurodivergent students, students with disabilities, etc. So it’s really a matter of understanding who’s in your classroom and taking down barriers to learning for all of those students. So all of your students are going to have some strengths. Some are going to be really stem focused. They’re going to have a great math background. They’re going to be ready to go into your physics lab with really no problems. And other students might come in with a lot of other maybe verbal strengths, reading comprehension, a lot of vocabulary, but need a little bit more structure and background with some of the math as they move into your physics lab, okay? So we’ve got lots of challenges and opportunities for all of them. What does that look like? Well, I can use the example of my daughter, who was a typical college student and would have shown up in many of those STEM classes, because she was a STEM major, and she was kind of the typical, you know, most college students now are young women… [LAUGHTER] that’s the majority… are girls. And by the way, she gives me full permission to share this story. And thinking of all of you college lecturers and professors out there, how many of you have had a student named Emma? Probably 100% of them. So she’s in your class, but what you don’t realize is that there are lots of things that are part of one student’s identity that you don’t know are making it harder or are barriers. So being a girl in a STEM class is sometimes just a little bit awkward, when even though they’re the most of the students, they are less represented in STEM classes. So when she was in a physics class, she was in the minority. No big deal. But when you keep adding on a few other things, like the fact that, as a left hander, she really couldn’t write as fast as everybody else, and so she needed a little electronic iPad to work on there. That’s number two. And then the fact that she’s neurodivergent in two ways, with ADHD, anxiety, that made it a little bit more hesitant for her to go up and ask questions when she was feeling far behind. And then add on to that, a chronic condition. So she has to get immunotherapy and go to a clinic. That means she has to leave campus. She has to get those set up even before the semester starts. So, too bad it conflicted with her physics class because she didn’t know which section she was going to be in. So all of those things are things that created barriers for her, and I’m not saying that the faculty member needs to know all those things, you don’t, but those are the kinds of things that are part of your students’ identities. So if you can make sure that you have accessible slides, that if a student has to miss for no fault of their own, they can get that information, then you’re doing a service to your students. In fact, you’re just making it accessible for your students who can’t be there. If you’re making sure that everybody knows, really well organized, what the material is, and that they can go back and look at things that were building up to that particular date, if you have to miss, then those are just other ways that you are supplying the information to all of your students, no matter their identity, if they have a lot of background or a little bit of background, etc. So that’s the kind of thing that we’re talking about. And it’s not just accessibility, but it’s just thinking about you’re going to have students that are coming at your material from a different direction, and they are going to be better served by a variety of ways to get to the end point, not just the one way that you’re the most comfortable with. Because every student is different. Every faculty member is different. My favorite saying that I use all the time is we really don’t want the same students to be uncomfortable all of the time, but we want all of the students to be uncomfortable some of the time, and so if we can spread that out, we get all of the students into that learning zone where they risk just a little, but they’re not always in that risky fight or flight mode, then we’re better serving our students. So that’s what I say about identity for our students, it does matter just because everybody’s different.

Rebecca: Sounds like we also need to think about as faculty also sometimes having that experience of being uncomfortable like that goes… [LAUGHTER]

Lillian: Yes, exactly.

Rebecca: …not all the same faculty need to be uncomfortable all of the time. So building on what you were just talking about, Lillian, and what Tom was mentioning before about the social aspects that are now in UDL. Can you talk a little bit about some of the ways in which our courses can more effectively foster a sense of belonging?

Lillian: Yes, yeah, so I’ll go all in on these two questions right now. So we’re all in this learning together, and all of our students and faculty have a relationship, whether you realize it or not, it might be a terrible relationship where students are afraid to come into class because they’ll get called on and they’re anxious about it, and they’re like dreading two o’clock on Tuesdays and Thursdays. That’s a relationship that you may or may not know, but it also could be a very warm and inviting relationship. I know that I’m going to go into class and there’s going to be something interesting and exciting and feel like they’re valued or ready to learn something new. So anyway, everybody’s got some sort of relationship. But how can we leverage, really, the design of the course into learning together and making it really a social learning opportunity? There’s lots of different ways to do that. So you can do that in person, online or hybrid. For example, if you have even a large in-person class, a way that all of the learners can really understand that they’re in the same place, they’re all learning together, is if you were to use, like an in-person or synchronous poll, and it can be completely anonymous, and then the faculty member could really determine, like, where are we as a class? There’s no points associated with it. It’s completely anonymous. So, could throw out a concept and see where people are, maybe on a continuum. Or how confident are you in this theory? Do you think you could solve this problem? And then the students could be like, “Oh, it’s not just me. About the third of the class thinks we need to review this again.” That’s just one way that everybody in the classroom is understanding their learning together. And it doesn’t have to be like, throw everybody in a group. You don’t have to do that, although that’s also helpful, but that’s just like, one of those things. In an online class. I love using a Q & A forum, so this is different, but this is understanding or seeing how somebody else in the class may solve a problem differently than you, but you want all of your students to attack this problem, and you don’t want it to be a discussion board where the first person answers, and then everybody else says, “Yeah. I was thinking that too.” Instead, this is something where everybody has to come up with their own answer, or way they would solve the problem or their thoughts on it, but they don’t get to see anybody else’s until they’ve answered. You have to wait 15 minutes after you answer to see if anybody else has and there’s some great mechanisms to do that. And that could be in smaller groups, it could be in the whole class, but we really do learn a lot and have a much more rich and more nuanced understanding the material when we see that other people are learning that, like I can read a passage and get something out of it, Jen can read the same passage and see something completely differently. Tom will see it a whole ‘nother way and pick up different things, and I’m a much better learner when I see that rich and nuanced perspective of my classmates. So there we all are learning together. So you can design for group learning, whether it’s hybrid, in-person, online, if you give also really good roles for any group work you want them to do. Don’t just throw them in the group and say, “have at it.” Is there going to be someone who’s the convener, who’s going to be sending out the emails? What are the roles? Is there an editor in the group, or what are they supposed to be doing? And also, what sort of document are they creating? Is it accessible, so that all of those students can really find what they need to do. So giving really good instructions on those things can help people learn together. And finally, I think the last thing is, have fun: games, quizzes, review days, trying things out together, making it sort of fun in learning is another way, I think, that we can bring that sense of community into the learning process.

John: One of the things you mentioned was doing some sort of a poll at the beginning of class, or gathering information about the identities of your students. One useful resource for that is the “Who’s in Class” form that was put together by Tracie Addy, Khadijah Mitchell, and Derek Dube, and we’ll share a link to that in the show notes, as well as an article they wrote about that. One of the other things in the new guidelines is that there’s an increased focus on addressing biases in the use of language and symbols, as well as biases related to modes of expression and communication. What are some of the ways in which these biases may appear, and how can we address them in our classes?

Tom: This is actually one of the big shifts in Universal Design for Learning 3.0, is that there’s an increasing awareness of and a need to create space for the intersectional identities or characteristics that learners and instructors bring into learning spaces. So those kinds of biases can be often unintentional ones. So for example, physical assumptions. When I’m giving a webinar and I say, “as you can see on the screen,” well maybe everybody can’t actually see what’s on the screen, so I’ve taken to doing visual description, or just using ableist language. When we think about something that’s really difficult, we say it was crazy or it was insane, and those are small but telling, unknowing ways of saying, “Yeah, if you fall into one of these categories, maybe this is not for you, or you don’t belong here.” Our colleague, Ann Gagne has a wonderful approach to addressing some of these biases, is using very neutral language in very charged or meaningful ways. So if you had a challenging day and someone asks you, “Hey, how did things go?” You can say “it was bananas,” and Ann, in a recent webinar, she used that kind of language, and she talked about one of her colleagues just saying giraffes. It was like a herd of giraffes, and it’s evocative, yes, ableist less so. When Jen and Lillian and I held a webinar for the Professional and Organizational Development Network Accessibility and Disability special interest group, or the POD A and D SIG for short, recently, we each got to talk about one of the new Universal Design for Learning considerations. The one that I was facilitating with some colleagues was 6.5: “challenge, exclusionary practices.” And this gets to your question, John. We can kind of examine power dynamics now within universal design for learning in a way that it wasn’t explicitly included before. So when we’re thinking about power imbalance, the people who teach in higher education, we are not ourselves a homogeneous group. We experience and express privilege and power in varying ways. So the same inclusive practices that white male instructors like me might use in order to lower barriers and smooth the way for learners, those might result in pushback and mistrust when they’re implemented by younger colleagues, women in the classroom, people of color who are at the front of a classroom. So we should acknowledge the privilege and standing that are needed in order to implement many of the UDL considerations. And when we design for variability, we should do so thinking about the variability, not only of our learners, but of the instructors who are going to be in communication and community with them. So when we’re thinking about biases, we can think along lots of different vectors: we can think about socioeconomic, we can think about gendered bias. There’s lots of those that listeners, you are probably familiar with. And when we think about Universal Design for Learning, just as we’re thinking about the variability of the learners, we also should start thinking about, “Okay, who has the privilege to be able to do these things and facilitate them well in the classroom. So that’s one piece where I’m hoping to see more research come out. We’ve actually got a number of different researchers who are working on these kinds of questions as their doctoral work over the last couple of years and into the future in the next two or three years. So I’m excited to see some of this research come out.

John: We should note, Tom, that you wrote an article with Chavella Pittman on this issue a while back in The Chronicle, and we had a podcast discussing that as well, and we’ll share links to both of those in the show notes. So, when talking about agency and power dynamics, the new guidelines seem to provide a bit more focus on the development of student goal setting, executive function, and metacognition, while also recognizing the individual, institutional, and systemic barriers to learning. What are some ways of incorporating this into the design of our courses?

Jennifer: Yeah, well, John, one of the biggest changes that we saw in the framework of UDL from the 2.2 to the 3.0 was moving executive functioning, which was a part of the previous version, but it was only one of the nine guidelines. So in this iteration, they have expanded that to be the entire bottom row, and so that’s including several different pieces of executive functioning. And that whole row now is labeled as executive function. So one piece of that would be emotional capacity. That would be things like emotional regulation or motivation, even inhibition and metacognition are part of that emotional capacity piece. It also includes the building knowledge guideline, and that used to be called comprehension, but is now renamed to building knowledge, and that piece is really going to be talking a lot about the information processing and the working memory piece that is part of this umbrella term, executive function. And then the last is strategy development, and that’s the one where we get into those, what people typically think of as executive functioning, goal setting, planning, organization, time management, that kind of stuff, all falls under that category as well. And I’m really glad that this is the way they have restructured this, because I’ll be honest, I ran into a lot of questions from folks in higher ed and K-12 as well, because previously there were two guidelines. One was executive function, and then the other one was self-regulation. And so I had a lot of people saying, well, aren’t those kind of the same thing? And I don’t want to go down too much of a rabbit… well, that’s not true. I do want to go down a rabbit hole… but I’m going to refrain from going down said rabbit hole. But it’s really interesting to me that now we’ve kind of acknowledged that, “Okay, yeah, the self-regulation part is a piece of executive function.” I think that’s why they went through and renamed some of these on the bottom row. And these are really important. We’re noticing, at least at my university, and I would guess at others as well, that we have quite a few more neurodivergent students than we used to have, and some estimates would say even up to a third of our students could be considered as neurodivergent. And for neurodivergent students, that executive functioning row, that bottom row of the new 3.0 guidelines, is a game changer, because a lot of the pieces that we forget to mention in higher ed kind of that hidden curriculum that we might talk about, or that hidden structure, those things are made explicit through that bottom row of the guidelines. So, for example, if we wanted to think about strategy development, we could be looking at something like walking students through what journal of articles or academic articles look like in your field and how you read those. So, I’ll use myself as an example. I did not know until my PhD program that you don’t just start at the beginning and read all the way through an article to the very end, like front to back. In my mind, that made sense, because that’s how we read everything else. But as it turns out, my professors in my education department were telling me, “Well, there’s kind of a different way you can go about this. You can start with the abstract, read through that, see if it’s still within the realm of what you want to know. Then you can kind of skip to the discussion part at the end. Read through that a little bit, figure out if that’s what you need to know. And then, if you’re still interested and still want to use this article, then you can read some of the other pieces.” Well, I had no idea. Like no one had ever said that. So why would I know that? So when we can pull those kinds of things out and make them clear and explicit to our students, transparent is another word that we might use for that, and that reminds me of the work of Mary-Ann Winklemes in transparency, in learning and teaching, TILT. You may have heard of TILT Higher Ed or transparent assignment design, and those are really all kind of doing the same things. We’re taking the guesswork out of our teaching and learning, and that’s, of course, going to help our neurodivergent students, but that’s going to help a lot of students as well, first-gen students, international students. And if we’re able to do that and make those things really clear, then we’re going to help a lot of students. We could look at another example, if we’re thinking about the, and this is in the building knowledge guideline, so 3.2 is highlight and explore patterns, critical features, big ideas and relationships. There are lots of ways that you can do this, but one way I really appreciate, because I kind of lean toward visuals, is having a concept map, or some visual way of connecting the information to the other information, like connecting it and helping me to orient myself within both the thing that we’re learning about, but like, what am I learning about there, in relation to the whole course? So being able to kind of see those relationships laid out in a visual way is also going to be really useful. And then Lillian actually talked already a little bit about this idea of group work and how we might set that up. And there are several considerations that talk about this kind of idea of, like, working together in some way. And one of the ones I really like a lot in the emotional capacity guideline is talking about cultivating empathy and restorative practices. And one way, of course, we can do that, and Lillian actually kind of mentioned a little bit about this as well, is helping students to reflect on, like, what is actually helpful for me when I’m in a group setting? Like, what kinds of things make me feel comfortable enough to share my thoughts and ideas? What kinds of things get in the way of that? Like, what are some of the barriers that I encounter in a group setting? And having students actually work through that, kind of process that information and process their own metacognitive view of themselves working in a group before we get into a group. And if we can do that first and reflect individually, then we can come together as a group, and now we can start to build this trust framework, this sense of belonging, because I can now share those things. I can say, “Gosh, when I’m in a group, I don’t always talk first like, I’m just not usually that person. But if someone is like, ‘Jennifer, what do you think about that?’ And they ask me, then I’m going to contribute to the conversation.” So things like that really helped me in a group setting. And if I can say that to my group, now they can understand, perhaps, why I’m not jumping into the conversation immediately, and that builds that empathy. And then we can also figure out what are some things that I can make sure I don’t do so that I’m supporting my other group members as well. So I’ve actually got a really good group contract activity kit that I put together for one of our workshops at the University of Kentucky, and I would be happy to put a link to that in the show notes as well. And this actually has kind of that breakdown. First, there’s an individual reflection, then you get together in your groups, and you do a group discussion with your reflection, and also you put together a contract, and it kind of walks professors through how they could go about doing something like that. So I will make sure I get that in the show notes.

Rebecca: That sounds awesome. As we’re thinking about changing times, we also can think a little bit about generative AI and how that might support the UDL framework in its new form.

Tom: It’s a wonderful question, and I had the pleasure at the Online Learning Consortium OLC conference of being on a panel on UDL and AI with the two of you. So I’m going to not actually recycle some of the ideas we talked about there, but I’ve got some new ones. I want to talk about a lopsided house and listeners. I’m going to describe this for you. There will also be a visual that we’ll put into the show notes. I was recently talking with Marcus Popetz. He’s the person who runs Harmonize, the geek company, and we’re going to be putting together a webinar later on, on universal design for learning and artificial intelligence resistant assessments. And if that sounds like I have my wooden shoe ready to throw into the cogs of the machine, quite the opposite. So the lopsided house is an idea that I’m currently playing with. Imagine a rectangular diagram so time moves from left to right. On the bottom, there are learners who are beginners, people who don’t know yet, and they’re just starting to learn a concept or a skill or something we can teach them. In the middle are proficient learners, people who have enough to actually start doing the things in our field. And then over at the other end of time is practitioners. These are learners who are para-professionals. They’ve got the skills, and they probably their next step will be becoming colleagues of ours. Now in that rectangle, draw a diagonal line from the very bottom of beginner all the way over to the very top of practitioner. And what you’ve just done is you’ve created an artificial intelligence tool use diagram that matches up with what we might call designed UDL. So when someone is a beginner, we want to, as designers and instructors, provide them with and there’s that provide word that UDL doesn’t use anymore. We want to provide optimized choices for learners in their learning experiences, and we want to do a lot of that, and we want to be a little heavy-handed with it, right? We’re the experts, and we kind of know how to guide people through those beginnings. Also, at that beginning stage, there’s very little use of tools. We’re helping people to understand the foundational ideas, get them practice, even if it’s the long way around or the manual way to do things. For example, I teach writing courses, and my freshman writers, I want them to create paragraphs that have a thesis statement and details evidence and examples that then help to support that thesis. This is not the time to let them loose on Claude or Copilot or ChatGPT. This is the time to say, “Please don’t use those tools, because I want you to learn the foundations.” Now move forward in time, and it’s about half and half. As people get more proficient as learners, we take away some of that designed UDL, we actually give them more agency. And here’s where you can draw another triangle on top of that, it’s kind of a lopsided house, where as we reduce the amount of designed universal design for learning opportunities, we start increasing where we’re asking the learners themselves to learn more about their learning process and be agentic, to take control of and show where they want to go with their learning. And when you’re more proficient, you can use tools to start working on things like giving study questions to generate off of a reading. A bad use of artificial intelligence… I’m here with Stacy Truex when she says “You can’t ask it to do something you don’t know how to do yet.” So give me a summary of the reading… bad idea. Give me some study questions from the reading and then quiz yourself on the reading that you’ve done…very good idea in terms of asking proficient learners to use artificial intelligence tools. And then when we move all the way over to sort of the end of the timeline, where people who are our learners become more like practitioners. They become paraprofessional. That’s when we really ramp up the tool use. That’s when we let them loose and we say, “Hey, artificial intelligence tools, that can help you skip the foundational steps… Now you know, now you have enough skill and knowledge to skip those steps so that you can focus on higher-order complex thinking.” And this is also the point in people’s learning careers where we want to have the least amount of designed UDL. We’re not guiding them anymore. We want them at that point to have the greatest amount of agentic, universal design for learning knowledge. And when we’re thinking about examples here, I’m thinking about study question generation, but also creating alternative formats of things. “Hey, this document is only a PDF, can you make a read it out loud to me version of that?” Artificial intelligence tools are really good at making alternative versions of existing content and materials. And the same thing when we’re thinking about as instructors or designers, because we have that expertise, artificial intelligence tools can help us to create multiple means of engagement, multiple means of representing information very quickly. You want a question bank that is tied to each of your learning objectives and segments things into question pools of three questions or more. Bang, couple of minutes, and it’s there, and then you can review it from your expert perspective. Creating quizzes, everything kind of speeds up when we’re using artificial intelligence. The key though is, in order to use those tools well, we’ve got to know what we’re talking about first, whether we’re learners, designers, or instructors. So I want to leave that sort of as a provocative thing to hang into the conversation. But AI is going to help us with universal design for learning in lots of different ways.

Rebecca: One of the things that you just underscored in your lopsided house, but also a little bit earlier, is how you might use UDL in the way that you’re designing a class as an instructor a bit differently depending on the expertise of the students in the class. You pointed that out in the difference between maybe K-12 and college, but then again, between this like beginner and expert. Can you talk a little bit more about the different ways that instructors might engage with UDL in really thinking about the expertise of the students in the room when we’re kind of thinking about level?

Tom: Yes, and I’ll be really quick here, and I want to open it up. I know Jen has done work on this as well. When we’re thinking about the students, we’ve been actually engaging in a little bit of a heresy in our conversation today, and that is, strictly speaking, Universal Design for Learning is what we do when we design learning engagements before we ever know who’s going to be in our learning spaces. So whether it’s a classroom or it’s a librarian working with students, or it’s a mental health counselor or academic counselor helping students navigate systems, the design part is what we do before day one of the class, before anybody ever walks through the door and says, “Can you help me?” Then, when we start noticing patterns among our learners over time, we can take that into account. We’re starting to see many more learners who are first-generation college learners, or students who don’t speak English as their first language at home. Or we’re seeing a lot more folks who identify as neurodivergent. Those attributes are things that we can now plug into universal design spaces, but technically speaking, we don’t know who the learners are yet, so we have to assume that there’s going to be wide variability among their characteristics. That’s what UDL is all about. It goes really well with pattern recognition and what we in K-12 call differentiated instruction. I want to make a pause here and see if Jen and Lillian can take this further too, because it’s a cool question.

Jennifer: So, I do a lot of work, both in higher ed and in K-12, and in both of those areas, one of the things I tell people is that UDL is for when a human brain is in a learning situation, and that doesn’t necessarily matter if that’s for preschoolers or if that’s for undergrads, or if that’s for postdocs, if that’s for elderly, it doesn’t matter. If you’re in a learning situation, the framework of UDL should help you to be able to meet the variability, and support the variability, of your learners. So that’s one piece. Having said that, I don’t remember, Tom if you went a lot into this, but I know we talked about this the other day in our webinar. You know, currently the UDL framework is still a little K-12 heavy, and actually we’ve actually got a project going, and I’m gonna throw that in here real quick while I’m thinking about it. So if you’ve ever gone in and looked at the CAST studio guidelines website, and you’ve drilled down to one specific consideration, so let’s say options for choice and autonomy is a good example. And you’ve gone to that little page where it tells you a little bit about this is what this means and here are some sort of generalized ways you could think about doing that in a teaching situation. There’s also a place on that page where you can look at the research that was used to put that consideration in place. Because, as I’m sure everyone knows, in the field of education, we don’t just base our educational and teaching decisions on one study. We’re basing this on a body of research that is built up over time, so that we know, over time, we’ve seen that giving students choice and autonomy is going to help with engagement. Engagement, right? So that’s what this list of all of these different pieces of research and studies are about, and most of those are focused on K-12. So we were really wanting to put together a version of the research per guideline, and even getting down to the consideration level, or the bullet-point level, if you will, that also has some research for higher ed folks in it as well. So we have put together a Google form, and we will get that in the show notes as well. But I can just throw out this quick Bitly, if you’d like. It’s Bitly, that’s, B, I T, dot, l, y, slash, UDL, dash, H, E, which stands for higher ed, H E, dash research. And so basically, we’re wanting you to think about, do you have some like, go to research that you share with people a lot that aligns with UDL? And if so, if that’s higher ed focused, we would like to know what that is, so we can start to build up our own body of research around the guidelines that really is focused in on higher ed a little bit more. So having said that, there’s a lot of ways that you can do this, that you can build UDL in. When I think about our undergrads, Tom talked a little bit about executive functioning earlier, and for the most part, yes, we’re done cognitively developing when we enter college, but we do know that there’s still some parts of your brain that are cooking up to age 25 for most and up to even 30 and 32 for others, and that’s our executive functions. So with our younger students, for sure, but honestly, with almost all of our students, really focusing in on the executive functions piece is a good place to start, because we can just go ahead and assume that our students are going to struggle with executive function. Part of it is because of their age, but also we know that this is one of the most anxious groups of students we’ve ever seen in higher ed, and anxiety, stress, trauma, all of those things compromise our executive functions. So we can assume today that you will have students in your class that are going to struggle with executive functioning. And so I think that’s a really good place to start. And some people like to start with the top row instead, because this top row is really kind of your entry point into thinking about UDL, and it would be changing kind of the concrete things in your class. So thinking about, yes, they have a paper textbook, but can I also provide some kind of a digital format for that as well, or is this something I can maybe give to them in a visual that might make sense in that way? So you’re kind of thinking about those concrete pieces of teaching, learning, and the materials and all the things, and that’s also a really good place to start, is looking at that top row. And I’ve actually seen somebody put together a version of the guidelines where they put a little welcome mat above that and said, This is your welcome mat into the UDL framework is starting at the top and going down from there. And that’s my thoughts on that.

Lillian: In addition to the differences that UDL can be used in K-12, and then also the difference between a freshman seminar, which is what I teach, and what you might use in a senior level or grad course, is thinking about UDL can be used and should be used everywhere, because we’re dealing with people, and people are different, and everyone is unique. And so what it might also look like is thinking about it in workforce development. So not just are you providing multiple ways for your students to learn, but if you have a company, are you providing multiple ways for your learners that are in your company to move forward, to learn new skills, but also to do the things that they’re doing? So do they have options for working off hours or in remote settings? Or do you have a human- centered approach to doing whatever it is that you’re doing, so that there might be quiet spaces, or just the way that everybody can work to their strengths, so that you don’t push aside talent that you can really use, and that the people who are working for you or working in this job or in part of a company are the ones that know themselves best, and so can they offer or select specific requirements or ideas that might help them the most. And the more that we can be flexible, but still have the same end point, like we want to figure out what the goal is, but there can be multiple ways to get to that goal. Does that mean if you’ve got a project, and somebody can do that project, whether they’re in the office or not, and do a really great job, does it matter that they have to be in the office for 40 hours at the time that you’re there, just those sort of lessons we learned during the pandemic. Let’s not forget them, but that it can be something that’s really useful in the world. And think about how we teach our students. So if we’re going to teach a student to create a document, why not tell them also about how to be accessible, so that when they’re in the working world, they know to put alt text in a picture because they’re going to put it on the web, they know how to do headings in a document, so that somebody who needs a screen reader or is blind can figure out what’s going on, because your company wants to reach all those people. So just thinking about the ways that, if we’re learning it in higher ed, then that is actually going to be a really marketable skill to take on into the workforce. So it doesn’t start or stop in the education world, it just is for all humans, because we all learn and we’re all different.

John: It’s a good note to close on, and we always end with the question: “What’s next?”

Tom: Well, my what’s next Universal Design for Learning is now moving from individual micro level applications to at-scale barrier-lowering across entire institutions.

Jennifer: Well, I am in the process of shopping a book around, and my book is going to be focused on neurodivergent learners in higher ed and what we can do to support them. And if we’re going to be honest, it’s mostly about UDL, but I’m also talking about other things as well, but it’s a book that needs to be written, and because it’s a book I wish my professors had when I was an undergrad with undiagnosed ADHD. So this is something that I’m hopefully going to be moving forward on very soon.

Lillian: And what’s next for me? I’m not writing a book, but I’m still podcasting, so I like to listen to everyone’s stories and try to get those out about how UDL really is changing the world for the better, and the more I can hear from folks who are implementing UDL, the better off I think we all will be. So that’s always it seems to be, what my next thing is, is the next podcast and the next guest. And other than that, you know, world domination. No, no, not that. Just UDL.

John: And I haven’t missed any of your podcasts since you started. In fact, I heard about the podcast at the OLC conference a number of years before, when someone was talking about planning this podcast.

Lillian: Excellent.

John: I very much enjoy your podcast.

Lillian: Thank you.

Rebecca: Well,thank you so much for joining us. It’s always a delight to talk to all of you and to always talk about one of my favorite topics.

Tom: Thanks for having us and listeners, we’d like to hear your stories too, so comment and you can find us on the socials, and we would love to hear from you as well.

Jennifer: Yeah, thanks for having us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

390. Critical Teaching Behaviors

Posted in active learning, evidence-based practices, feedback, higher education, learning, pedagogy, professional development, reflective practice, teaching
April 23, 2025

Student feedback is important to improving teaching, but may not be aligned with evidence-based teaching practices. In this episode, Lauren Barbeau and Claudia Cornejo Happel join us to discuss a midterm student feedback instrument focused on critical teaching behaviors, an AI-assisted tool for analysing the feedback, and strategies for debriefing with students.

Lauren is the Assistant Director for Learning and Technology Initiatives at the Georgia Institute of Technology. Claudia is the Director of the Center for Teaching and Learning Excellence at Embry-Riddle Aeronautical University. Lauren and Claudia are the authors of Critical Teaching Behaviors: Defining, Documenting, and Discussing Good Teaching as well as a whole series of other resources related to this book.

Show Notes

Transcript

John: Student feedback is important to improving teaching, but may not be aligned with evidence-based teaching practices. In this episode, we discuss a midterm student feedback instrument focused on critical teaching behaviors, an AI-assisted tool for analysing the feedback, and strategies for debriefing with students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Lauren Barbeau and Claudia Cornejo Happel. Lauren is the Assistant Director for Learning and Technology Initiatives at the Georgia Institute of Technology. Claudia is the Director of the Center for Teaching and Learning Excellence at Embry-Riddle Aeronautical University. Lauren and Claudia are the authors of Critical Teaching Behaviors: Defining, Documenting, and Discussing Good Teaching as well as a whole series of other resources related to this book. Welcome, Lauren and Claudia.

Lauren: Thanks, John.

Claudia: Thank you.

Rebecca: Today’s teas are:… Lauren, are you drinking any tea today?

Lauren: I sure am. I’m always drinking tea. So I am drinking a passion fruit peach tea, courtesy of my friend and colleague Karen, who went to the SOTL conference in Savannah and brought this back for me from the Savannah Bee Company. So if you’re listening, Karen, thank you.

Rebecca: That’s a nice little story. How about you, Claudia?

Claudia: I am also drinking tea that comes to me by way of a conference, mine was in Albuquerque, New Mexico, and the tea that I’m having is a Rajasthani chai tea.

Rebecca: Oh, yum. How about you, John?

John: I am drinking tea that came from a Wegmans, [LAUGHTER] which is a black raspberry green tea from the Republic of Tea.

Rebecca: And I have a Blue Lady tea from Scotland.

John: Very nice. We’ve invited you here to discuss the critical teaching behaviors framework that you have jointly developed and some recent extensions of that framework. Could you tell us a little bit about how this project got started?

Lauren: Sure. So this project really started when we were working together at Georgia Southern University, and at the time, we were very much in need of a campus language in which to discuss what good teaching is. What happened after that is we found out lots of other people were in the same place. They also needed some sort of a common language in which to ground their conversations about teaching on campus. And so what started off as meeting a campus need ended up expanding to meet a broader need in higher ed.

Rebecca: Can you provide a little bit of an overview of the framework?

Claudia: Sure. So the critical teaching behaviors framework is based on sn extensive literature review that Lauren and I did, starting with the Scholarship of Teaching and Learning educational research, primarily in the context of higher education, but also some K through 12 insights found their way into it as well, and based on that literature review, we narrowed it down to six important behavioral categories that all of us as we teach, should ideally engage in in some way, shape, or form to support student success, and those six behavioral categories are align, making sure that our course materials activities align with each other; include, making sure we’re creating an environment and opportunities for all students to be successful; engage, making sure that students are taking an active role in their learning journey; assess, making sure they have a chance to self-assess, to get feedback on their learning; integrating technology, we cannot really think about teaching and learning without technology anymore, and so because it’s such a big part of teaching and learning, we need to think about it intentionally; and then finally, reflect, which is what comes back to us and the importance that we play a role also in continuously growing and improving our teaching.

John: You’ve also developed a number of instruments for faculty use in alignment with this framework. Could you tell us a little bit about these tools and why faculty might be interested in using them?

Lauren: So, this book was designed to be a one-stop shop for faculty. The first half of the book is an overview of the research we did on those six categories of behaviors, as well as reflection questions to help faculty readers think about how they can implement those behaviors. The second half of the book is really all about helping faculty document the fact that they’re doing those things. So we start the second half with some narrative starter templates and some deep reflection to identify your core teaching values, because our values should guide the principles that we use when selecting strategies to implement in our classrooms. That should frame everything that we’re doing. After that chapter, we do a deep dive into peer observation of teaching, which, coincidentally, is our next project. I know I’m jumping ahead to the end where we talk about what’s next, but we do have a book under contract with Routledge to explore peer observation of teaching further. But in this chapter, you get just the basics of what you need to go out and do this right now. So that includes email templates, logistical information collecting templates, some reflection starters for the faculty being observed. It also includes a note-taking instrument for faculty who need help figuring out what I should take notes on when I go into the classroom. And finally, it includes a report form for faculty to collect what they saw in the classroom, with space for the faculty being observed to write down their own reflections. That way, we’re giving faculty agency and voice in the process of creating this artifact. Chapter after that moves into thinking about student feedback, and how we collect student feedback from our students, and how we use that to make decisions about our teaching as well. That chapter really focuses on student feedback, in particular at the midterm, not necessarily at the end of course, which may be what most of our listeners think of when they think of student feedback, but a lot of the work that we’ve done more recently has been about collecting midterm student feedback. As we know from research, this can help boost your end of course evaluations when you debrief it correctly. I always want to add that caveat when we say this, it isn’t a magic wand that we can wave and ensure that we get better evals at the end of the semester, but when we debrief it with our students correctly and meaningfully, one of the things we found when we looked at the research on it is that, yes, it can boost your end-of-term evaluations.

Rebecca: So we’re hoping we can dig into that midterm feedback aspect a bit more. In the Fall 24 issue of To Improve the Academy, you published a formative midterm feedback instrument that you developed and validated. Can you talk a little bit about this instrument and why faculty might want to use this instrument and midterm feedback just generally?

Claudia: Yeah, happily. The critical teaching behavior, midterm feedback instrument, we started implementing at Embry-Riddle in 2021 and we have since collected data on its use across the different departments and across different instructors. And one thing that I’m super excited about is that from the first year that we implemented it, we had about 20 or so faculty using the instrument per year, and now we are reaching about 20% of the Embry-Riddle faculty with that tool, which is pretty amazing. It has been very well received. It combines differently than some other midterm feedback processes. It combines multiple-choice rating questions with open-ended feedback. Generally, midterm feedback often focuses on only the qualitative responses, so open response feedback from students, and that has really resonated with the STEM faculty, specifically at our institution, because it gives them a place to look at, okay, here’s where we started, here is where we are next semester. This is what we had at midterm. We made changes, we talked to students. Now, looking at end-of-course feedback, we can see that there has been growth or not. So that’s been really helpful for a lot of faculty here, because we’ve had such a lot of participation with it, it also gave us really good data to look at: “Okay, does the tool that we’re using, is it actually statistically reliable and valid?” And so that is the study that we published in To Improve the Academy, we validated the tool and did a confirmatory factor analysis, and it was valid and it was reliable, but importantly also, as part of this study, we asked faculty on their perception of the tool, and we asked students to also tell us, “Okay, these are the questions that we’re asking in the survey, how do you interpret it? What does that mean to you when we’re asking this question? What would you look at in a course to give you information on how to answer the question?” And so triangulating all of that different information and data, we found that the instrument is working really well for students, for faculty, and also the data that we have shows that it is reliable and valid. So that was exciting to see.

Rebecca: It’s always nice when we take on a project like that, and it works out, right? [LAUGHTER]

Claudia: Absolutely

John: A lot of the teaching evaluations used in many departments often are not very well designed, and they often include many categories that students may not be the best source of judgment. One of the nice things about your evaluations are that they focus on observable behaviors that are evidence based and they are things that should be observable to students, and if they’re not observable to students, the instruments would reveal that in ways that suggest maybe the faculty needs to address those. So one of the things you suggest is that this instrument might also have some impact on the bias, which has been demonstrated to show up in student evaluations of teaching. Could you talk a little bit about that?

Claudia: Yeah, and you see me nodding along and getting really excited, because that is another thing that, when we first published the book and the article, we had some preliminary data that suggested, yes, it reduces bias broadly, but we actually had an opportunity to gather additional data on that and to look at that, and we are still in the final stages of preparing the data and analyzing it, but we have done some analysis on this now that shows that if we’re looking at multiple factors that all impact teaching and learning and student feedback in some way, the questions that we are asking with a critical teaching behaviors midterm feedback instrument actually show now that statistically, gender in particular, is no longer a significant factor. So that means that, basically, gender is not one of the demographic categories that seems to be influencing the student responses to the questions that we’re asking. We do see, in the data analysis that we have now, that teaching experience, so years of teaching, as well as age, are still statistically significant, but gender is one of the categories that is no longer statistically significant. So that was exciting to see.

John: You mentioned that age and years of teaching experience were still significant. What was the nature of the impact of teaching experience and age on the evaluations?

Claudia: So if we’re looking at the data that we have, it shows that teaching experience and age, so they go parallel, the longer teaching experience that you have, as well as the older you get, it generally signifies an increase in the rating that students give you on how frequently you are engaging them, on how frequently you are creating an inclusive teaching behavior in the classroom, on how frequently you are providing feedback that is meaningful to them. So overall it increases until around the plus 55 where it falls off. But that is also probably a limitation, because we had very few respondents in that age group. So, that’s what we found.

Rebecca: I can imagine at this point in the conversation, audience members might be like, yes, this sounds good, but what does it look like? Can you talk a little bit about what some of the questions are? You mentioned that there’s some multiple-choice questions, some open-ended questions. Obviously midterm implicates when in the semester you might implement this tool. Can you talk a little bit about what those questions actually say?

Lauren: So let me talk first about what midterm student feedback is. Obviously this is something we collect at midterm, specifically between weeks five and 10 is what research recommends. There are a lot of tools out there [LAUGHTER] to collect midterm feedback. The standard forms are all open-ended questions. Typically they’re two to three questions long, maybe a little bit longer if you as an instructor choose to add an open-ended question or two. The most common questions that are asked are: “What’s working well in this class? What’s not working? What additional feedback do you want to have or want to share?” There are also versions of this that look like, “What should I start doing? What should I stop doing? and What should I keep doing?” So I just want to put that out there that we are not the originators of midterm feedback. This has been around for a while, and there are lots of different ways of doing it, although they’re fairly similar and standard in the fact that they are qualitative, open ended. What makes ours different is that we did choose to depart from the purely qualitative aspect of this by introducing some quantitative questions. So with our feedback tool, there are still the standard questions: “What’s working well, what’s not working well?” From a data analysis perspective, I think it’s interesting to note that we chose to put the quantitative questions first on our survey for students. So we have on the validated tool 15 quantitative questions that students answer on a never to very frequently scale, and those questions are broken down by each of the CTB categories, except for reflect, because students really can’t see us reflecting on our teaching, so that’s not something we ask them to give feedback on. But an example question would be… so from the aligned category, “The instructor states the learning outcomes, development of specific skills and knowledge to be accomplished in the course assignments and activities.” And then students would answer never or very frequently. So each of these questions is designed to capture how often an instructor engages in particular behaviors that we know from the framework research are proven to promote student success. When the instructor gets this report, they see a breakdown of their behaviors by category, so that they’re able to look at the quantitative data and see overall strength by category. So for instance, you might see that you’re doing really well in “engage,” but students said that you only rarely or sometimes gave feedback that was helpful for them. So maybe the “assess” category is an area for development. So what’s nice about these quantitative questions is that it’s allowing you to get the nuance of individual behaviors while also capturing overall pictures of what’s going on in these categories from the student perspective. Now, I mentioned that we put the quantitative questions first on the survey. There’s some interesting stuff going on there in terms of what students end up putting down in their qualitative feedback, and this is something we would like to study in more depth, is how much does the language of the quantitative questions prime the students in terms of what they give feedback on and the language that they use. This is not something we’ve run a study on yet, because we’re running all sorts of other studies, but it is a study that we do plan to run on the data we’ve collected through midterm feedback, because if this bias study has shown us anything, the tool does help mitigate bias to some extent, if the tool also does some priming of students to help direct their feedback to particular areas of teaching and take them away from comments about “I love her shoes” or “I hate his shirts,” then it has an additional use factor, which is that the quantity of questions are doing some work to teach students how to give good feedback. That’s our hope, anyway.

Rebecca: So let’s imagine that I’m an instructor, and I use the midterm feedback form and I get some feedback about areas that maybe I’m doing well in, and areas that I might need to improve upon. What do you suggest are the instructor’s next steps?

Lauren: Well, I would suggest that you consult with your Center for Teaching and Learning. Oftentimes, this is a service that Centers for Teaching and Learning offer, and there are trained consultants like me and Claudia who can help you make sense of what you get out of that experience. In fact, we can help facilitate that experience as a neutral third party, which can help students feel like they’re not necessarily going to be judged on their responses. Even if you make it completely anonymous, there are always those students who say, “Well, but if you really want to, you can figure out who said what.” But if you have a neutral third party who is facilitating this experience, it can make it a better experience for students overall. Not everybody has the luxury of access to a Center for Teaching and Learning. And when we were designing our tools, we were thinking about that. So the next thing I would say is, if you don’t have a Center for Teaching and Learning and trained consultants, you do have colleagues. One of the things that is very important to us in this project is that we’re creating dialog around teaching and learning. So if you have a friend or a colleague who would also like midterm feedback, maybe you can conduct this for each other and have conversations with each other about this instrument. Now, of course, you can always just administer the tool yourself, collect the data and sit down to analyze it. If you’re going to sit down to analyze your own data, here’s what we have for you. If you request access to our folder of materials, you’ll see in there an Excel spreadsheet that Claudia, who is an Excel guru compared to me, has created that will help you decipher the quantitative data. It’ll spit out some really nice graphics for you to be able to analyze, but I’m super excited to be able to share that we’ve also developed a way to help you make sense of the qualitative feedback on teaching. This is where we are debuting the CTB FIT, critical teaching behaviors feedback insight tool that is an AI-powered platform that we developed in conjunction with a student at Georgia Tech, Elliot Roe, and his partner in crime, Duncan Johnson, who’s at Tufts. And these students helped us train the AI platform on the critical teaching behaviors categories, again, not including “reflect,” because it’s not something that students can give feedback on. And what this platform will do is you feed it the student feedback, and it will output an Excel spreadsheet for you where it tags the qualitative data with the five CTB categories. It’ll pull out snippets and code them into each of the CTB categories. Then it will do a count of the number of times it tags the category and it will produce a nice bar graph, so you get a visual of how many times something shows up, for instance. This has been really useful to the faculty who are conducting midterm feedback. We’ve gotten some great preliminary insights from faculty saying that this helps them make sense faster of the data that they have collected from students, and it helps them focus less on that one-off comment, you know, the one, the one that gets under your skin, and you have all these other wonderful comments praising your teaching, but there’s that one that sticks with you, because we just can’t shake the one negative comment. This is why it’s so helpful to have a consultant do this, or to have a peer do this, because they can move you past the one negative comment. But if you’re doing it for yourself, it’s so easy to get hung up and miss the bigger picture. What FIT does is it organizes the data, provides you AI summaries in each of the CTB categories that it tags so that you’re not focused in on that one negative piece of feedback, you can now focus on the big picture and take away insights that allow you to report back to your students and make informed decisions about how you’re going to teach your class. Claudia, what would you add?

Claudia: Just reiterating, I think the importance of using this also as a tool for conversation. Lauren already mentioned that colleagues can play a really important role in helping us make sense of this. We have the tool that’s really available for anybody to use who’s interested. But I think the other important thing to mention also is when we’re looking at the feedback that we are receiving, one important conversation partner is also our students, and so making sure that whatever we’re learning from the feedback, whether that is positive or room for improvement, we go back to the students and say, “Hey, I learned this from you. I appreciate you for providing the feedback. And here’s something that I can change, maybe in response, here’s something that I will not change because I have a good reason. Let me explain that.” But yeah, so I think just reiterating the importance that students have in this process, not just in providing the feedback, but also being a conversation partner with us.

Lauren: And I would close that thought by adding to do this recent study that we’re working on, we ran a couple of focus groups with faculty. One of the really interesting insights that came out of that focus group for me was that faculty who do midterm feedback and debrief it with their students, go back and they talk to them about what’s not working in the class, what they can do to change it, and what they don’t have the power to change. But there was this big revelatory moment in the conversation where a faculty member said, “I have never talked to my students about what they say is working well. I’ve never gone back and said, ‘Thank you for letting me know.’ I have never debriefed that with them.” So if there’s anyone else out there who is realizing that they also only focus on what’s not working and what they can and can’t change, this was a really great moment where faculty came together and said, “Oh, we also need to talk to our students about the positive feedback that they’re giving on our teaching and what is working. So that’s part of the loop that we need to close as well.”

John: Now, you mentioned that this is an AI tool. Was it created as a GPT in ChatGPT, or did you use some other system for creating it?

Lauren: This is a question primarily for Elliot Roe, who’s not here today, but Elliot created a pipeline that is trained on the critical teaching behaviors, and it goes to ChatGPT. So what we were careful to do thinking about: this is student data, this is faculty data, we want to protect privacy. We don’t store your data. First of all, that costs a lot of money, and second of all, that gets into all sorts of privacy concerns. So if you run any sort of data through the platform, you need to download that report immediately, or the next time you go to run a report, it will be gone. So it’s actually stored in your cache, it is not stored on our site, so that data belongs to you. As we’ve set it up, it’s not saving the data in any way, shape, or form, and it’s not being used to train ChatGPT on anything, as far as we can tell from talking with Elliot, who is our technological guru here.

John: We have encouraged faculty to consider using ChatGPT on anonymous student surveys on our campus, because it has all those nice advantages that you’ve talked about, that it can summarize things very nicely, and it can focus on the major themes that occur, and it can also offer some suggestions on ways in which faculty might even address that if that’s programmed into the chatbot. So it sounds like a really great tool. Are you sharing this more widely beyond your campus?

Lauren: Yes, that is the goal. It’s free to use where we want to keep it free to use, because our goal is to improve teaching in higher education, and we believe that this is a tool that can do that. To be clear, you’re not going to sign into ChatGPT when you do this, CTB FIT is its own platform that you sign into and put the data into and it will output the Excel spreadsheet for you through that platform. We do ask you to sign up. One of the ways that we’re trying to protect privacy and security is to have sign ups, but that’s not because there’s a pay wall, it’s just because that’s an additional security measure to have a sign up on it. So we’re putting it out there. We hope that faculty use it. Our idea behind this is that, yes, you can run your data through any AI platform you want, and it will give you some insights, and it will probably do some analysis for you. The problem I’ve seen when I run data through it is that it’s not trained on any sort of teaching database in particular, so depending on its mood that day, [LAUGHTER] you will get all sorts of different answers out of it, and some of them might be good, but as we know, some of them might not be. We think that the value of FIT is that it’s grounded in that common language. So if you get your report back and you see that students are seeing a dearth of behaviors, for instance, in the “Engage” category, and you want to know what behaviors you could implement in the “Engage” category, well, we’ve got a whole book on that for you. You can go to the chapter, you can reference the research that we’ve done. You can dig deeper into that category. There is a real knowledge base behind this platform that allows faculty to dig deeper into their own teaching. You can dig as deep as you want. You can go reference the chapter in the book. You can start looking at the references and go look at the research for yourself, and then the book, of course, is structured to produce reflection, so that the reflection that faculty engage in produces growth. So if you then want to set a goal for growth, we’ve also got tools in the book to help you do that. So really, the idea of FIT is that it can be as surface-level analysis of data as you want it to be, but because there’s this knowledge base behind it, you can dig deeper and deeper and deeper as much as you want.

Rebecca: One of the things that you’ve talked about is that debrief, but I imagine the setup of the midterm feedback is equally as important. So can you talk a little bit about what the setup is as you present the opportunity for feedback to students, as well as what that debrief ideally looks like? You’ve hinted at some of those things, but I know people are always looking for very concrete, tangible ways of doing things.

Claudia: Yeah. And so when we are facilitating midterm feedback as a third-party facilitator from the Center for Teaching and Learning, we usually go into the classes, if it is a face-to-face class, and give that introduction to students, telling them that this is something that’s voluntary for the faculty. They are not required to do that. They are engaging in midterm feedback because they really want to know how things are going for the students and whether the strategies and materials that they have chosen are meeting the students’ needs, and so emphasizing that it’s voluntary, but then also telling students that not all feedback is equal. And so if I’m just saying, “Oh, this class is awesome, that is great affirmation, but it does not help me really get better at my teaching.” And so also telling the students how important it is to think about providing feedback that is specific and actionable, pointing out, if they are saying it’s amazing presentations, giving some more feedback on well the PowerPoints are great, or it’s a great interactive lecture that helps me think along. So really providing some of that detail. If we are not in a face-to-face classroom, we do have that information also available as a script that faculty can use and then share with their students through the learning management system, as an announcement, again, mentioning it’s voluntary, it’s really helping me, as the instructor in your class, grow, it’s helping you as a student provide me feedback now, rather than waiting until the end of the term, please provide feedback that is as specific as possible. And so that is usually what the setup looks like. For the debrief for our campus, we are meeting with the faculty member one one-on-one to facilitate the debrief, and it is really about the conversation where we are looking at the data primarily and then make sense of it together. So for us, it is really important to not be prescriptive, even if we’re saying, “Oh yes, maybe engagement is a category where there’s room to grow,” it is not that we’re saying, “Oh okay, engagement is not good right now, let’s do think-pair-share every single day of the week.” So really thinking through also there are many ways in which we can engage our students. And what are some options that really work for your course, for your context, if you are online, face to face, what is your personality like? What is your teaching persona? And really finding strategies that are helpful for the individual, rather than a silver bullet that works across the board, which doesn’t exist.

John: One of the things I really liked about your book is part two. Part one, I really liked it was very comprehensive, and it provided some really good strategies that are evidence based. But the second part of it, and the appendices as well, are very helpful, because a lot of faculty, especially when they start their teaching career, but often continuing for quite a while, suddenly get hit with this need to write up a summary of their teaching activities and a reflection on their teaching that’s used for promotion and tenure. And there’s very few resources out there that address that, and your book does a really nice job of giving faculty information on what they may want to include in these reports and what they may want to submit as part of the reflection. This seems like a gap in the professional development literature that your book fills very nicely.

Lauren: First of all, thank you, [LAUGHTER] that’s a lovely compliment. When we were developing this framework, the need to have the conversations around teaching grounded in the same language was a paramount issue on the campus we were working at, but we were very attuned to the needs to document teaching, because this is part of the conversation. So we can have conversations with peers, we can have conversations with administrators, we can have conversations with students. Those are sort of the three different types of conversations we have, and they all need to be grounded in the literature of good teaching. But it’s really hard to have that conversation in any sort of personalized way if you have none of your own data to back it up. This is a very timely conversation, because just yesterday, I ran a two-hour session on reflecting on teaching for faculty at Georgia Tech, and part of what we did is we started with, “What do you think are your teaching strengths and what are your areas for growth?” We just put that out there into the ether and let them put something down. But after we had talked about the CTB framework, we went through a more structured teaching practices inventory and had faculty sit down and go through the framework and say, “What are you actually doing in each of these categories? Now look at it across categories. What is your teaching strength and what is your area for growth? How does that compare with what you set when we first started this session and you had no common language, you had no scaffolding to help you make that decision?” It was a really interesting conversation, because someone said “it was hard,” like, “well, it was hard in the sense that it was work. It was a lot of work to sit down and really look this closely at my teaching.” So I think part of the reason this doesn’t happen more often is because faculty are busy. We’re busy people already, and making time to formally reflect on teaching is difficult because it’s just another thing that we don’t have time for, and it seems like there’s always another fire that we need to put out. So it doesn’t become a fire to put out until you need to create your materials for promotion, and then it’s an issue because you haven’t created any of this documentation along the way. The goal of our book is to help faculty reflect as they go to build in just a little bit of time to do that structured reflection so that they can identify areas of strength and set intentional areas for growth. Why? Because a narrative of growth over time is persuasive, and when you go up for promotion, being able to show intentional goals, data you’ve collected in support of those goals and data doesn’t have to be a full-blown research study. It can be student feedback over time that shows you’re getting stronger in these categories. Having all of these aligned CTB tools gives you a common language to collect data from students, peers, and yourself, so that when you go to create that portfolio to showcase what you’ve done, you no longer have to reconcile “My peers are talking about my teaching this way, but students talk about teaching this way, and admin understands it this way. How do I put all these different perspectives together in a way that’s going to make any sense when they seem to all be speaking different languages?” Well, the tools we’ve developed are all aligned to the same common language, and when you go to put together your portfolio, it’s that much more powerful and persuasive to have the common language, to have collected the data over time, as opposed to be sitting down at the last minute saying, “I’ve got to put this together, and now I’ve got to figure out what I did,” as opposed to 10 minutes a week could allow you to create a much stronger teaching portfolio.

John: And you provide a number of resources in various appendices at the end of the book that faculty can use for some of these purposes. C ould you just briefly talk a little bit about what’s included in these appendices?

Claudia: Yeah, so in the appendices we have the framework, of course, but also we have the peer observation report form, as well as associated materials that help peers set up the observation as for information before we even go into the classroom, or before we go into the online class. It has a prior version of the midterm feedback, so there is an updated version that’s available on our online share documents that anybody can ask for, but it gives a nice preview. But it also has some reflection questions, as Lauren said, that help prompt some of the reflection for faculty to start putting together the material into a coherent format for a portfolio. Also you mentioned it is Creative Commons licensed, and that is true, that was also an intentional choice, because we recognize that while the critical teaching behaviors framework, we hope, provides a great starting point for everybody, there are institutional preferences. Sometimes, when it comes to language, there are institutional focus areas that may not necessarily be captured with what we are providing. And so it is intentionally Creative Commons licensed. So it is a starting point that people can build on, based on departmental needs, based on their individual needs based on institutional needs.

Lauren: It was important to us that these materials be Creative Commons licensed for meeting contextual needs. I think as much as we recognized that a common language was necessary in higher ed, we also recognized that institutional context matters, individual context matters, and that there can’t be one definitive way of doing things. The Creative Commons license doesn’t just allow you to adapt to your context. It allows you to make these multifunctional. So let’s say that you really want to do a dive into your assessed behaviors, and you want to have a peer observation report that’s just focusing on your assessed behaviors. You can do that if you have an institutional initiative. Down here in the south we have quality enhancement plans for our accreditation. Let’s say you wanted to focus on just one aspect of the critical teaching behaviors to align with institutional objectives. You can do that. The only thing that’s not Creative Commons licensed is the framework itself. So that still stands, but we’ve had a number of institutions take that as a place to develop their own institutional framework. So Harvey Mudd, in particular, has done a great job with this. Most recently, they even added a whole new category on mentorship, because that meets their campus need. But we’ve worked with a number of institutions now that have really done a good job of taking this not as: “we told you, this is the language,” but saying “somebody went out and did the research for us so we can develop the language that we use here at our institution.” We see more and more of this happening, but when it’s most effective is when it’s grounded in evidence-based practice. And not everybody has the time to go out and do that research, but apparently we did, [LAUGHTER]so we went out and provided that for people to go and build their own frameworks.

Rebecca: It’s always nice when there’s a starting point and you don’t have to [LAUGHTER] start from scratch, and that there’s templates and things that you can iterate on so that you can move forward faster. So we always wrap up by asking, what’s next? You’ve given a little preview, but…

Claudia: We just look forward to talking more about critical teaching behaviors no matter what, because we believe it can make a huge difference in fostering conversations across campuses. But then also, as Lauren said, we are really looking into peer observation, more specifically, as a next focus to really dig deeper, as a way to continue those conversations, as a way to apply critical teaching behaviors, but also as a way, again, to have a starting place to then customize something that meets somebody else’s context.

Lauren: So we always have a little bit too much in the way of what’s next. [LAUGHTER] In the way of what’s next. I think we would like to continue the research projects on midterm feedback in particular. So midterm feedback, we would like to do that study on whether the quantitative questions have any bearing on the qualitative responses we’re getting from students. That is an area we’d like to follow up on, but we do have that book contract, and we’re supposed to have that manuscript done by June of 2026, so that’s gonna have to take precedence over any other projects that we might want to delve into in the next year.

Rebecca: Well, thanks so much for joining us today and sharing some of your work and the framework.

Lauren: Thank you for having us.

Claudia: Thank you for having us. It’s been great.

John: Thank you for joining us. And again, this book is an excellent resource for faculty at any stage of their teaching.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

368. Pedagogy for Large Classes

Posted in classroom community, engagement, evidence-based practices, evidence-based teaching, feedback, higher education, large classes, large-class instruction, learning, pedagogy, teacher training, teaching
November 20, 2024

Building a sense of community is an important component of inclusive teaching. In this episode, Anna Logan and Ann Marie Farrell join us to discuss strategies that can be used to cultivate belonging and increase student engagement in large classes. Anna is an Associate Professor in the School of Inclusive and Special Education and the former Dean for Teaching and Learning at the Institute of Education, Dublin City University. Ann Marie is an Assistant Professor, also in the School of Inclusive and Special Education at Dublin City University. Anna and Ann Marie are both recipients of multiple teaching awards. Since 2018, they have been running an annual Pedagogy for Higher Education Large-classes Symposium.

Show Notes

Transcript

John: Building a sense of community is an important component of inclusive teaching. In this episode, we discuss strategies that can be used to cultivate belonging and increase student engagement in large classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Anna Logan and Ann Marie Farrell. Anna is an Associate Professor in the School of Inclusive and Special Education and the former Dean for Teaching and Learning at the Institute of Education, Dublin City University. Ann Marie is an Assistant Professor, also in the School of Inclusive and Special Education at Dublin City University. Anna and Ann Marie are both recipients of multiple teaching awards. Since 2018, they have been running an annual Pedagogy for Higher Education Large-Classes Symposium. Welcome Anna and Anne Marie.

Anna: Thank you.

Ann Marie: Thank you.

John: Today’s teas are: … Anna, are you drinking any tea?

Anna: I am. Generally I drink Irish Breakfast tea. But on Earth, this occasion, I got something special. I’ve never had it before. It’s called Emperors Clouds and Mist, so it’s a green tea, I don’t know. Hopefully it’ll work out alright. And I’m having it in a keep cup, a glass keep cup.

Ann Marie: Well, I did not go down the fancy route, so I am drinking Barry’s tea with, I don’t know what type of actual tea it is, except that it’s Barry’s tea, and it is a staple of pretty much every cupboard in every kitchen in Ireland. So it’s standard ordinary tea.

John: And we have some of that in our office, actually. We have a nice variety there.

Rebecca: On this occasion, I got some Irish Breakfast tea. [LAUGHTER]

John: And I should have, [LAUGHTER] but I am drinking a ginger peach green tea.

Rebecca: A good favorite of John’s, of course. So we invited you here today to discuss your work related to large-class instruction. Could you each tell us a little bit about your background and interest in large-class instruction?

Ann Marie: Well, I suppose we’re both teacher educators. All of our students are either student teachers or are qualified teachers who come back to do some of our postgraduate courses. But particularly in relation to student teachers, some of our classes are quite large, and we were anxious, I suppose, to model good teaching practices in that context, especially as our students will themselves be teachers. So a number of years ago, we began to look at our work in the large-class context, particularly on a four-year Bachelor of Education primary teaching program, which, in each year group, has over 400 students, and we were teaching all 400 together in a large tiered lecture theater. So that was, I suppose, my interest in that area.

Anna: And I suppose, just to add to that, for many years I’ve taught a small number of classes that are in a large cohort, that kind of 400-plus size, and always enjoyed it, always got a buzz from the experience. And then, I think a couple of years ago, I was one day teaching that large cohort at nine o’clock in the morning in that tiered lecture theater, a quick 50-minute lecture. And later that day, I was teaching a group of six doctorate in education students, and I remember being really struck by the kind of very differential model of resourcing there: these six, eight doctoral students, really experienced practitioners, getting a lot of support and so on, versus perhaps that introductory class. But then a number of years ago, Ann Marie, who we go back a long, long way, invited me also to co-teach some of her sessions with the large cohort. And I really enjoyed that work so much, so I’d already enjoyed the dynamic of working with the large group. I had questions about it, about the equity issues associated with it. We might come back to those a little bit later. And also, I just really enjoyed working with Anne Marie and the dynamic of working hard with that large group.

Ann Marie: The pupil-teacher ratio was still one to 225.

Anna: We go back a long, long way.

Ann Marie: We were in college together with our primary degree, and then 10 years later, just totally by chance, we ended up back in college together on a postgraduate diploma course. So we go back to the 1980s as students, and then later as colleagues. [LAUGHTER]

Anna: And we’re still talking to one another, which is good. Always helps. [LAUGHTER]

John: That’s wonderful. Can you tell us about your Pedagogy for Higher Education Large Classes project?

Ann Marie: Yeah, I suppose I can tell you a little bit about that. The Pedagogy for Higher Education Large-Class symposium (or PHELC) symposium, began really as a one-off event. We were invited to run a pre-conference workshop at a wonderful conference held annually in Valencia every year called the Higher Education Advances Conference, or the HEAD Conference. So in 2019 we ran that pre-conference workshop face to face, and again, a pre-conference workshop for the same conference in 2020 online when the pandemic hit. And in the course of that, we set up a website. I suppose, we set up a model for the symposium that then became an event in its own right. So we’ve run it now for six years, and we’ve attracted people both online… the last five have been entirely online… and we’ve attracted people from all over the world who are engaging with very large-class numbers and large-class cohorts, albeit in different contexts. But many of the challenges, and more importantly, many of the possibilities that have been experienced by those higher education teachers, is our focus at that symposium. So we look for papers based on research, but also based on reflective practice, with the aim of reducing the fear and angst around large-class teaching and giving people who are doing some really good work in that area, a platform to share that work with others. So we also set up a website, and most of the presentations and all of the papers, we asked presenters to write an accompanying paper along with their presentation. They are all available on phelc.ie, PHELC.ie, for anyone to look at. The Symposium is free, we usually get funding here in Ireland, and I suppose our main aim is just to enable access to anyone who wishes to participate, either to listen or to present, that there’s no barriers. So hence the online, and hence no conference fee for it.

Rebecca: But Anne Marie, you said it started as a one off, I think it’s still going.

Ann Marie: Oh, it’s still going. We’ve done six [LAUGHTER] and our initial intention was just to do the pre-conference workshop in Valencia in 2019 but it became evident that there was an interest in that area. The Conference Committee invited us back again the following year to do a second workshop on this topic, but we just got a lot of very good feedback from participants. We also had put in a huge amount of work into developing the website, developing the workshop or the symposium, and we love large-class teaching. We had done a lot of work in that area ourselves, and it just allowed for our pedagogical practice and that to be mapped onto the scholarship of teaching and learning, and to give people an opportunity to engage in scholarly discussions, be it at the symposium or in terms of the papers that they submit, that they could align their teaching practice along with their academic writing, researcher-type practice. So we’re still going. Now we are at a crossroads, Anna and I were just wondering if we will continue with our annual symposium, or maybe we might move to a symposium, perhaps that maybe every second year, and broaden out maybe to do some more focused publication and research, perhaps every other year, we’re not sure. So we’re due to carve out a day or two to work out what we’re going to do going forward, but we will be doing something going forward.

Anna: And we were always committed, John, to open education resources, so all the publications are available as OERs and so on. So again, I think reflecting Anne Marie’s observation, we tried face to face, we tried HyFlex, we tried online, and what we’re really interested in kind of equity and making sure there’s no barriers. So we have had people from all across the world, and we’re committed to the resources we produce being available to everybody.

John: Excellent, and we’ll include a link to the website in the show notes, so if people would like to check out those papers they can do so. One of the things we’ve noticed, in the U.S., at least, is that as we brought in more first-gen students and more low-income students, we see increased DFW rates and one concern that a lot of people have with large classes is that it might be easier for students to get lost. What’s your perspective on addressing that issue of students feeling disconnected from their class and their instructor, when they’re in such a large-class environment, and I teach a very large-class as well.

Anna: Yeah, that’s a really interesting question, John. And just thinking a little bit about it, and it’s interesting, it was actually some colleagues that I met on my sabbatical where I first heard that term “weed out,” because it’s not a term that would be much used, in the Irish context anyway. On the one hand, you know, looking at the data, I suspect that that notion of attrition or non-progression or drop out is potentially maybe slightly lower in the Irish context. The latest figures coming from our higher education authority are putting us at about 85% progression from first year, which is, I think, pretty good. That said, it’s interesting that the rates of progression are considerably lower among male students, but also among students who come from socio-economically disadvantaged groups. And coming back to your question and the question of relationships. I suspect that that failing to build relationships, or build connections with classmates or with lecturers or to feel that they belong in the community, whether it’s large or small, is very likely to be a significant factor in that dropout. Again, I think interestingly, it reminds us when we look at that, and it certainly was something I know we discussed at our most recent PHELC Symposium the importance of the relationships when teaching at scale, and that not making any assumptions. Sometimes, maybe as lecturers, we assume that students are well connected, that they have a good network. That’s very often not the case. So while, as I say, the data might not be quite as strong and we don’t have maybe that concept of a “weed out class,” I think it still doesn’t preclude the fact that a concern to foster a community that is relationship rich is important, regardless of the context.

John: I noticed in last summer’s PHELC symposium, there was a paper that you presented Ann Marie with Peter Felton on relationship-rich education in large-classes. Could you talk just a little bit about that?

Ann Marie: I suppose myself and Peter, in his context, would look at very large classes from the perspective of building a relationship, that it is possible to build a relationship, whereas sometimes, I think it can feel, sometimes, to some higher education teachers, that it’s almost impossible to have a relationship but you don’t know anyone’s names, or you might know very few names. But you can build a relationship with a really big class in different ways, even though you don’t know their name: very simple things like smile when people are walking in the room, say hi as they’re passing you heading up to the seat, have a wander around the room, don’t stay stuck down at the bottom, move. So like when I’m in a class, I share my proximity. I wear a headset. I move away from the podium. I’m up the back while I’m speaking, looking over the shoulders to make sure they’re not booking flights or watching YouTube while I’m chatting. And we use different teaching approaches and strategies in that large-class context to mix up how we relate with the students like Anna and I co-teaching. So when we co-teach, we tend to use a workshop format for at least some part of that session, which then allows myself and Anna to walk up the steps of the room. We chat to small groups. So we literally, physically bring ourselves closer to the students. You can have a bit of fun in class. If you do get to know some of the students, you can invite those students to help you change the dynamic, the colder dynamic, to make it warmer, then we can talk about that in a minute, in terms of chatting with Class Reps and that sort of thing. So I suppose in the first instance, it being the potential of having a relationship, like, if you take it completely out of teaching, really huge groups like Coldplay, Robbie Williams have a massive relationship with 10s of 1000s of people standing in front of them, and they don’t know any of them, but there is a feeling, there’s an empathy generated in that really large context. And I’m not saying that we have to be Robbie Williams or Chris Martin at all. That’s not what I’m saying, but it’s just really in the teacher’s perception of what is possible there. And I know with my work with large-classes, they are my favorite. My final-year students, I’d say I know names of 10 of them. There are 430 in the room. I love that class because there is such a two-way flow in communication in the room and between sessions on back channels. It could be something like we have, in the Moodle we use here, an in-built chat function, then a module page on Moodle, but something like Padlet where students can ask questions anonymously without having to put their hands up in a really big group, allows to build up that relationship as well. And it doesn’t just mean you answer the questions in class if they ask them in class, but it also allows them a way to reach you between classes, so if something might occur to them afterwards. So I would check that kind of a back channel between the sessions, and whether I answer the questions in class or not, I answer them in writing, one way or another, on that back channel. And you don’t even need technology in class. If people are a bit reluctant to speak, you can use a scrap of paper or post it, or something like that. It’s just to get the student voice and to capture that whatever way you can, so that you’re capturing their voice and their feeling that you’re listening and that they are engaging at that level in the class. So all of those sorts of strategies build up a teacher-student relationship. The way you would in small class of 20 may be a bit more naturally, but you just need to think about it in a different way. That’s about perception, around how you perceive or what you perceive as a possibility in that very large context. And it means that, in my own case, when I look at like I’m currently, this semester, working with 430 final-year Bachelor of Education student teachers, and they’re my favorite class. I love them, and I feel I have a relationship with them, and I feel I have a good relationship with them, even though, out of the 430 I probably could name 10 people outside, but I feel I have a relationship with them in spite of that. But just I think it is how we as teachers perceive numbers and what’s big. What does a large-class mean, actually? So for me, I think 400 is large, and then I visited somebody working in Sweden, who has over 600 in his classes. So mine suddenly seems very little compared to his.

Rebecca: I think your connection of using performance or concerts or theater as an example of where you can feel connection to performers is a good example of how students can feel connected to their faculty in these large settings.

Anna: Ann Marie has mentioned, and she’s right, that it’s about our perceptions of size and what’s possible and the possibilities. There’s also a little bit of it that is about students’ expectations and students’ perceptions, and sometimes a lot of our large-class teaching is in your traditional tier lecture theater with a podium at the front. And it’s no accident that Ann Marie has a great relationship with her final-year students, because she works with them in that large group in first year. But it’s also, I think, about over time, building their expectation that they are valued, that, as Ann Marie said, simple things like smiling, that there’s an expectation that we want to have a relationship with them, that we value that, that it isn’t that disembodied voice that’s standing at the podium and lecturing at them, that it is about relationships and it is about community and assessment is another way in which we can set that up, too.

Rebecca: We’ve talked a little bit about the differences in how you might build relationships in large versus small settings. And Anna, you also talked about the difference in your experience of going from a big class to a small, intimate doctoral setting. [LAUGHTER] Can you talk a little bit more about, from the perspective of faculty, the difference in instruction between teaching these large classes and teaching small classes?

Anna: Yeah, again, I think it is, apart from thinking about the equity challenges and the message that that conveys, sometimes, some of the literature talks about information transmission approaches in a large class, versus dialogic teaching and exploration and so on. And it sometimes strikes me, and having worked closely with Ann Marie over the years as well, that actually the difference in the large class, as Ann Marie said several times, it’s about kind of not just ruling anything out, but thinking, well, if I do want to create a relationship, or I do want to elicit learners voice in that context, how am I going to do it? Rather than just saying, “Well, it’s just not possible, and it has to be that information transmission piece.” For me, it strikes me that in order to do it well, the instructor, the educator, has to be much more deliberate, much more careful in their planning, not just thinking about their learning content, but thinking about the things we’ve spoken about, like the relationships, like, how am I going to elicit students voice? How am I going to build in formative feedback? Because it’s not, perhaps, as natural as in that context, where I was working with six or eight doctoral students who were coming to the session with me, having set prior reading and that they were anticipating and fully expecting. Their expectation was that was going to be dialog, lots of exchange of views, and that’s why I think two little things can make a difference that break down that barrier in the large-class, in terms of setting our expectations of what’s possible, for not just for ourselves as higher education teachers, but also for our students as well.

John: In large-classes, students are often somewhat reluctant to volunteer responses or to ask questions. You’ve already addressed that a little bit in terms of having a backchannel. Are there any other strategies that you use to help students become more comfortable participating in larger classes?

Ann Marie: Yes, so I usually, when I have a large-class, email the Class Rep prior to the module beginning, and I try to meet them, either online or face to face for 10 or 15 minutes, at least for the first couple of weeks. And when I meet them, I request that they will help me to teach in the first couple of weeks by asking and answering questions in class. In most cases, they kind of step back a bit in horror and because they don’t like speaking in the large-class context at all. And these are Class Reps who have put themselves out there anyway, but they’re still not comfortable in that context. So what I usually do is I go through the forthcoming session, I tell them the questions I’m going to ask, and I give them potential answers to those questions. And then for other parts of the session, I give them questions to ask me, and then I get one student to monitor the back channel so that I’m not reading out the questions that have been posted. It’s a student reading it out, one of the reps as well. And I have a little mic called a CatchBox that’s in a foam dice type item. And it’s a microphone that can be thrown around the room like a foam football, and it allows the students to speak in their normal voices without need to raise them. So the reps that I have set up then models answering and asking questions. They model the use of the CatchBox. And even in the first session, you find that some people that I have not planted who are on the edge of courage tip over and have the courage to ask the questions they’re dying to ask. And by the time they get to the third session, I don’t need to set up anyone anymore. There will always be a few people who will ask questions, and the fact that the student is monitoring the Padlet or the backchannel of the typed and posted questions by default, means you get student voice as well. So I have found that a really useful way of gathering the student voice, and also, if you do set a task in the class, or if Anna and I are co-teaching in the workshop, because they’ve spoken, we do something like a think-pair-share where they’ve chatted one or two others, they have a bit more courage to respond as well, because they’ve already sussed out the response. They’ve already sussed out the question. They’ve teased out a couple of things between themselves, plus we’re walking around the room answering questions. So sometimes when that happens, I might be chatting to a group, two or three, during the task, and we have a little chat, and then I say, “Right, I’m going to come back to you now with the CatchBox, and I want you to say the whole class exactly what you’ve just said to me,” and again, you get a little worried look, but one of them will do it. So again, I think Anna, what you said there is right, the very deliberate actions in that big setting really help to gather the student voice.

Rebecca: You’ve talked a bit about student reps. Can you talk a little bit more about what they are and how you’ve designated them,

Ann Marie: They either volunteer or sometimes there might be a vote amongst the student body. So every year group, on every program, has student representatives, and the number of representatives vary depending on the size of the year group. So in a year group with 400 students, there are a minimum of six class representatives, and they’re volunteers. They’re linked with the students union, and their role is, I suppose, as a liaison or a communication between the union, between the Program Chair, and the staff of the program, and the students. So they’re a conduit for information between staff and students and between the university and students.

John: You mentioned using a CatchBox, and I’ve always wanted to use one. I saw them at a conference in SUNY Oneonta a while back. Do you toss it to specific students, or do you just toss it in the crowd to get responses?

Ann Marie: So the first time we bought it, I was delighted. I walked into the large class, and I asked a question, and I threw it in to the middle of the crowd, and they threw it back at me, [LAUGHTER] a few times, [LAUGHTER] so I never did that again. [LAUGHTER] So that is why I model it, that is why I set up the first couple of sessions. So the students I choose, or the rest, or whoever’s helping me, they scatter through the room, and then I make it very clear that when I ask a question, they have to put their hand up high so I can see them. And then I will throw the Catchbox to them. They throw it back to me. If I’m on my own, if I’m the only teacher in the room, with two aisles, I’ll stand on one aisle, and I might ask a random student to stand on the other aisle, and I’ll throw it across to that student if I’m trying to reach somebody on the far side. So in my experience, I would caution against just randomly throwing it into the group, because people are very happy to use you as target practice. It doesn’t hurt if it hits you on the head, by the way. [LAUGHTER]

Anna: In those large-classes, we tend to use a headset mic because we want to do all that moving around the room in terms of building relationships. And when we first started that, before we had the Catchbox, we used to have hand-held microphones, the ones that move around. But I think we’re all the same. We tried to put that up to a student who we know has something wonderful to add from the work, but they all recoiled in horror, but I do think they enjoy the Catchbox. But again, as Ann Marie said, it’s about being deliberate and setting it up. That’s why it works.

Rebecca: So you both talked a lot about co-teaching and working with each other. Can you talk a little bit more about the benefits of this practice?

Anna: Co-teaching is interesting because you mentioned at the beginning, we’re both in the School of Inclusive and Special Education. And I know the language about university structures is different, but basically we’re in a department of 18 academics, and we’re all focused on inclusive and special education. And I know it’s the same in the United States and in many countries in the world, but one of the models of support for children in elementary and high schools, for example, that is very often used as a way to support more inclusive approaches, rather than withdrawing children from their mainstream classroom, is co-teaching. So the concept of co-teaching, as in, let’s define it, let’s use a standard definition of two or more adults in a single physical space delivering instruction. So that’s the definition of it that is used in the compulsory education sector, in elementary schools and high schools. We’ve looked a bit at the literature. The definition of what constitutes co-teaching is not at all as clear, it seems to us, in higher education, because co-teaching can be used just to describe a number of faculty members, sequentially teaching separately, within a module or within a course, for example. But, I suppose, coming back, co-teaching is an important concept. It’s kind of a threshold concept as part of looking at inclusion for a lot of our work with undergraduate students in those large courses and also larger courses. So a few years ago, I would have been responsible for teaching content in relation to working with special needs assistance, but also delivering some input on co-teaching. So it’s a concept that we want our students to grapple with, but we also enjoyed teaching the large class. So in the interest of modeling, walking the walk, talking the talk, Ann Marie invited me into the classroom to model that co-teaching. That was a really interesting experience for us. We did a little bit of research about it. We got students’ perspectives, but we also brought in some observers to observe us co-teaching. We know the literature about co-teaching, that for it to work well, it has to be very carefully planned. And we did plan. Initially, Ann Marie, I think we spent a lot of time really thinking, mapping out that 50-minute session. Typically, we’re talking about a 50-minute session, later we did longer sessions, but we really mapped it out, and what exactly I would be doing at a particular point in the lecture, what Ann Marie would be doing, what students would be doing. And we also thought about the kind of different models of co-teaching, because literature talks about lots of different models of co-teaching. So we thought about those, and what were we actually teaching? The really interesting thing was that the observers told us we needed to be more explicit, because, again, it’s no more than that kind of notion of maybe students come into the university and come into that tiered lecture theater, and they think about it like a theater, for example, and they think, “Well, I’m here to be passive and and I’m then going to get these wonderful thoughts from the professor on the podium at the front of the stage. So they also seem to be, in some instances, a little bit thrown by the fact that… Ann Marie, you might talk a bit more about that… about the two of us being in the room. It kind of threw them out of their comfort zone in terms of what was the expected model. So our observers noted that and reminded us that we needed to be explicit. This is why we’re modeling this. We’re modeling this because we think it’s really important for you to understand this concept and how it works, because you’re going to be a teacher. That’s taken me well away from the original question, which was about why we were using it. We were using it because we wanted to replicate workshop type activities that we would typically engage in with a much smaller group. And one of the supports for doing that was to have more than one teacher in the room.

Ann Marie: Anna said one of our reasons for using it was to model the process of co-teaching, because our students are going to be teachers. But other people have used co-teaching to model, let’s say, social workers. I can’t remember the author, but there is a paper on social work education and social workers using that co-teaching model to model different perspectives around a case conference or something like that. So it also enables, if you’re a higher education teacher, engaging in something that’s a contested concept or a contested idea. You can use the co-teaching model to show that contestation, because the two people are very likely to be coming into the room with perhaps different views and different insights. And you can model that carefully in that context. The most important thing, I think, around the co-teaching, is that you’re fairly comfortable with the person you’re co-teaching with, and that it is carefully planned. It’s a fair bit of work when you first do it, but once the planning is done for a session, like Anna and I started this about 14, 15, years ago, and where we have continued to co-teach for subsequent years, the work’s done. We know what we’re doing, but it was a heavy investment of time the first year we did it, but after that, it’s worth it, absolutely worth it, because it enabled us to do workshops with very large groups that I would have struggled to do on my own.

John: And it can provide a really rich experience for students. When I was a college student, two of my most memorable classes were team taught by two instructors. One was on bioethics, where there was a biologist and a philosopher, and the other one was on the philosophy of social sciences, where there was a philosopher and a sociologist teaching together with very, very different perspectives on all of it. And the ability to go back and forth and present different perspectives is really healthy. You had mentioned, though, doing some work with think-pair-share and some group activities. Could you talk more about what other type of small group activities you do in your large-classes?

Ann Marie: So usually what we do in those classes, it depends on the workshop, or it depends on the focus. So for example, on Monday, a couple of days ago, myself and another one of my colleagues, co-taught a workshop with the ED 4 group, and the focus of the workshop was on their assignments. I use universal design for learning in my own pedagogical practice with these students to model how to use it in teaching and learning, and part of the Universal Design is in the assignments. So I give them an awful lot of choice in terms of topics. I use authentic assessment. I’ve asked them to write for an audience that is not me. They choose their audience and so on. But they’re also allowed to work with who they want and how many they want, between two and five. Some people are very comfortable working in small groups, others are very happy to work with four others or whatever, but I give them all those choices. And then last month, the workshop we ran was for one and a half hours on planning for that assignment. So my objective was they walked out of there at the end of that session, before our reading week next week, with a plan of action to tackle that assignment. So the way that was structured was, I suppose, almost like a dance between myself and themselves. So I brought them through each criterion, they were supported by a writing frame that I gave per group, and I asked them to complete each step of the plan for their assignment based on the criteria and the expectation of the structure of that so that’s an example of a type of in-class task where they work together, in that case, choosing who to work with. And we stepped it so that we have six steps, but they completed one step, myself and the other teacher, Christina, wandered the room, and we came back after each step and summarized what we were seeing and filled in the gaps and then moved to the next step. That was a full tiered or stepped task. In other workshops or in other contexts, we might ask students to just talk about a key concept, they might not necessarily write anything. Give them five six minutes to talk about it. We’ll wander the room and then come back together and ask one or two to share with that. Sometimes, we might use the backchannel on our Moodle page for them to post what they’ve taught or things like, what does the word exclusion mean to you in a school context? So it just depends. We don’t structure the group because we can’t. They’re too big. The other thing I would say, though, is, in terms of very large-classes, it often, sometimes with me and often exclusively in other modules, they’re split into small groups, so they’re in workshop groups of 13, 14, or 15 workshop groups. In some modules, they’re never together as a big group, and there’s a bit of thinking, because it’s in a large-class, but what you’re managing there is the consistency across teachers’ input in class, formative feedback and so on. So yes, the group is smaller, but you’ve a whole curricular planning thing there to consider that I don’t have to consider when I have a big group. However, when the big class is split into those smaller groups of 20 or 30, you know who you have in your group, so you can plan for those groups of 30, whereas in my big class, I don’t know who’s talking to whom, so I can’t manage it at that level.

Anna: And we mentioned the think-pair-share, we would use that quite a lot, but using the same activity, but with lots of different kinds of materials. So it might be looking at samples of children’s work, for example, and within that group in the large lecture theater, we might roughly divide the room into four areas, and each quadrant, let’s say, would look at a particular piece of work, and then we’re looking for them to listen carefully when we take feedback from, it might be the same child’s piece of work, so they’re invested in kind of hearing of the other groups, while they might actually only be actively working in their small groups or in their pair on one piece of material, but they’re actively taking feedback from the activities that the other learners have been engaged in in the large-class. The other thing that Ann Marie uses really effectively is the panel idea within a large-class, bringing in a panel of speakers. I think that’s particularly effective in that large-class.

Ann Marie: It’s highly engaging. So with one of my groups, the module is on inclusion and inclusive pedagogy and inclusive practice in education, all contested terms. So to model the multiple perspectives, I bring in panels of guest speakers. So for example, I have one panel of parents and young people who talk to my students about their experiences of primary school. Now you could hear a pin drop [LAUGHTER] in those classes. It’s highly impactful, 10 times more impactful than me summarizing what the literature says about parental and student experience of inclusion. It literally demonstrates multiple perspectives. I have a second panel then, with teachers in different teaching contexts, and a third for a different team. So in other words, with the very large class, that’s this particular final-year group, every session is different. So I use three sessions where I have panels, I’ve two or three workshops where we have asked students to engage in a particular task, and a couple of sessions that are much more like a traditional lecture, albeit that we try and find a space for them to do something in that session, just to break it up a bit. So no two sessions are the same, and I try as well with those very large classes. Your question to Anna was about losing students, particularly in first year. I do wonder sometimes, is it not so much the class that’s large, but the program, it’s not just your module and your class is large, but lots of classes are large. There’s a multiplicity of experiences those students are trying to navigate. The large-class context can be the glue that brings some of those isolated learning experiences that are at a modular level back to the class. So for example, I think, after reading week next week, two of the co-taught sessions are with people who are teaching other modules with this same group. So they’re coming in to my session, we’re going to co-teach the session. They’re going to come in and make the links with what they’re doing in their modules, and I’m going to make the links within our module so that they’re getting an opportunity to see two of us working across the program, as well as somebody like myself and Anna, who are within the same module, but these are people in different modules. I do think that goes back to helping students make the links and the engagement for them to see that the two teachers in two completely different modules are now working together.

Anna: The other thing I would just add, in terms of Ann Marie’s panel, for example, in the panels that Ann Marie brings in, there is something particularly impactful of those real stories, real lived experiences. When they are heard in a very large context that enhances the impact of the experience for example. Because we were just talking there a couple of days ago, Ann Marie and I, and sometimes we talk about engagement, and I don’t know whether it’s the case in other universities, but a big focus on active engagement. There’s almost an assumption that that means that learners have to be up and about doing something, writing something, engaging with a student response system like Vevox or Menti or whatever it is, listening and listening to a powerful story like that, particularly when you’re part of a very large group, and it’s a powerful story, it enhances the impact of that, in fact, being in that large group.

Ann Marie: It does. And by the time we got to our third panel last week, one of the students in the group was on the panel. She came to me with something that had struck her, and she agreed, with a little bit of persuasion to be one of the panelists. And that’s powerful.

Rebecca: It always strikes me how, when I’ve had guest speakers come in and talk to students, that they can say the same thing that I’m going to say, but if someone else says it, it’s truth.

Ann Marie: Because it is their truth. So all of our speakers are speaking from personal or professional standpoint. It’s their lived experience, whether it’s personal or professional, whereas sometimes we might be talking about someone else’s experience sort of a step away. And I suppose that’s another thing, especially in the arts classes, is sharing yourself with the students and just giving them an insight into your experience. In my case, my past teaching experience, my own experience of my son’s engagement with school and little hurdles he’s had to hop over and those sorts of things and that, in terms of relationships in class, makes a difference as well. You suddenly were human, rather than the person who popped in for 50 minutes, dropped the pearls of wisdom and then runs out the door again.

John: What types of formative and summative assessment techniques have you found to be most successful in large-class instruction?

Ann Marie: I suppose some of the things we’ve mentioned are based on formative assessments and feedback. So a lot of those workshop type scenarios, and even when students are working on different case studies and different items and materials on the Moodle page, we will then provide feedback on all of those items, including the ones they’re not using. And sometimes I’ll use a little screencast, just a 3, 4, 5 minutes, to explain and fill in the gaps by feel from the class that they maybe they haven’t got it or whatever. Summative assessment can be trickier, and I think one of the challenges higher education teachers face with very large cohorts, the biggest one might be around the assessment and being fair, being manageable and so on. So I’ve grappled with this. I’m still grappling with it. I’ve two very, very large-classes, and I do something different with each and that’s because of the amount of content in the module that I’m part of. Other people who also have assessments in it and so on. With that fourth-year group, I have found that taking on the perspective of Universal Design for Learning in designing assessment and coupling that with authentic assessment approaches and viewing assessment as learning rather than of learning has changed how I experience the assessment process. So with that group that I have at the moment, for example, I have 430 students. I’m going to get, this December, about 140 assignments across all of them, a fair bit of work in that, in terms of marking and feedback and everything. I won’t deny that, but the most interesting set of assignments that I get, because they’re all doing something different, and they all present, so I’ll get websites…. They’re all creating a resort for a named audience, but they can create it in whatever way they like, and they really seem to buy into it, because they’ve got so much choice. So with that group that works, the multi-layered choices work, who to work with, what to do, who to write for, because they’re fourth years. But with my first-year group, which is 450 in it, I don’t offer those same range of choices. In that case, they have a fair bit of assessment in that particular module, so I have to cut my cloth to be fair to the student, and I’m not entirely happy with the assessment I have. It’s very applied. It’s based on case studies. We do similar, aligned case studies inside the class, in a workshop style format. They’re the co-taught sessions that Anna and I do, and then they do something similar themselves, but I still haven’t nailed that large-class first-year group, giving them options around what to do, who to work with, isn’t as fair as it is for the fourth years just because of where they’re at in the course of the size of the module they’re engaged with. So that’s my next task now, is to have a think about that and changing that before I meet that particular first-year group. It is challenging, but I would say particularly with my fourth-year group, the Universal Design for Learning Framework and applying that range of choices slowly and iteratively over time to see what worked, has completely changed my assignment/assessment marking experience, completely.

Anna: The other side of that coin, of course, is the feedback piece, which, as Ann Marie has said, the assessment piece at scale is particularly challenging. So is feedback. And again, it’s a bit like sometimes I do a lot of group feedback using exemplars as proxies for feedback. Sometimes, I think lots of higher education teachers have the same experience. Students nearly need to be hit over the head and told, this is feedback. This is relevant to you, that unless it’s actually on their assignment. And I know investing a lot of time, Ann Marie has said, in that first-year model where we’ve done the co-teaching, we model the task, we scaffold the task. There’s a lot of connections between the Moodle DLE in the classroom, so there’s lots of exemplars there. And that took an awful lot of time, sort of building that up when we started first, a lot of time, but it doesn’t change. I think of the example, Ann Marie, where we were giving feedback, asking students to write S.M.A.R.T. learning targets. That’s kind of a universal element, for example, in teacher education. And the investment in time in giving collective group feedback on the DLE, on various iterations of less and more specific, measurable, achievable, realistic, time-bound targets. It’s a lot of investment time initially, but it doesn’t change.

Ann Marie: Then it’s done. It took us hours the first year, then it’s done. You get the same type things back from following year groups.

Rebecca: So we always wrap up by asking: what’s next?

Ann Marie: Well, we are rethinking PHELC. So we’ll watch the space for that. We just need to think about where we might go with that, how we might expand it, maybe beyond us, and what we might do above and beyond our symposium. And then, as I said, in terms of the large-class teaching, it’s always what’s next. Like no two years are the same. We change. I love changing it. I’m basing the feedback from students as well. Each iteration of a module, the delivery of a module, might be a little bit different, and I am hoping, particularly with that fourth-year module, that the assessments that some of the really excellent ones, I’m developing a repository for those, which I’m hoping will become a site for publication at undergraduate level, in the first instance, within the university, but maybe further afield down the line. That’s a long term, big project that I’ve just dipped my toe in the water off. So I’ll let you know. [LAUGHTER]

Anna: And I suppose, just beyond our own work, one of the things we mentioned was the co-teaching and how much we have enjoyed over the years working together, and that has mainly been in the large class. So for me, and I mentioned that, one of the things we’ve done was we invited people to come in and observe. So one thing that I’m really interested in is that opportunity to be in other large classes, to be that observer, to maybe support other colleagues in their large-class work and so on, and any opportunity I can get to co-teach and to work with other people out there, embracing.

John: Well, thank you. This was a fascinating conversation, and we look forward to talking to you more in the future.

Anna: Thank you, John, and thank you Rebecca, for the opportunity. This was great.

Ann Marie: Absolutely. It’s been lovely to speak with you. Thank you for inviting us.

Anna: Indeed.

Rebecca: Well, thank you for some great ideas.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

366. Connecting to Core Values

Posted in authentic learning, classroom community, diversity, engagement, equity, evidence-based practices, first-generation, growth mindset, higher education, inclusion, income inequality, learning, motivation, pedagogy, racism, social justice, STEM education, teaching
November 6, 2024

Engaging and motivating students starts but doesn’t end with inclusion. In this episode, Bryan Dewsbury joins us to discuss ways of connecting class content to students’ core values to prepare students to productively engage with their communities as we work with our students to make our disciplines more equitable.

Bryan is an Associate Professor of Biology at Florida International University. He is the Principal Investigator of the Science Education and Society research program, an Associate Director of the STEM Transformational Institute where he directs the Division of Transformative Education, and a Fellow in the John N. Gardner Institute. Bryan is also one of the co-authors of The Norton Guide to Equity-Minded Teaching. He is the developer of a free MOOC on Inclusive Teaching, offered through the HHMI Biointeractive and Science and Education Society. Bryan is a highly regarded keynote speaker and workshop leader.

Show Notes

Transcript

John: Engaging and motivating students starts but doesn’t end with inclusion. In this episode, we discuss ways to connect class content to students’ core values to prepare students to productively engage with their communities as we work with our students to make our disciplines more equitable.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Bryan Dewsbury. Bryan is an Associate Professor of Biology at Florida International University. He is the Principal Investigator of the Science Education and Society research program, an Associate Director of the STEM Transformational Institute where he directs the Division of Transformative Education, and a Fellow in the John N. Gardner Institute. Bryan is also one of the co-authors of The Norton Guide to Equity-Minded Teaching. He is the developer of a free MOOC on Inclusive Teaching, offered through the Hhmi Biointeractive and Science and Education Society. Bryan is a highly regarded keynote speaker and workshop leader and he recently provided the keynote address at the Academic Affairs Retreat here at SUNY-Oswego at the start of the current semester. Welcome back, Bryan. It’s good to see you.

Bryan: Thanks for having me back. I hope that the welcome back means that I was good the first time. [LAUGHTER]

John: People are still talking about your keynote. They were really impressed.

Bryan: Well, I’m glad you all enjoyed it. How are you all doing?

Rebecca: We’re doing great. Today’s teas are:… Bryan, are you drinking some tea?

Bryan: I’m not right now because I actually did have some for lunch. So I’m on campus right now at FIU, Florida International University, and I had lunch with a colleague, and we did have some tea. So, does earlier count?

Rebecca: Sure. Do you want to tell us what you had earlier?

Bryan: I don’t know if I said this in the last episode I was here, but I grew up in Trinidad. I know I said that part, but when I was growing up, the main tea that at least my mother would buy is Lipton. Like it’s almost to the point where most of us didn’t know there were other types. [LAUGHTER] So you come to America and you see, like 19 labels in the grocery, but of course, because of that, that’s kind of the tea I like the most, because that’s just what I was used to. So when I do see it, I like to have it. I’m not a big tea drinker. Oh, that sounds like sacrilege to the podcast, but when I do drink tea, that’s sort of my preference, and it’s just maybe kind of a nostalgia thing. So that’s what I had.

Rebecca: That makes sense.

John: I have a tea which is new to me. It is a black tea with cherry from one of Rebecca’s local tea shops, and I forgot the name. I had to rush here from a meeting, so I didn’t get a chance to jot down the name. But it’s delicious.

Rebecca: Oh, I’m glad you enjoyed it. I’ve mentioned it before. I think the smell of the cherry teas are really good, which enticed me to buy them. And then I discovered I like, absolutely hate the flavor of cherry teas.[LAUGHTER] And so I gifted them all to John because he likes them. I have just a Scottish afternoon tea today. John.

John: Very good.

Rebecca: Nice and plain, without any cherries. [LAUGHTER]

John: We’ve invited you here today to discuss your work on inclusive teaching. Could you tell us a bit about how inclusive teaching has become a substantial focus of your work?

Bryan: Yeah, so it’s a good question to answer, because, how do I put this? I actually don’t think it has. It has, but it hasn’t. Let me explain why. I do talk about inclusive teaching a lot, but the real focus of my work is education that is designed in a way that everyone can access it, and when they access it, they can be their best selves in that experience. And when they are their best selves, then they become this really engaged, loving, thriving participants in an evolving democracy. That in three sentences, if that was three sentences, that’s the thing we are trying to unpack as a group, not just through research, but through communication, through professional development. And so the inclusive part is a step towards that big a goal, because for the thriving to happen, for the access to happen, we have to sort of figure out, what are the barriers that are currently present that don’t allow people to be their best selves when they’re in a classroom, that don’t allow students to access certain institutions, that don’t allow them, even when they do access them, to graduate with the kind of credentials and the kind of mindsets and ways of knowing, ways of being, that allow them to understand how crucial it is for them to be engaged participants in a democracy. So inclusion is part of the riddles that we are trying to solve, but it’s not the end point, and to that extent, we have written a lot about it, I’ve talked a lot about it, but I think it’s important for people to know that our goals are bigger than that, without diminishing inclusive work, our goals are much bigger than that.

Rebecca: Can you talk a little bit about your work in the STEM Transformational Institute in particular? I think it’s interesting in that role, how the role of democracy in your work, in moving democracy forward, fits into STEM because we don’t always hear those in the same sentence,

Bryan: Yeah. Well, yeah, or never, right? So I think that there’s two questions you’re actually answering. So I’ll talk about the institute first. It’s a pretty remarkable group. I’m one of several fascinating individuals who work here on the faculty side. Many of us, like myself, have a departmental appointment, whether it’s biology, engineering, physics, math, chemistry, and some other percentage of our appointment is with the Institute. What that allows is most… all of us, actually… were trained in some STEM discipline at the undergrad and PhD level. And so we can teach the classes. We can teach basic science classes. We can think about that ecosystem and that culture, right? Biology culture, physics culture, but discipline-based education research compared to other fields of research in biology, physics, or any of the above is relatively recent. And I think there is an understandable challenge of traditional STEM scientists in understanding and evaluating the kind of work that people like myself do. And so having a place like the STEM Transformation Institute creates several things. Number one, a community, for those of us who are thinking about education research in STEM to collaborate, to be in community, to ask questions, to write grants together. But also, there’s a mechanism for us when we kind of go through the ladder process for the STEM transformation Institute, to contribute some kind of understanding, to put all packages in perspective. Obviously, when you have this amount of faculty, and I think it’s maybe 20 of us, I can’t remember the exact number, some economies of scale benefits come with that. And so you can write bigger grants, because you have the intellectual capital there, you can have much more collaboration across the different STEM disciplines and with non-STEM disciplines. And then the Institute also has a unit called FIUTeach, which mainly focuses on training pre-service STEM teachers, a pretty robust, maybe the nation’s largest learning assistant program, undergraduate learning assistant program. And there might be some other pieces to it, but those are kind of the three things we tend to be known for on campus. So now, pivoting to your question about STEM and democracy, without taking up your whole podcast [LAUGHTER] giving a history lesson on Enlightenment thinking, the STEM instruction that we are seeing in classrooms today is really a product of maybe, I would say, a mix of the post-World War Two US higher ed-government alliance that kind of proliferated at that time, and the Humboldtian model of graduate education that became popular in the early 1900s. I think John Hopkins may have been the first place that some of that started and then some of the Sputnik-era stuff. And so the way too superficial summary of it is, if you were trained as a subject matter expert in STEM, your role in class was basically to tell people stuff. It wasn’t, perhaps said explicitly, but you were a big gatekeeper to that information. Because, remember, there was no World Wide Web, there was no Google search. In some cases, there wasn’t even really a lot of textbooks, right? And if there were, prohibitively expensive. So the only way you got to find out how big the glaciers were in in the Arctic was to hear the person who got on the boat to go there to talk about it. And there’s some other pieces to it. And I would encourage listeners to read the book Amateur Hour by Robert Zimmerman, because it’s discussed way more specifically and way better than I’m describing it, but there was some gender dynamics to this, because remember the normal schools that existed in the 60s and 70s, and who were the faculty there, and then, because of that association, how college faculty felt about teaching as a skill in and of itself. So there’s a brief period where there was some discussions about college professors getting trained. That died. [LAUGHTER] That didn’t last very long. So okay, I’m saying all of that to perhaps take us even further back to the role that Enlightenment thinking played in the Declaration of Independence, and more specifically, the influence that Enlightenment thinkers like John Locke, in particular, who wrote Treatises of Government played an influence in Thomas Jefferson and some of the language that you see in the Declaration of Independence, and I’m thinking of one line in particular: “We submit these truths” to “a candid world…” Book recommendation: Jill Leporie These Truths, some of this is me interpreting her writing. It’s basically kind of saying, Look, man, we have these 13 colonies, we’re gonna try this out. We’re not gonna get everything right. We’ll see what happens and we’re gonna have to keep working at this. And in the same way, the scientific method and Enlightenment thinking comes with some expectations of critique, self critique, public critique, falsification, seeing new evidence and changing your mind on things and all of the above moving towards the sense of whatever you’re trying to understand. It improves over time because you are willing to keep refining and refining and refining. So in so doing, STEM, or specifically, science, has so much to teach us as citizens of a nation that is constantly trying to improve and what an improvement process looks like from a social perspective. And this is what I tell students: same when you collect data and okay, right now, it seems like you can’t disprove your knowledge, however you want to put that, but somebody’s going to come across and collect different data and critique those assumptions and improve on your methods. And that’s how we have vaccines that could be made in like two hours now, right? I mean, I’m exaggerating. So that same humility you have to display when you approach your intellectual pursuit is the same humility that I’m asking you to think about when you are in the public domain, when you are at church, when you are listening to news, when you are making consumer purchases, when you are listening to information come across your television. When you have to absorb all of the endless data that exists out there, that same critical mindset, but also mindset of humility that we ask of you in this culture of science and the culture of STEM, that has transferable lessons to other parts of how you live your life. And I think it’s a very under discussed component of STEM teaching, of how just STEM itself and science itself, the practice of science, is discussed with students. And I guess, Rebecca, what I’m basically said to you in five minutes is that we lean heavily into it, right? We talk about these things openly.

Rebecca: That’s great.

John: And we’re living in a world where we do need an informed citizenry, because there’s so many sources of information out there that are not so reliable, and the development of those critical thinking skills are incredibly important right now, given all the various sources of misinformation.

Bryan: And part of it too is I’m only… I don’t know if I should say only… but I’m 44, so I’m old enough to have grown up in a time when you had to go to an internet cafe to check your email for $20 an hour. We lived through this just remarkable transition of information ubiquity. I mean, I had a Encyclopedia Britannica set from A to Z like my mom used to joke with me, I would read it like a book. That’s how nerdy I was, right? And I remember looking at those 18 books or whatever, and thinking like, how much information is in here. You have that times infinity at the click of a button. I would just wonder, if we don’t pause long enough to reflect on what a sea change that is in terms of having easy and quick access to information. Now, that’s not the same thing as having knowledge. [LAUGHTER] So now the challenge has shifted. So if maybe initially it was being able to access that information and knowing all of the beautiful things that is happening around the world, now it’s how do you sift from all of this to pick out the information that’s useful, that’s credible, that’s reliable, that’s truthful. And I think our teaching and the way we think about teaching needs to shift with that. I don’t know, man, sometimes John, Rebecca, I have all due respect for online classes, this is not like a for or against, but I’ve always taught in person throughout my career, and I just absolutely love it. But as a commodity, and using the class that I teach a lot, intro bio, all of the learning outcomes I have in my syllabus, all of them, you can find a very good lecture on YouTube for free. You’re not gonna find maybe the test bank and all that. I get it, all of that, right? So in terms of accessing the information that we talk about in the class, the tuition and the three credits, and Bryan Dewsbury, I’m not a buyer. You just need a laptop and good working internet. So what is the added value I provide? What’s the added value that the institution provides by asking you to fork over a tuition check and housing and gas money to get to campus and all of that stuff, to physically be here. And I’m really curious, just as a thought experiment, if you were to put that question to all instructors, what will we say? What can we honestly say? Maybe 10,15, 20 years ago, you would say, “Well, how else would they know how DNA replicates? [LAUGHTER] I got a PhD in DNA, so therefore I have the right to tell you, right?” But I don’t think that argument holds as much water anymore, any water. And that, to me, is not perhaps the challenge, is the opportunity of the new information landscape.

Rebecca: It’s interesting that you bring up the encyclopedia in that kind of curated experience, but there’s also curated experiences online. Sometimes it’s hard to find those curated experiences to know that something’s of good quality, but even if it’s a curated experience, and even if it’s an inexpensive or free experience, you still have to have the motivation and the determination and the context, maybe, to put the pieces together to kind of figure it out on your own. [LAUGHTER] If you have something on the side, like you’re trying to do. How many of us have tried to sit and learn and do something on your own? I’ve been sitting and trying to crochet in the round in the evening, and I sometimes get frustrated using a video. [LAUGHTER] I want someone to show me how to do it, like, sit there and I can ask questions, and I want it to be interactive. This is just one example of just a thing that you want to learn from a person and have a dialog and be able to troubleshoot and have someone’s expertise to work through. And you think about the opportunity that you have as a faculty member to curate experience with readings and activities and classroom experiences, and then faculty maybe complain that students don’t engage in that full experience, and sometimes it’s because students can’t quite figure out what’s relevant to the next step or whatever. Can you talk a little bit about some of the ways that you might connect those assignments or other things that you’re doing in your classes to give students that relevance, or to actually answer your own question, like, what is it that we’re doing in our classes that give the students the reason why to come to class or why to do this experience, as opposed to do it on the side?

Bryan: Yeah, great question. Rebecca. So I’ll give you three examples that address your question, perhaps in slightly different ways. To me, another way of asking your question is when we do something in a class or when we have things in our syllabus or curriculum, why is it worth doing? That is a fair question for somebody, especially at the prices college is asking students to pay. That is a fair question to ask. And there are perhaps too many situations, maybe not intentionally, where there is an implicit eat your vegetables approach in that, “because I said it’s worth doing,” right? Like I have the great book, and I’m the PhD, and it’s worth doing, and so therefore you should believe. And I think my first part of my answer to you is to make that “why” more explicit. So we start the class not even talking about bio. First assignment is a reflection assignment that I shamelessly stole from National Public Radio called “This I Believe,” it’s on, it’s a public website, it’s: thisIbelieve.org/guidelines and the prompt asks… I’m summarizing here: “Describe the values that shape your deepest passion.” Before we get into anything technical. I want us to start this community by reflecting on what your why is, because your why, if you understand that clearly, or if you at least begin the mental process of exploring that, that is a big of why you’ll be motivated to be your best self, pursuing the technical piece. This could have easily been, you show up and I give you a bunch of materials, you show me if you know it or you don’t know it, and move on. But forcing you to really ask: “Well, why pre med? Why medicine? There’s so many things you could be doing in biology. There’s so many other great things you can be doing with all the other majors in this school. Why that choice?” So that sort of begins a very particular kind of conversation, because I expressed them my why and why I chose to be a professor, and why that was exciting to me, and why I love this class and love them. The second thing is, when we’re doing certain activities, regardless of what the subject matter is, they are implicit behaviors in how you do a technical task and get a solution that I think in a lot of times in a lot of teaching approaches, it happens as a function of the class, but it’s not made explicit that those behaviors are very important skills that you should be developing as you go forth as a technical expert. So when you put students in groups, for example, it might make things easier to grade, because now you’re just doing x divided by four or whatever assignment, there’s that piece. And I understand, like, you know, what the active learning literature says about group work and things like that, but what you’re really doing, my friends, is you’re teaching people how to work with difference. If you set those groups up right, If you talk about what positive group behaviors look like, what implicit biases look like, you are preparing them for a world of work, a world of engagement, where they will be in one type of group or the other. I can’t think of a profession… well, I could think of one, but not that many… where you will essentially be solo and so some of those group behaviors that really makes group work effective, like active listening, respecting silence, by hearing different opinions, being willing to accept critique, being able to give respectful critique. We have whole discussions about what that’s like and why it’s important. So it’s not just going to groups, and we hope everything works out. It’s going to groups, and this is a thing, and in order for this thing to go, well, let’s actually have a discussion about why that thing matters. The third part of my answer is, at selected points in the semester we try to make, or not try, we make very clear the ways in which scientific practice intersects with society. And this is where I think people get a little confused, perhaps, about what we mean by inclusive or equity-minded teaching. I remember once I saw on Twitter, and I know Twitter is what it is, right? But somebody put a statement that said, what is so inclusive about the mass of boron? [LAUGHTER] And I get where they’re coming from. I get why they would say that, but it, of course, betrays a misunderstanding of what we mean when we talk about inclusion in a classroom, there’s a lot of technical things that we discuss in an intro bio class, and not everything we talk about, we sort of bend over backwards to then relate it back to some social context. But there are things that do allow themselves to make that connection very, very clear. And the connection basically comes down to: science doesn’t exist in a vacuum. Science is done by people, and the people is human beings who ask those questions. Human beings ask those questions, and human beings decide which questions to ask. They decide which questions are the important questions to ask. They decide when they ask those questions, and they carry out studies, they decide how those studies are conducted. And they don’t only decide how those studies are conducted, they interpret what data they collect from those studies, and they write the narrative that comes out of that analysis, that comes out of that interpretation. This is not to demonize any particular individual who does anything on any one of those steps, but it is to point out the reality that since the bridge between the existence of nature and life and life science and our understanding of how this works, if the bridge is the human being or human beings plural, who help us to understand that, then by definition, the social positioning of that individual, of those individuals, may sometimes color how we come to understand the world as a collective. And we have slight examples of this: that they took, what 10, 15, years to ask what female chimpanzees did, because only men were in the field, right? And then we have very, very dark examples of this when we thought eugenics was a real thing and we were sterilizing disabled people in North Carolina. So throughout history and even today, we are saying to students that you will go forth to be some wonderful whatever you plan to be, whether it’s medical doctors, whether it’s health professionals, public health, whether it’s research scientists, whatever it is. And there will come a time when you will be a decision maker. You will be the one they come to to try to understand, how do we reduce COVID rates? How do we keep people safe? And when you get this, it’s important for you to understand that COVID doesn’t care who it infects, the differential rates of deaths and the differential rates of infection is not because the virus itself made that differentiation. It is simply following a pathway that was already constructed along social lines. And so if you really want to get at the root of the problem, you have to understand how those social lines were constructed. I’m not saying you have to get a PhD in Sociology, but I’m saying I have to know enough to be dangerous. And so that’s the argument that’s made to talk about Flint, Michigan when we talk about the effect of lead on the nervous system. That’s the argument we make to talk about Tuskegee when we talk about the study that took place there. And it’s not to say, and this differentiation is important, This is not to just say that “Hey man, America is a terrible place. We’ve done some really terrible things.” I mean, we have done some terrible things in history, but what the goal of that discussion is to say “but we can learn. We can do better.” We have examples of looking back and saying “That was a mistake.” As a social science researcher, every study I do, I apply for Institutional Review Board approval because those things happened. It’s unfortunate that those things had to happen, but now we know better. So it’s important that that piece is there because sometimes, unfortunately, the narrative or the interpretation of having these discussions is you’re just trying to make people feel bad. You’re just trying to make America to be this terrible place. And no, no, it’s a great place, because of the capacity to learn. So anyway, I hope that answers. Those are the three parts all woven together, Rebecca. I hope that’s fine.

Rebecca: Thank you.

John: So. much of what you’re discussing there is just finding ways of helping students connect what you’re teaching in classes to their own lived experiences, rather than just seeing it as this isolated material separate from the societal context in which it exists?

Bryan: Yes. Yes would be the short answer to that. And just to get into the specifics, right? I mean, I think sometimes a listener might hear like, “Okay, now do I have to go and find out every single student’s life specifics and find something that connects to that? And again, I’m not saying that that’s what you have to do. I think it’s good, and I appreciate that some people have very large class sizes, so sometimes some of these strategies can become overwhelming, depending on what your load is. But there are low-lift ways to get a sense of who your students are, where they’re coming from, what was the class they took before yours? If you’re at an institution where there’s sort of a very set matriculation pathway. If most of them are coming from local high schools, there are ways for you to get at least a summarized sense of the students that you teach. The other piece to it, though, is not just like, what’s your community, let me make it relevant. The classroom is also a place where you hopefully get introduced to an understanding of the world that you may not have encountered had you not come to that classroom. Part of the beauty of teaching is the ways in which we open students up to new experiences, new ways of thinking, including ways that may challenge the ideologies and ways that they had before they walked in the door. I’m not saying that this is what you were doing, John, but sometimes people think about culturally relevant teaching, and it gets trapped in a “if you’re from this neighborhood, let me talk about this neighborhood.” It’s like, well, no, it’s way more than that. It’s way more than that. One of the blessings of a country like this is the amount of diversity there is across so many lines, and you need to have to physically live next to the diversity, to know and learn and appreciate and just see its beauty. This is another place that college does it just through the classes, through texts, through learning, through studying, through inquiry, you learn more and more about the world and yourself. So it’s a little bit of both.

Rebecca: Along the same lines, one of the great things about being a faculty member is that same thing, that the students bring some really great experiences that we haven’t had either, and allowing them to bring their stories and experiences into the classroom allows us to continue to grow and learn on a constant basis too. It’s great when they can bring those things too.

Bryan: I mean, especially when you grew up going to internet cafes to use the internet. At this point, they’re coming in with high-speed computers in their pockets, right? [LAUGHTER]

Rebecca: Yeah.

Bryan: They grew up in a completely different environment. [LAUGHTER]

Rebecca: So you started off explaining how you get to know your students a little bit at the beginning, you start not even talking about biology in a bio class. Can you talk a little bit, maybe further down the line in the same class, and how civic engagement may enter into that same course later down the line?

Bryan: Yeah, so and this, you can perhaps consider this more social context than civic engagement, but I just so love this assignment and love the kind of discussions that come from it. We talk a lot in bio class about DNA as a molecule and all of that stuff, and it’s chemical structure. But we also spent some time talking about what led to the discovery of DNA. It’s a fascinating story. It’s a story about a time when there were not that many high-resourced labs in science. And so you can look up the book Double Helix, if you want to get a fairly brief version of it. But there’s also a story where if you were a woman in science, how the community responded to you, was very different to how they do now, although they’re pretty well documented cases where, not in all cases, that have changed, right? And so it does open up a conversation. First, we talk about the lack of credit that Rosalind Franklin initially did not get, but then we also talk about where we are now. There’s a really good documentary called Picture a Scientist solely focused on that issue, and we have discussions about why those things matter, and the role that power and hierarchies and power dynamics play in the experiences that people have. And I really want to emphasize, this is not to say I’m preparing you for a doom and gloom world and everything will be terrible. I’m preparing you for reality. And the reality is, I certainly wish for you a lot of great experiences, and I think you will mostly have that. But the reality is there’s still a lot of work to be done for us to get to the place of equity that we claim we have, [LAUGHTER] and the discussion of the gap between what we think we have and where we need to be is really my invitation to you to help work on that problem, but also to be aware of what may come when you get there. And I hope listeners take my comments in that spirit. So to the extent, Rebecca, that’s sort of an example of what you might encounter when you get out there in society. That’s one example of a conversation that takes place.

John: In addition to the issue of trying to make STEM classes more engaging for students, one of the problems that we see is that there’s some pretty significant differences between the DFW rates for first-gen students, for Pell eligible students, and students from historically minoritized groups, which is tied to differences, in large part, in the quality of education that they receive, and that serves as a barrier where many students enter into college planning to go into a STEM field, but end up not being as successful and then either transferring into other disciplines or leaving college altogether. What are some ways in which we can overcome the opportunity gaps that students face if they haven’t come in with as strong of a preparation for success in STEM?

Bryan: So a couple years ago, I published a paper about inclusive teaching methods and active learning methods, and a big crux of that paper, we talked a lot in the paper about what we did in the class and how much we cut content to really focus in on this sort of very social relationship-driven model of teaching. And in that paper, we not only reduced the DFW rates that you just described, John, but also when that group went on to future classes, they did fine, not just fine, they actually did better comparatively to peers from other sections. And some people, perhaps who hadn’t read the paper yet, kind of assumed that when we said everybody did fine, it meant everybody got As. And that is not the truth. We have a pretty high pass rate, but the class size average over the years that I taught it was in the 150s, it was a large class, and then maybe 10 or 15 years, there wasn’t a ton of students who… we had a very diffuse assessment model, a lot of quizzes, a lot of case studies and other stuff like that. And so what I’m saying that to say is that I don’t want people to feel that, for students who may come into your institution, who had 18 years of a social structure that wasn’t set up to engineer the best out of them. It is very difficult to right all those wrongs with one semester of an intro class. Now, at the same time, I can hold multiple truths in the same conversation, it is all well and good. In fact, this is just my personal belief, I don’t know if everybody shares it. I do believe, no matter what your circumstances are, even if they were really terrible, that within you there’s your potential to be great at science. There’s a potential to be great at biology. The question then becomes, given all of that history, what is needed to get you to a place where you can see that and take the reins of the chariot and go be your best self. And sometimes that may mean half of that first semester. Sometimes that may mean two semesters of Cs or failing a class and coming back. But the important thing is, I start from a place of the potential is there. I don’t know when the light will start to shine, but I’ve been doing this job for now almost 11 years, and I’ve been with students It took years for that light to switch. So what would happen, I can just tell you, on the ground with that intro bio class, for some of them that turn around point came like nine weeks in, and it’s one of the most satisfying and gratifying experiences as a teacher. When you see that student is like, “Ah, it’s not that bad.” And of course, by that time, there isn’t enough assessment for them to magically get an A between mid-October and December, but they might get to C plus, might get to B minus. They have to really nail a bunch of stuff. But here’s the difference, when they start the second semester, not with me, when they start the second semester of the institution, your now a different person. You’re not the same person who came in first year not sure of yourself, not confident, didn’t think you belonged in science, had a fixed mindset about math and all of that stuff. No, no, no. Now you know, yes, if I have the right strategies, I have proven to myself, and some of this is coming straight from motivation theory. I have evidence that if I take the right approach, good things will happen. And that’s why I brought up this paper. That’s why those C pluses in first semester turn into As in their second semester, because you’re not dealing with the same student. So that in a nutshell, is my lesson, in that there’s a whole discussion to be had about primary and secondary education. There are brilliant people having those discussions, read everything. Diane Ravitch writes, that’s a whole discussion we need to have in this country. There is also a role that institutions of higher education can play in that discussion, but specific to people who are privileged to teach classes, especially at the introductory level, that to me is a mindset I would like to have, assume the best, assume that it’s there, and know that the work that to teach, if I had to define teaching, to teach, is a degree to which you can walk a student from their current state of readiness to that place where they see what their potential can really be. That, to me, is the act of teaching, the content is part of the mechanism of the vehicle that drives that process. But teaching is a process, and that’s the process,

John: So encouraging a growth mindset is really critical to this process, it would seem. But the other thing is, you’d have to design your course in a way where you can’t use too many high-stakes assessments, especially early on where students could get deterred. You need to provide some mechanism so that students can, by the end of the class, get those Bs or C pluses and so forth, right?

Bryan: So, yeah, I mentioned earlier in this conversation a diffuse assessment model, right? And that’s done on purpose. So the psychology behind the high-stakes assessment, especially ones that are early. If it’s super high stakes and very early and they don’t do well a couple issues: One, mathematically, it makes recovery very difficult, you’re already down to a B minus after exam 1, you don’t have much wiggle room to even pass the class; the second thing is, and this is why I like to talk about teaching as a process and a relationship, because there’s a psychology to it… there’s a psychology that you need to understand, especially with first-year students, and what it means to transition into this environment. We don’t have enough time on this podcast today to summarize all the literature on emerging adulthood, on sense of belonging, on John Henryism, that all these things that are invisible, that are in the air and that even the students themselves may not be able to articulate, but it’s happening. And so what it means to be a teacher and be able to read the subtitles and see how certain behaviors are indicative of some of those processes happening, that’s the challenge. So your high-stakes exam, especially that first one, could be simply measuring the presence of those psychologies, not actual ability, not actual learning. You can just be capturing the fact that somebody is scared to death. You could be capturing the fact that somebody is carrying the weight of a generation at home and you must be a doctor, you must be this, you must be that. And that’s high stakes. This is playing out on that exam. They’re never going to say it to you that way, but it doesn’t mean it’s not happening. [LAUGHTER] And so part of the diffuse assessment model is to say, “Look, I am going to do my best to design assessments that accurately reflect learning, and the stakes are going to be high enough that I think you will take this seriously. But the goal of these assessments is to provide both of us, myself and you, the student, information that gives us some sense of where you are in this learning journey. And when we get that information, depending on what it says, we can have a discussion. We can have a specific discussion.” And I can say things like you do really well on recall questions, but you seem to struggle when I say explain. You have all of the language of this process in there, but it’s actually not coming together as a story. I can give you feedback that’s very, very specific, because with the sort of fixed/growth mindset dichotomy, giving you things that are changeable is what gives you an opportunity, it gives you an avenue to change your behavior. If I just said you need to try hard, what does that even mean? So I hope I’ve been underscoring the deliberateness of everything, everything, even from the way feedback is given, to how and why we set up exams in certain ways and how much we do. It’s all based on, again, this philosophy of teaching is a process.

Rebecca: I think we often use the word specific and say specific all the time, but you’ve just illustrated what you mean by specific, which is really helpful. So it’s really fun to talk, and we could talk forever and ever and ever, because there’s lots of fun things to talk about, and you’re teaching such interesting classes in interesting ways. But one thing we want to make sure we get to before we wrap up today is the asynchronous class that you’ve developed on inclusive teaching. Can you talk a little bit about this course and who the target audience is?

Bryan: Yeah, so this course was a collaboration between my program and Howard Hughes Medical Institute Biointeractive Program. It is based on a paper I wrote back in 2019 called “Deep Teaching in STEM.” The secret is that, yes, it says STEM because it went to a STEM journal, but the things we talk about in that paper really applies to all teaching. The other secret is that there’s nothing in that paper that’s particularly new. It’s just that we in STEM had ignored it for so long, and so perhaps that’s why we felt a need to re-center some of those ideas that, oh, by the way, it might be good to build relationships with your students and know who they are as people, and not just treat them as student ID numbers that need a grade. So that paper talked about five components that goes into what we call deep teaching, and those components then came to represent a module on that massive open online course. One of the key features of that course we wanted to really model was the very sort of introspective, spiritual, even soulful, way we think about teaching and we are asking listeners and participants in that course to be reflective. We wanted to model that in how the course itself was delivered. So we were very selective in who we asked to guest lecture, and the nine lectures are very, very long. We’ve infused at the end of each module. So there’s a main module, and there’s sort of four sub-modules that represent each component. Every main module, one of the sub-modules is an artistic piece. So we commissioned a painting. We had poetry, spoken word, even a Negro spiritual, and it was not random. They all thematically connected to what the module is talking about. And it was built in a Canvas shell, which is, as a MOOC, it’s free for anybody to access. But also what we’ve done is we’ve been working with a lot of universities who uploaded it onto their own Canvas shells and running faculty learning communities with a local facilitator going through the LMS. And the course has many practical, specific things. You want to think of a reflection in your course, here are some examples of prompts. So you want to have more transparency in your assignment, consider Mary-Ann Winkelmes’s framework. But what it also has, especially because of the nature of the way the course was set up, just a lot of moments for reflection. We didn’t want it to feel like a worksheet. You just go through this, do that. Go through this, do that. Like, when you hear somebody talk about motivation, like, think about when you were motivated, what made you motivated, and sit with that reflection, because that sort of empathetic approach is what really is going to make the course impactful for you. There are many 1000s of people who’ve just done it by themselves and have found a lot of meaning for it, but we’re really having a lot of fun seeing universities do it as a FLC on their campus. So that’s a nutshell. I think the web name is a little long for me to remember, but I know if you just Google inclusive teaching biointeractive, it should be the first thing that pops up.

John: And we’ll include a link to it in the show notes as well. I see that you have two books coming out in a little over the next year. Could you talk a little bit about those?

Bryan: Yeah, so it’s actually three, but one is not in the next year, right? So the two that I’m working on, one I’ve been collaborating with just a wonderful group, Peter Felton, Isis Artze-Vega, Dan Bergen, and Leo Lambert. Leo nd Peter wrote a book a few years ago called Relationship-Rich Education, which is just phenomenal, and that has led to a few follow ups. So we’re working on one of the follow ups to that project, so more details on that soon when that’s a bit further along. The other book, I’ve been collaborating with a colleague on campus. We’ve been interviewing people about telling personal stories, about times when they either felt a sense of belonging or have not felt a sense of belonging in STEM, and as somebody who comes from a storytelling background, from a literature background, the use of this discipline and this art form and this communication style, I think, is an underused approach in science in general, especially when you’re trying to convey something like that. And so I’m just very grateful for those people who’ve been willing to share their stories with us. So that will come on as an anthology, and, I guess dates TBD, but pretty excited about it.

Rebecca: Those both sound like really exciting projects. So we always wrap up by asking: “What’s next?”

Bryan: Well, I would say what’s next, like my most sort of next next, so I’ve been part of a group, a Gates Foundation funded group that has worked pretty hard over the last couple of years doing a landscape analysis that trying to identify what are some of the things that institutions around the country are doing to do equity-minded practices, especially in the intro classes. And that landscape analysis incorporated a lot of things like document analysis, interviews, data mining, and things like that, and we are launching the findings of our analysis as a report on October 25 in Washington, DC, at FIU, in DC’s campus there. It’ll be a free report. Obviously, there’ll be a Zoom link for anybody who wants to attend, and the report will be made available on our website. And we’re pretty excited about it, because there just is a lot of institutions doing great things out there. And to this point, we hadn’t seen a place where those great things were just collated in one place. And the obvious next step from this project is to take those great practices and say, “Okay, now that we know what’s good and who’s doing it, and how it’s worked, where they were, can these things all simultaneously exist in one institution? Can we learn from all of these best practices and say, if we have a goal of success for all on our campus, this is a place where we can look to to get some real both like comprehensive but specific ideas on how to make that happen. So that’s October 25 and pretty excited for that launch.

Rebecca: That sounds really exciting.

John: And we’re looking forward to hearing more about these books once they are ready to come out. Well, thank you. It’s great talking to you again, and we look forward to more conversations in the future.

Bryan: John, it’s always a pleasure. Hopefully next time in Oswego.

John: That would be great. [LAUGHTER]

Bryan: Alright, man, thank you.

John: Thank you.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

361. Becoming a SoTL Scholar

Posted in evidence-based practices, higher education, scholarship of teaching and learning, SOTL, teaching
October 2, 2024

A growing number of faculty members participate in the scholarship of teaching and learning, or SoTL. In this episode, Janice Miller-Young and Nancy Chick join us to discuss a new open educational resource designed to assist faculty interested in pursuing SoTL research.

Janice is a Professor of Mechanical Engineering and a past Academic Director at the Centre for Teaching and Learning at the University of Alberta. Nancy Chick is the director of the Endeavor Foundation Center for Faculty Development at Rollins College in Winter Park, Florida. Nancy had also served as a Professor of English within the University of Wisconsin System, where she codirected the Teaching Fellows and Scholars Program for all of the system’s 26 campuses. Janice and Nancy have both published extensively on the scholarship of teaching and learning and have each co-authored influential books on SoTL methodologies and signature pedagogies.

Show Notes

Transcript

John: A growing number of faculty members participate in the scholarship of teaching and learning, or SoTL. In this episode, we discuss a new open educational resource designed to assist faculty interested in pursuing SoTL research.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are: Janice Miller-Young and Nancy Chick. Janice is a Professor of Mechanical Engineering and a past Academic Director at the Centre for Teaching and Learning at the University of Alberta. Nancy Chick is the director of the Endeavor Foundation Center for Faculty Development at Rollins College in Winter Park, Florida. Nancy had also served as a Professor of English within the University of Wisconsin System, where she codirected the Teaching Fellows and Scholars Program for all of the system’s 26 campuses. Janice and Nancy have both published extensively on the scholarship of teaching and learning and have each co-authored influential books on SoTL methodologies and signature pedagogies. Welcome Janice and welcome back, Nancy.

Janice: Thank you.

Nancy: Welcome. Thank you.

John: Today’s teas are:… Janice, are you drinking any tea?

Janice: I am. It’s my favorite flavor by Stash. It’s called licorice spice.

John: Very nice.

Rebecca: I think that might be the debut on the podcast. I don’t think we’ve had that one yet.

John: I have had it, but I haven’t had it on the podcast.

Rebecca: I think so.

John: It’s very nice. And Nancy, are you drinking tea?

Nancy: I am drinking some iced tea. I’m in Florida, so there’s no way I’m going to be drinking hot tea. So some good southern sweet tea.

John: Very nice.

Rebecca: I just have some awake tea this afternoon, John.

John: …and I have an oolong tea today.

Rebecca: We have invited here today to discuss Becoming a SoTL Scholar, your most recent co-edited book on the scholarship of teaching and learning research, which was released this summer as an open educational resource in the open access book series in Elon University Center for Engaged learning. Before we discuss the book, though, could you tell us a little bit about how you became interested in SoTL research?

Janice: So I started my post-secondary career at a teaching-focused institution, Mount Royal University, in Calgary, Canada, and I was teaching engineering courses, because I’m an engineer, and some of them were very physics based. And it so happened that at the time I started, there was already a lot of disciplinary based educational research happening in both physics and engineering. So naturally, I started reading it as I was starting to teach, and I got started presenting about my teaching shortly after that. It was, as I said, a teaching-focused institution, so that was considered quite a normal thing to do. I didn’t do any formal inquiry into my teaching and learning until a few years later, when Mount Royal started supporting the SoTL Scholars Program.

Nancy: I got started in SoTL back in the 90s, when I was a graduate student. I was at the University of Georgia, shout out Bulldogs, and I was part of a graduate student organization of Teaching Award winners. There were, I think, 30 of us from across campus, none from the same departments, and we would gather, I think it was monthly, to talk about teaching and to read about teaching. And that program really got me through graduate school and whetted my appetite for what I wanted to do as a faculty member, what I wanted to focus on. Later, when I became a faculty member in Wisconsin, I found out that what we were talking about was SoTL, and so when I was on the tenure track in Wisconsin, I tried to take every faculty development program, every SoTL program, I tried to read everything, and I just knew that that’s the research and that’s the community that I wanted to be a part of. And I’ve never left. They can’t get rid of me.

John: That’s a wonderful experience, but I don’t think that’s typical of the past that most graduate students have. Most graduate programs focus on training students to do research and not to prepare for a career in teaching, even though most faculty end up at institutions where teaching is a primary responsibility. Why don’t graduate programs provide more preparation for research related to the scholarship of teaching and learning?

Janice: That’s a tough question. [LAUGHTER] I could make a stab at it. I don’t think that’s true of all programs. I know that the program we have at the University of Alberta does touch on the scholarship of teaching and learning a little bit towards the end of the program that we have for training graduate students related to teaching and learning, I think it’s a lot to ask of a graduate student to learn all of those different things. They’re learning to be a researcher in their discipline. Hopefully they’re learning about teaching and learning a little bit, at least, through either having some teaching assistant experiences or a formal program that a university might have. I think if they’re at least exposed to the idea of scholarship of teaching and learning, that would be a great start. And I know that some are, but again, I think it’s a lot to expect of a graduate student, to learn all those different things. I don’t know. What do you think, Nancy?

Nancy: Yeah, I think historically, certainly, the focus of graduate education was on doing your disciplinary research, like you said, partially, if they’re lucky, learning more about teaching and learning, but thinking of SoTL as an area of research, as an actual field… that I think is more emerging, and so I’m not sure that it would occur to graduate programs that this is an area of research that people could do, maybe that people could read, but not necessarily that they could do. However, I will say, in our book, we’re seeing that that may be shifting gradually and in very small ways. But we have one author, for example, who started out her career, actually, as an undergraduate student, knowing that this is what she wanted to do. She got a Master’s, she just defended her dissertation, and is now in a position, I think, at the University of Rhode Island. So gradually, I think that might be opening up, especially now that we have at least one example in the US, and I suspect, more and more in other countries.

Rebecca: Can you talk a little bit about how your book came to be and maybe how you selected some of the contributors to the project?

Janice: I guess, selfishly, it was the topic that I was very interested in. I would call myself a mid-career SoTL scholar, I suppose. I have enough experience now that I feel like I can be the senior person on a project and mentor others into the Scholarship of Teaching and Learning. It took me a long time to get to that place, I have to say. But now that I’m here, I was interested in literature about the next stage of a SoTL career, specifically, and I didn’t really see very much in the literature about that. And so I wanted to pull together the literature that I already knew existed about the identity of SoTL scholars, and also contribute to that topic by putting out a call for others to contribute to the book.

Nancy: And I have the good fortune of being in a position where wonderful people occasionally call on me to come along for the ride. And Janice and I have worked together some in the past, and she just asked if I wanted to come play with her on this project. So I was happy to do so, but it really was Janice’s idea, a real sense of curiosity of what this would look like in different careers and career stages. So really, hat tip to Janice for the project. Janice, how did you pick the authors?

Janice: Well, we put out a call through several listservs and through our networks for people to make proposals for the book. And I think we accepted everybody that we thought fit within the vision of the book, and that was very broad. We did get some applications that we didn’t accept, and not because they weren’t good applications, it’s just they weren’t really about becoming a SoTL scholar in any way. They were maybe about supporting SoTL scholars, for example, and so anything that fit within that broad vision, we accepted and then put together the sections of the book as a result of what we received.

Nancy: And I think an interesting part that ended up playing out really well was how the proposals that really matched this vision did align with different phases of a career. We could have gotten all kind of mid- career folks, or all early folks, or all very experienced folks, and we got people all along the spectrum. I know we were hoping for more contributions from other countries, from around the world, that we kind of acknowledge in the introduction as a bit of a gap, but it’s what we had to work with. So perhaps someone else will follow up and add a more international perspective.

Rebecca: Yeah, that would be nice to see.

John: So in that introductory chapter that the two of you authored, you described the growth of SoTL work over a little more than 30 years or so since Boyer suggested the need for this type of research, and you address some of the controversies associated with that, and one of the issues that came up is the diversity of approaches that are being used in SoTL. Could you talk a little bit about the diversity, and also to what extent different disciplines have standardized on a particular methodology, and to what extent is there more diversity in some academic disciplines.

Nancy: I’d say one of the conversations, sometimes debates and controversies, in SoTL, from the very beginning, has been the nature of disciplines within the multidisciplinary field of SoTL. We don’t often work with people in other disciplines in our regular work. So when So TL brings together people from the arts, from the sciences, from the social sciences, from the humanities, from the professions, students, inevitably, there’s going to be some disagreement about what this thing is supposed to look like. Sometimes that disagreement resolves in the sense of, “Ah, it’s a diverse field,” and sometimes it ends up with some voices saying, “No, no, good SoTL, good research, looks like x, y and z, and inevitably, that kind of list excludes some of the many approaches that are in the field. Just one example, on a recent thread on a very popular listserv here in the US, there’s a conversation about the gold standard for SoTL being multiple institutions, because that’s the only way you can create generalizable findings. Now that’s a very disciplinary approach that doesn’t apply across all disciplines, and so to say that that’s a gold standard for So TL immediately knocks some of us out of the conversation. So I think there’s a constant effort to remind that in this field, there are diverse approaches. There are many approaches. Even within some of those singular approaches, there may not be consensus on what it’s supposed to look like. And importantly, in So TL, that’s not a bad thing, that diversity means we’re asking a bunch of different questions, drawing on a bunch of different kinds of evidence, artifacts, data, using many approaches to generate knowledge, but it’s a constant tending to that diversity.

Janice: Yeah, sometimes I get frustrated that we’re still having this conversation. I’m sure Nancy would agree with that. If this is a practice that is done by faculty and instructors from across higher education, then SoTL, by definition, has to be inclusive of their different ways of doing research or scholarship, whichever word you choose to use. There is a recent paper in teaching and learning inquiry by Jennifer Löfgreen, and she makes the argument that we really need to start thinking in terms of paradigms of approaches to inquiry, and move away from talking about disciplinary approaches, because for fields that are so diverse, paradigms are one way that we can communicate to each other what are our basic assumptions about the thing that we are researching, and I really like that paper.

Rebecca: So we start digging into your book a little bit. The sequence of chapters in the book is organized by career trajectory. Can you talk a little bit about each of these sections?

Janice: Well, as Nancy said earlier, we organized the book according to the trajectory of careers. So our first section is really meant to speak to both students and early career faculty who are interested in starting to pursue a SoTL career. Our next section is directed more towards people who have experience in academia, maybe have been teaching for quite a while, but they’re wanting to shift their career towards a SoTL agenda. And our third section is about starting to maintain that engagement once one has started in the field, so ways to continue or even enhance participation or one’s own scholarship. And then our final section is just a couple of chapters, and it’s exploring how we’re never really quite there. We’re always still becoming a SoTL scholar. We’re still in the process of exploring our identity and changing our identity as we do this work and learning new things.

Nancy: Well, it dovetails with a project that I was part of a little later in the book project, this was led by Sarah Bonnell, who’s now at Elon, but she led a project of a group of us thinking about SoTL generations. And in the resulting article, we break it down into… oh gosh, I forget, three or four generations, but starting with the people early on who were kind of well- established faculty members, and then turned to SoTL. But over time, people have entered SoTL earlier and earlier in their careers, and so now we’re at a stage where there are multiple generations, and where, even in this moment, people across those generations are coming to SoTL for the first time. So that idea of the trajectory of the career phase and the particular moment when we enter, I think, will always be relevant, because for many it’s not where we start. And so that idea of coming early, coming mid- course, coming later, I think, will always be relevant, which is one of the strengths, I think, of the structure of the book.

John: In addition to the sequential structure over the course of a career, you also provide alternative organization schemas to allow readers to choose different ways of navigating the book. Could you briefly describe these alternative classification schemes?

Janice: Those came about through… it was sort of my idea, but also prompted by Jesse Moore, who suggested that we could kind of make this book into a choose your own adventure, in a way, because it does speak to people across different stages of a career trajectory. And so, I guess that got me thinking in terms of the diversity of types of scholarship of teaching and learning that exist, and how that can also be confusing for folks who are new to the field. And so I wanted a way to communicate that. And I think because of my own sort of positivistic disciplinary background, I like to categorize things, so I thought that might be useful for others as well. And so we came up with three broad categories in which we label each of the chapters. So we label them by what they’re about. Some are specifically a SoTL inquiry, but some are more definitions of the field and that sort of thing, so that was one of the categories. We labeled them according to the context that they are about. Some are written about universities, and some are about polytechnics, for example, and some are about scholars from different disciplines. So that’s the second category. And the third category, which I like the most, actually, is different genres of SoTL. And again, the first genre would be a report on a research project, but that was one of six different categories that we came up with, actually. We had conceptual articles, scholarly essays, reflective essays, narrative essays, and one graphical essay…

Nancy: ….which was one of your chapters.

Janice: Yes.

Nancy: …very visual, kind of getting us a glimpse into how you think about SoTL, but represented visually, a real innovative chapter. Recently, someone reached out to me because he’s going to be leading a faculty book group using Becoming a SoTL Scholar, and acknowledged that maybe some of the faculty didn’t have time to read the whole book, and so was just looking for a little guidance on what we might say to a faculty member who wants to talk about the book but just really only has time to read a handful or so of chapters. What might that guidance look like? And so I just suggested, and this is just one way to think about it, I suggested maybe pick a couple of chapters that resonate with you because they’re from your disciplinary identity, or your career stage, or a familiar genre, and then pick a couple that seem very unfamiliar for the same reasons, and that kind of takes us back to that earlier point about the diversity of the field. The sooner we can get folks to engage with that diversity and recognize it and see across it, I think the stronger the field will come. So I’m hoping that that book group follows that idea and circles back to see how it went.

Janice: That’s a great suggestion. I like that.

Rebecca: One of the things that often comes up in conversations about the scholarship of teaching and learning is that it’s not necessarily always valued across all institutions or even within all departments inside of the same institution. So what advice would you give to faculty who are interested in starting a SoTL career, but trying to navigate these differences within their own institution, or as they’re maybe thinking about pursuing a career and finding an institution to land at?

Janice: I think I would have two pieces of advice. And one would be, start small, try a small project to begin with, get your feet wet. There’s always something you didn’t expect when you’re collecting and analyzing data. I know that’s one particular approach to SoTL inquiry. I sound very researchy when I say that, and I don’t want to exclude other disciplines where maybe collecting data isn’t really quite the right term, but anyhow, I think start small. Do a small project, you’d have to start reading the literature as a result of doing a project, because if you’re going to disseminate, you want to place your findings within an existing scholarly conversation. My second piece of advice would be to also think about kinds of research questions that would have an impact beyond your own practice. I think if you want to do this kind of work as a scholarly career, then investigating questions within your own practice is certainly important for improving your own teaching. But there are other questions that might be beyond your own practice that could result in a greater impact on your colleagues, for example, or even on the field itself. And so thinking in terms of where you want to make an impact and how is also important.

Nancy: And I think that piece of advice also connects with one way to think about the value of doing SoTL and the safety of doing SoTL, depending on where you are in your career. If there are particular initiatives or issues that your institution is wrestling with or promoting, prioritizing, that might be one way to get a little more support on your campus if you’re connecting with something that’s important to your institution. The piece of advice that I would give, in addition to Janice’s points, is to find someone to connect with beyond your institution. I got involved with this work originally because of the people I was interacting with, and how that enriched me and made me a better everything. And I think for those who want to get involved with SoTL and are at institutions that may not value it, or may not know enough about it, may not know how to count it towards tenure and promotion, connecting with someone who’s been doing this work for a while can help enormously. Because, as you mentioned, this conversation has always been happening because it’s not disciplinary research. So we’re always trying to figure out how to help SoTL scholars convey what the work is, what its value is, why it should be counted, and there are so many of us out here who love to connect with newer scholars. I was going to say younger, but that’s not true, because it could be any part of the career. But there are lots of people out here who have been through it, or who have helped others go through it, connecting with the International Society for the Scholarship of Teaching and Learning. That’s the professional organization devoted to this kind of work. And members, board members, committee members, interest group members, lots of people within that organization can offer help.

John: Pretty much all academic disciplines have some people who are working in the scholarship of teaching and learning, and most national meetings or international meetings will have some sessions on that which might be a good place. And then, as you mentioned, all these interdisciplinary groups or conferences could also work quite nicely in forming those connections as well as listservs. So yeah, that could be helpful in providing this sort of support network, which might not always be available on the campus.

Nancy: And just to agree, SoTL is indeed a field. SoTL is indeed a practice. SoTL is also a community, or communities, so kind of keeping that in mind and tapping into the one or ones that are most accessible to you or interesting or you already have a connection, definitely.

Rebecca: So we talked a little bit about where we can make connections if we’re interested in doing SoTL, especially beyond our campus, in terms of conferences, etc. And we talked a little bit about navigating some of those things and maybe building a little bit of administrative support for some SoTL research, but maybe we can push on that just a little bit further. You talked about picking up on some topics that are already being discussed on campus as potential topics to do some sort of research on. Are there other strategies that we might be able to encourage our colleagues to use to build administrative support for SoTL research on our campuses, if it’s not something that there’s a culture of already on our campus,

Nancy: One thing that comes to mind for me is a 2013 article by Dan Bernstein in the inaugural issue of Teaching and Learning Inquiry. I forget the full name, but I think it’s something like SoTL active faculty as institutional assets. And his whole argument in that article is that many faculty on their campuses are focused on the campus. They’re teaching what he calls local topics, whereas SoTL-active faculty are connected to their classes, their campuses, but as Janice alluded to earlier, t hey’re also connected to the literature on teaching and learning and to broader communities, and so they can act as conduits or liaisons between the evidence-based research, between what’s happening at other institutions and their own campuses. So that kind of asset can be helpful, but that’s just, I think, one kind of example.

John: In many disciplines, for centuries, faculty were teaching in pretty much the same way, without really much regard to what we’ve learned about teaching and learning. But during the pandemic, those things stopped being feasible, and faculty were suddenly exposed to evidence-based teaching practices that came out of SoTL research. I know there’s been some backsliding since we’ve moved out of the pandemic, into moving back to some of those old practices, but do you think faculty are more willing to take into account SoTL research, and perhaps, might that lead to more recognition of the value of that research on campuses?

Janice: That’s an Interesting question. I know in my discipline, faculty tend to listen to or be influenced by educational research that’s happening in their own context, like in their own discipline. So certainly there is an awareness of that research, and, like I said, a willingness to listen to it, if it is from a similar context. And I guess that’s another important reason why we need the diversity of the scholarship of teaching and learning. I don’t know if you could say that about all disciplines, but certainly in engineering, that’s the case.

Nancy: And I think after the pandemic, and of course, we always have to qualify after… [LAUGHTER] When will we be able to stop having to qualify after the pandemic? In the wake of the pandemic, I think faculty everywhere have been shaken by what they learned about students, what they learned about teaching, what they learned about technology, everything, and I think that sense of everything familiar is no longer familiar, and students, especially the generation or generations coming out of the pandemic, are so dramatically different in important ways, that I think many faculty are grasping for help for: “What do I do? How do I teach these students that are so different from the students that I had before the pandemic?” Now, those faculty will reach to different places, their teaching and learning centers, their colleagues, nowhere. Some will reach out to the SoTL literature, or colleagues who know SoTL, because they’ll make that connection to how much they value research-based knowledge. It’s always going to be a challenge, but I do think right now, there’s so much grasping for “what do I do? How do I do this better? How do I do this?” …that it’s a real, I don’t want to say opportunity, but it’s a real moment where I think SoTL is having a more recognized value among faculty who otherwise would never consider even looking to any of those kinds of publications.

Rebecca: Feels like a really good tee up to the question we always wrap up with, which is: “What’s next?”

Janice: Well, one thing is, I’m interested in understanding the Canadian context a little bit, so I’ve been proposing, with a couple of colleagues, a project looking at the different institutions and the kinds of support they have for SoTL across Canada. You said, for example, that most graduate programs don’t have SoTL built into them. I don’t know if that’s entirely true anymore or not. I’d like to get some evidence of that. I’m interested in the transformation that occurs when someone starts to do and continues to do the scholarship of teaching and learning. I’m interested in what is the process of learning, what is effectively another discipline, or learning to do research in a new field. As I said, I think there’s been a lot of literature about getting started in SoTL, and not as much on the more experienced faculty members. So I’m hoping to do some research in that area as well.

Rebecca: Sounds exciting.

Nancy: And what’s next for me? Yes, I have to go shortly have dinner with our President. But one thing I’ve been talking about a lot, and then I’m deeply interested in, and that I just want to put out there so that if anyone else is interested, they connect with me. I’m very interested in how, again, in the wake of the pandemic, the struggle, the difficulty, the challenges of learning that are necessary to learning may have become pathologized in some ways that make it even harder for faculty to teach difficulty, to encourage and support students to stick with the struggles, the challenges of learning and doing hard things in classes. I’m hearing a lot of faculty talk about this phenomenon, and I think we need to figure out what to do about it, because we know that that kind of struggle, that kind of this doesn’t feel good, or this is frustrating, is a necessary part of learning that we don’t want written out or smoothed out. So that’s where I’m going next.

Janice: Interesting.

John: Those are things we hear about all the time from faculty here, too, and I just came from a class where I had that sort of conversation with students about learning is work, and there are shortcuts you can use, but they’re not going to be very helpful for you once you get past the end of the term. And it was a productive conversation, but we’ll see what sort of impact it has. But that’s a good topic for some SoTL research. Thank you very much for joining us, and thank you for providing this wonderful resource, and we’ll include links to this and anything you mentioned in the show notes as well.

Nancy: Wonderful. Thank you for having us.

Janice: Thank you.

Nancy: It’s good to see you.

Janice: It’s good to see you, too.

Rebecca: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

350. Nudging, not Judging

Posted in classroom community, engagement, evidence-based practices, higher education, learning, pedagogy, professional development, teaching
July 17, 2024

During the pandemic, faculty participation in professional development activities expanded dramatically. Faculty involvement, though, has been gradually returning to pre-pandemic levels. In this episode, Sarah Rose Cavanagh joins us to discuss strategies for bringing more faculty into discussions of teaching and learning.

Sarah is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and also is a regular contributor to the Chronicle and many other publications. Sarah often serves as a keynote speaker and we were very fortunate to have Sarah join us for a keynote address at our Academic Affairs Retreat in Oswego last August.

Show Notes

Transcript

John: During the pandemic, faculty participation in professional development activities expanded dramatically. Faculty involvement, though, has been gradually returning to pre-pandemic levels. In this episode, we explore strategies for bringing more faculty into discussions of teaching and learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Sarah Rose Cavanagh. Sarah is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and also is a regular contributor to the Chronicle and many other publications. Sarah often serves as a keynote speaker and we were very fortunate to have Sarah join us for a keynote address at our Academic Affairs Retreat. Welcome back, Sarah.

Sarah: Thanks for having me.

Rebecca: Today’s teas are:… Sarah, are you drinking tea,

Sarah: I thought about trying to blow your mind and saying I’m drinking some Bergamot green thing. But no, I just finished my last coffee for the day. And now I’m on to water. [LAUGHTER]

Rebecca: You’re so consistent.

John: My tea today is a black raspberry green tea.

Rebecca: And I have an Awake tea, which clearly I need. [LAUGHTER]

John: We’ve invited you here today to discuss your May 7, 2024 article in The Chronicle, entitled, “After a Tough Year for Classroom Innovation, It’s Time for a Reset.” And I think this is something that’s going to resonate with a lot of educational developers and many faculty. You begin the article by noting that there was a sudden increase in attention to student-centered teaching in response to the pandemic, but has that shift been as persistent as many of us might have hoped?

Sarah: Oh, I think yes and no. I think that a lot of people started focusing more on their teaching. In those pandemic years, we all saw in our teaching and learning centers, higher attendance at events and a lot of engagement, both in these larger events, and then faculty coming to us for consultations. And one hopes, as educators, that learning doesn’t just go poof, that the effect of all of that attendance and thinking and consulting is still impacting people in their classrooms. But I think that there definitely has been a trend on my own campus, and most campuses that I talk to, about declines in attendance at those same sorts of workshops and talks and consultations. And there also, I think, has been a bit of a pendulum shift in some of the topics such as flexible deadlines, where I think we saw a lot of people move in one direction, and then are starting to see a little bit of a shift back. And so hopefully, we still have overall nudged the average over towards student-centered teaching and that hasn’t changed. That’s my hope. But I definitely see a little less intensity and interest lately.

Rebecca: We definitely have seen similar declines in professional development on our campus as well. And sentiments like, “I’m just really tired,” or “I’m done” [LAUGHTER] which you highlighted in your article, and one of our most popular workshops of recent years has been related to burnout. Can you talk a little bit about ways that we can re-engage our faculty or why burnout is such a popular topic? [LAUGHTER]

Sarah: Right? Well, I think that if we’re thinking about kind of a shift away from student- centered teaching, or even a backlash, as we’ll talk about in a little bit, that burnout is one of the many causes of that. I think that faculty are just tired, as you said, and that sentiment of “I’m just done,” like, I’ve heard those words multiple times from multiple different people. And a lot of student-centered teaching, and certainly attending professional development workshops takes time, and there’s just no time. And so there’s the burnout that’s related to the emotions, and the trauma of the pandemic, and just workload being completely ridiculous. But then there’s just also a paucity of time. And I think that part of that is this whole Zoom… we’re here, not on Zoom, but we’re here virtually, and those virtual options open up so many possibilities. But I feel like we took all of our pandemic solutions with the Zoom sessions, and all of that, and we brought them into the future. But then we brought back all the in-person stuff. [LAUGHTER] And so there’s zero time for anyone. And so I think that the burnout is so strong, and that’s why there’s such attention to that. And that is one of the most popular topics right now.

Rebecca: Feeling over scheduled definitely limits the ability to feel creative, or to dream or to think about new things. So if people are used to kind of being in that state, or historically have been in that state, and they’re just feeling like they can’t get to that state. I can imagine that’s why people say they’re done is because it’s hard to come up with new ideas or to put the energy into that kind of thinking.

Sarah: Absolutely. And I think also students are equally, at least equally, burnt out. [LAUGHTER] And so I think another thing that’s happening is faculty put all this time into student-centered teaching, but then it’s not working. They’re not seeing more engagement or better learning. And so you’re investing all this time, but then the students are so burnt out that it doesn’t look like it’s being effective, and that’s frustrating, and I think that that’s also part of the feeling done, like, if you’re just throwing everything you have at bettering your teaching, but then still feeling frustrated by the outcome, then that is demoralizing.

Rebecca: It’s interesting that the way you put it instead of like getting re energized… [LAUGHTER] like, depleted.

John: And we never observe the counterfactual, we don’t see what would have happened, had we not been focusing more on student-centered teaching after some really tough times. So, why do you think faculty are pushing back a bit more against student-centered teaching approaches, because we had a lot of buy-in on that during the early stages of the pandemic, and for a couple of years after that, but we’re seeing a little bit more pushback.

Sarah: Yeah, well, I have a long answer for this one, because I am a psychologist and so I think everything is highly multi-causal. And one major factor is the level of burnout, both on the faculty side and the student side. And so we don’t have to go back to that, but I just want to highlight that I think that that’s part of it, is this burnout, and I love Rebecca’s depiction of like how impossible it is to feel creative and do new things too when you’re feeling exhausted. And so I think that’s a big part of it. On my endless commute yesterday, [LAUGHTER] there was a Red Sox game (I work in the Fenway), I listened to your interview with Josh Eyler, which was excellent, of course, and I think it probably predated the article, but you were talking about just this topic of sort of a backlash, specifically against alternative grading schemes as a student-centered practice. And Josh said, that I thought was very insightful, as everything Josh says is, that he had this analogy of a rubber band, and that there’s just this effect. And I think of it as a pendulum as well, which I already referenced, that there’s just this natural effect when you move in one direction to kind of snap back a little bit. And he saw that as part of the cause of this. And I think that that is true. I think it’s sad, but we love pendulums as human beings. I’m always like let’s just settle into moderation, [LAUGHTER] but we love swinging from one extreme to the other. So I think part of it might just be a regression to the mean sort of effect. But I think there’s a lot of other factors. I think that one is, if you drill down into the recommendations of so many different theoretical approaches to good teaching, you come up with similar recommendations, whether you’re focusing on inclusive teaching or accessible, teaching, engaging teaching, all these things, they all drill down to these recommendations of being warm and welcoming, having a promising tone, seeing students as people, giving them autonomy and giving them choice, all these things. And I think that some faculty feel, after attending several years of the pandemic and all these different intros and frames, but then the same recommendations feel like they got it. [LAUGHTER] And that might be part of it. The final two causes of the backlash are a little more delicate. I think one is the student reactions. And we talked about this a little bit, that students themselves are so burnt out and so faculty aren’t seeing great results when they experiment with student-centered teaching. But I think that there’s also a rise in student incivilities. Beth McMurtrie had a great report in The Chronicle, about a year back on the rise of student incivilities. Another term that people use is reactance, when they’re pushing back against the instructor. I can tell you a brief anecdote. I was on a different campus talking to a faculty member. And she had been using alternative grading practices, it was kind of a standards-based approach where students had multiple attempts at achieving a standard, but they’re passing an assignment and would either be competent or not yet competent. And she was teaching in a male -dominated field. And this student of hers who was male, he got a “not yet competent,” and threw the paper back in her face and said, “You don’t know what you’re doing.” And so I think that that too, when you’re putting all this time and effort into student-centered practices, and then having reactance and incivilities in reaction, that’s also demoralizing. And then finally, just to get myself in trouble, [LAUGHTER] I think that we in the teaching and learning fields, bears some responsibility. I think that especially toward the end of the heyday of teaching and learning years on Twitter, things got a little bit ugly. There was some shaming of faculty pile-ons, and some of the pile-ons were of people engaging in student shaming practices and other things that are ugly on their own. But some of it was also just venting stresses of teaching, and then getting piled on. And I saw this really clearly because I was active on Twitter in the teaching and learning fields. And then because I was more of a research focused faculty for most of my career and all my social contacts in person are more like that, I’ve had all my faculty friends on Facebook, and they would post just some things. They all very care deeply about their teaching and their students, but they would post things and I was like, “Oh my God, if you had posted that on Twitter, [LAUGHTER] you would be dead in the water.” So I think that we, as we talk to each other, and we convince each other, and we get more and more excited about change, and about some innovative practices, that sometimes we get a little out of touch with the majority of faculty who are teaching on the ground, and their constraints and practices.

Rebecca: We spent a lot of time talking about students as humans, but teachers are also human, I think, [LAUGHTER] and I think we need to be, and you’ve mentioned this before, too, is just making sure that we’re treating everybody as humans, and that everybody has constraints, and everybody has different pressures. And in your article, you suggest that the time seems ripe for a complementary revolution, one that recognizes and respects the autonomy and bandwidth of teachers. Can you talk about what this counter revolution might look like?

Sarah: Sure. And this is easy for me to say, because in my position I have no control over budgets, [LAUGHTER] or faculty workloads. But I think that primarily what we need is we need a revolution in how we treat faculty. Faculty have been through the pandemic, I think the coming demographic cliff and all of these things that administrators push for faculty to be more and more responsive to students, more and more student-centered, to take on more and more things. At my last institution, they had us writing letters, personal letters to individual possible students to try to encourage them to put down deposits. All of this emotional labor time, regular labor. And so I think faculty in response need more time and money. And I think we’ve just piled on and increased the workload and increased the workload and expanded the scope of the work exponentially through these pandemic and post-pandemic years. And I think that faculty need course releases, I think they need stipends for some of this work. And I think that we need to put a lot more resources toward our faculty. I think it also means a lot more full-time lines with security as well. And those are going in the opposite direction. I think that the other thing that we need is hands off our classrooms. [LAUGHTER] I think that we need to respect faculty autonomy and faculty choice. And so many of us get into this work, because we want that autonomy, we want that choice. We’re willing to work harder for a lot less money than if we went elsewhere. But it’s because we want to be self determined, and do the things that we want to do. And I think that we need to respect that.

John: One of the things you’re suggesting in the article is that there’s no single approach to student-centered teaching that works for all faculty or for all students. Could you give some examples of approaches that work for some faculty in their context that might not work as well for other faculty, given their context?

Sarah: Absolutely. There’s so many… class size is one, you know, are you teaching 25 students? Are you teaching 750 students? …very different strategies are going to be effective there. Courseload… are you teaching two courses a semester, or you’re teaching five courses a semester, that is going to affect the choices that you can make. Precarity of position… both pre-tenure and post-tenure, but also: Are you full time? Are you contingent? Are you on a semester to semester? Contract… identity, these characteristics of the instructor. We had Chavella Pittman come out and talk to our faculty at Simmons University last semester. She was fabulous. She talked a lot about student incivilities, and how many more faculty of color and other minoritized faculty receive of those student incivilities and what majority faculty can do differently in terms of their policies to help support their minoritized faculty. Your teaching philosophy, I think, is important. And again, autonomy is important. And a little bit your teaching talents. For instance, I’m not funny. [LAUGHTER] Some people are really funny. And that is like being humorous in front of the classroom is a really engaging strategy, but it’s not open to all of us. And some people are gorgeous lectures and actually, lecture might work better in their class than someone who’s not. And there’s just so many contexts, there’s so many constraints that are going to influence this vast buffet of teaching choices.

Rebecca: Sometimes when we’re thinking about offering students choice, that also means there’s an impact on faculty too. And so you’re kind of underscoring a faculty choice and some of the individual faculty needs where they might be balancing other things may have certain kinds of time constraints, physical limitations, there’s all kinds of things that could be at stake. How do we help folks understand the need to balance student needs with also faculty needs, and to make those kinds of decisions?

Sarah: It’s so tricky. I often think of all of teaching as kind of a dance, and there’s an art to it. And there’s a little bit of an intuition to it. I’m mad at myself, because I meant to look this up before this talk, but I will send you a link to something later on. Brian Dewsbury and some other folks put together an inclusive teaching massive online course through HHMI. And I lead faculty through a learning community using it. And they were discussing, and I’ll send you the link and the person’s name after. But it was an approach to teaching that, instead of being learner centered, it’s learning centered. And this balances out this competing tension between faculty needs and student needs, but also between, and I think we’re going to talk a little bit about this in a bit, students to student needs, you know, variations in the needs of students. And instead of centering the learner, centering the learning, and thinking about what is going to be most effective for the goal of this course, which is learning, and what can I do in terms of my teaching choices, and the structure of the course and the types of assessments and things like that, that centers the learning, which is going to center the learner as well, because it’s their learning that will happen. But I liked that subtle shift in how we think about this. And I think that might be one approach to approaching this dance with balance.

John: And that brings me back to when I first started working at the teaching center. The year before that an advisory committee put together a draft statement of the purpose of the teaching center, and they went with learning centered. We’ve still tried to maintain a focus on learning, because that’s kind of what we’re here for. And that’s why students are here. So I think that focus is really good.

Sarah: Oh good. [LAUGHTER]

John: But going back to that issue about different techniques working for different faculty, isn’t the same also true for students? That the flexibility that many faculty provided to a greater extent over the years immediately following the pandemic might work really well for many students, but might that provide a loss of structure which is really important for many of our students?

Sarah: Absolutely. A question I often get when I give talks about emotions in the classroom is about my recommendations for a lot of warm-up activities and small-group discussions, and getting students interacting with each other to maximize the sense of belongingness and community, which we know is important for learning and especially for marginalized students. But I often get the question of what about socially anxious students? What about introverted students? What about neurodivergent students? A lot of these students might really struggle having so many social engagements during a class period. And I don’t know that my answer is very helpful, [LAUGHTER] because I always try in my own teaching to have varied methods. So it’s not always social, that there are some classes that are much more independent, there are options to write in small books and things like that. And in the piece, I referenced Sarah Silverman’s work on access friction, and she didn’t coin the term she gives credit to Areley McNeney, and others, but about this friction that arises when one student’s accessibility needs are different, or even opposing in tension with another student’s. I think variation is good, you know, varying things around and flexibility and talking to your students, using who’s in class surveys to find out about student needs, always respecting students’ accommodations, of course, but having that conversation with students and drawing them in. And one thing that I really tried to do in my own teaching, is to be transparent about why I’m doing a certain thing. So I often start my classes with an explanation for why there’s going to be so many community building sorts of activities and how it is the research suggests that that’s important for their learning. So we think that kind of transparency, varying strategies, not doing the same thing all of the time, and then just acknowledging that certain teaching strategies are going to be more effective for some students versus others. And so let’s not just blast one teaching strategy the entire semester.

Rebecca: I think one thing that often happens is the use of the term universal design, not necessarily universal design for learning but this idea of universal design implies that one size can fit all, but usually universal design, if we’re really following those principles is that there’s variability and is that the space or the technique or the situation can adapt to the person and vice versa, rather than necessarily having one solution that works for everyone. I mean, it’s part of it, but there’s usually this flexibility piece that’s key, but flexible doesn’t have to mean so much extra work for a faculty either. There’s different ways to have flexibility.

Sarah: Yeah, multiple points of entry for students always is great.

John: When I first started teaching, I was told by the department chair at the time, who’s long since retired, don’t waste too much time on teaching, spend your time on research and other things that are essentially the currency of higher ed. We’ve moved quite a ways from the time where many faculty are getting that sort of advice, and those of us in educational development often become really passionate about encouraging student-centered teaching approaches, and we generally provide some criticisms of high-stakes exams, the sole use of lecture, multiple choice tests, and so forth. Those were pretty much considered to be standard practice not too long ago, and are still very common in many disciplines at least. You mentioned Twitter, many people would consider some of those techniques to be sort of like malpractice, where they’re actively doing harm to students and learning. Have we perhaps gone a little bit too far in pushing for changes in instructional practice, in the direction of a one-size-fits-all approach that perhaps we should back away from somewhat to bring more people into the conversation?

Sarah: Sure. So multiple layers there. I think one thing is, this is one of the places where I see the conversation on social media among teaching and learning folks as being a little disconnected from reality, if I can say that, [LAUGHTER] because I think that there’s so much of this still going on. I think that we talk to each other and then we talk to these wonderful faculty who come out to our professional development events. And that’s just like the tip of an iceberg. And we don’t tend to talk to people who don’t come to our events, [LAUGHTER] or don’t engage with us on social media. Where I see in this Facebook versus Twitter differential is I see my friends who are in much more research-focused careers, and all of their demands and rewards are all based on their research productivity. So you say we’ve moved away from that, but I think we’ve moved away from that some, and we have higher teaching demands even on research faculty, but I don’t think that the focus on research for a lot of those folks has changed very much. And when you’re burnt out, and all of the values and reward systems are rewarding you for research and not teaching, and you have TAs doing a lot of the on the ground face-to-face stuff with the students. I think there’s a lot of these so-called traditional practices still going on. And so I think it’s happening, and I think using the word harm might be a whole other big conversation. I would like us to step away from that word when talking about individual faculty and the teaching choices that they’re making in their classes. I think that there’s a host of problems with that word. And I think that part of the backlash, too, is this feeling of feeling judged by teaching and learning center, folks, and I think that that is not going to get us toward the sort of change that we want if there’s kind of these scores of faculty who are not part of the conversation yet. They’re not going to step into the conversation, if they’re feeling like we feel that their teaching practices, that they feel like they’re doing the best they can, are poor or actively doing harm to their students.

Rebecca: And the reality is they’re probably not actually harming students. Students are still learning and they’re having some other experiences and they’re facing challenge. That’s not, as you noted in your article, that’s not harm. Can you talk a little bit about fostering challenge for students?

Sarah: I don’t like to equate high-stakes exams and all these things… this is challenge and then student-centered teaching is low challenge, because I don’t think… and I’m not saying that you’re doing that… but I just want to be clear on my position for that. I think having really high expectations of students is challenge. I think asking students to take emotional risks and be a little vulnerable is challenge. And I think that all of that is done most effectively in a classroom where community has been built where students feel that they belong. I think reducing stakes is a really powerful tool to developing that feeling of safety in terms of willingness to take risks. So I am pro low-stakes. [LAUGHTER] And I’m not arguing that we should all be going back to sage on the stage and three exams, certainly not. And I have worked really hard over the last 10 years to try to nudge us away from those practices. But, I do think that they are not necessarily causing harm. And I think most of us, in our intellectual journeys, thrived under those conditions. I do think that people who have thrived under those conditions, usually people who’ve had a lot of privilege and a lot of resources. And so that’s another reason why we need to move away from these practices. But I think if we want to pull people with us and reduce high-stakes exams and reduce some of these practices that may not have been as effective at bringing all students to the table, then we need to do so with an open hand and also have ways that faculty can make incremental change. I reference in the article, Michael Palmer had a POD presentation at one of the teaching and learning conferences, in which he presented 15.9 million choices [LAUGHTER] that a faculty member makes in the construction of an individual class and argued that if we really want to make lasting change, we should be nudging little changes in all of those choices towards student centeredness, toward more inclusivity, rather than asking faculty to kind of throw everything out and start fresh and do something entirely new, and I found that compelling. And so if we could do that nudging, in the absence of judging, and by suggesting that there might be better effectiveness and equity, and doing that nudging, rather than saying you’re actively harming students with your current practices, I think would be a lot more compelling for a lot of those faculty who aren’t necessarily coming to our workshops.

John: So, nudging not judging, that’s a nice way of summarizing that, I think.

Rebecca: Yeah.

Sarah: That is a much more catchy way of saying it. [LAUGHTER]

John: I was thinking it would be a good title…

Rebecca: Yeah.

John: …or the podcast.

Rebecca: I was just thinking about, you never know what small little change you might make, that actually makes a humongous difference for a learner.

John: To use the technical term

Rebecca: I only use [LAUGHTER] technical terms. [LAUGHTER] So I think what you’re highlighting is really important, because these small little changes can make a big difference for an individual learner, or a set of individual learners. And you just don’t know what little tweak you’ve made, and what kind of impact it might have made. I’ve certainly had the opportunity to have a conversation with a student where they tell me about something that made a really big impact for them. And to me, it was like something that was really small, like, “Oh, alright, well, I should keep that.” [LAUGHTER] But it’s helpful. And they all add up over time.

John: What can we do to bring some of these other faculty who haven’t been as active in discussions of teaching into conversations, and to make it easier for us to do some of that nudging? We can’t really nudge people if they’re not talking to us or not coming into teaching centers?

Sarah: Well, I think more dedication of resources from the administration would be good. I think that bringing people to these events who haven’t traditionally come might happen if there’s more time and money, a course release or something, aa stipend for something else. I saw a presentation by Betsy Barre, a wonderful presentation where they had participation in a one-day teaching celebration event that was at something like 67% of the faculty attended. But it was very well resourced. They had a bunch of really fantastic speakers, a lot of great food, it was an entire day, and the provost canceled classes and like kind of shut down campus for the entire day and said, “This is important, come to this.” And they got something like 67% of their faculty at Wake Forest to attend that. So I think more resources is an answer. I think also having more varied offerings. So I tell the anecdote in the article when I was at Assumption University, one of our faculty came to us and he was from a department that never attended anything [LAUGHTER] that we offered and he said that, and he said, “Actually, none of us ever come because we think that you’re doing harm to teaching with all of these approaches that he felt were not effective, and that were kind of degrading education.” [LAUGHTER] And we had a conversation, and so what the result of that conversation was we had one of our faculty learning communities that we do over dinner with a book, and he gave a presentation called “the art of the lecture,” and it was a defense of lecturing, primarily lecturing as a mode of pedagogy. And then we also had a few other faculty do little flash talks about how they did kind of dynamic innovative lecturing. And then we had a conversation and a debate. So we weren’t advocating for this, but we were showcasing him, and then faculty were engaging in debate and dialogue around when is lecture appropriate? Like what are the ways to make it more student centered. And I would like to say that his whole department came out to that, and they didn’t, [LAUGHTER] but he was there. And I think demonstrating that kind of respect for all the variations of teaching, I think is good. Another that we did at my current institution is we had a panel of faculty and we intentionally chose faculty with varied deadline policies, so on the spectrum of more strict, more flexible, and kind of in the middle, and had them present their rationale and their procedures and then did q&a. And that was really well received and better attended than some of our other events. So I think mixing things up and creating a space where it is clear that we are respecting faculty autonomy, and that we see variation as good. I think it can be powerful.

Rebecca: I’m hearing some themes of celebrating teaching and learning [LAUGHTER] and respecting and listening.

Sarah: Yes.

Rebecca: …listening and respecting a wide range of teaching in all of the examples that you just shared, which I think are two things to remember and keep in mind, because I think you also said not in this last answer, but a couple answers back something about [LAUGHTER] I think you put the words faculty and equity in the same sentence rather than we usually think about students and equity. But I was hoping maybe you might elaborate a little bit more on this idea of some equity around faculty as well when we’re thinking about teaching in particular.

Sarah: I think that probably what you’re referencing in my previous answers was when I was talking about Chavella Pittman’s visit. And I think one thing that we don’t always acknowledge is the impact that our teaching choices can have on other faculty in our department and at our institutions’ teaching choices. And so she was talking about how many more incivilities and classroom disruptions faculty of color and other minoritized faculty get compared to white cis het faculty. And that when we make choices in our own classrooms, that might have an impact on other faculty. And so specifically, when white majority faculty have a lot of policies in their classes, that they have a lot of flexibility, or they don’t have a classroom disruption policy, or the classic example is “Call me Sarah, and don’t call me Dr. Cavanagh,” that sort of thing. That actually makes it more difficult for faculty who, in order to have a good classroom climate, needs to demonstrate their authority. I know we don’t like to use that word when we talk about teaching, but all our friends need to be called by “doctor” and need to have a classroom disruption policy, because otherwise there’ll be a lot of classroom disruptions. If there’s also this perceived differential in those two classes, then those faculty are going to get accusations from students that :”Oh, you’re so strict, you’re so this or that compared to these other faculty who are just like, relaxed, why can’t you be more like them?” And so I think that we do need to think… I’m saying “respect faculty autonomy,” but then also, “let’s talk to each other about our classroom policies in order to make sure that we’re not engaging in these behaviors that are going to impact other faculty.” But again, it’s a dance, it’s a dance and a balance, a little bit of this, and a little bit of that.

John: And I’ll mention that Chavella Pittman and Tom Tobin had a discussion of this in an article in The Chronicle, I believe, and then they joined us on a podcast for that too. And we’ll share links to both of those in the show notes.

Sarah: This has been a great conversation, but I want to be clear that I am so dedicated to student-centered teaching. And that is why I feel really passionate about this backlash because Rebecca, you mentioned listening, and I’ve just been listening and listening. And I’ve been hearing it all over the place. And again, I worry with these pendulums that we could just snap back to a place that we really don’t want to be. But I think sometimes when I have these conversations, I come out as “Oh, pro high-stakes exam and let’s have three exams, multiple choice, strict attendance, policies, all these things.” And that’s not the case, I just think that we need to do the nudging, not judging so that we can continue pursuing change, because I think we had this accelerant with the pandemic, and it was great, with so many people in zoom rooms and in seats, and really eager to like innovate and all of these things, and that has kind of faded, and we now have these other tensions that are pushing in the opposite direction. And so let’s have some new strategies so that we can continue pursuing educational reform

John: And the strategies of having different people with different perspectives discussing it, is probably a really good way of getting people to state their opinions, but also to hear some things that might nudge them in another direction. So someone who only lectures hearing a little bit about the possibility of doing an interactive lecture would be a relatively small adjustment that they wouldn’t have heard if they hadn’t attended an event of that sort.

Sarah: Absolutely.

John: I think that sort of approach is a really good one that we should try doing more of here.

Rebecca: I think also having varied classroom experiences is healthy for students, it’s a good experience for students and helping students to see the value and having just different opportunities, different ways of flexibility being demonstrated to them is helpful to point out to students rather than being really careful when you’re having a conversation with a student about the classes that they’re in, for example, and highlighting maybe some of the skill sets they might be getting from a particular experience rather than kind of bashing a faculty member or particular class and helping them understand they might want a variety of different kinds of classes to help their own workload and interest and that kind of thing. So we always wrap up by asking: what’s next?

Sarah: Well, I have an NSF-funded network in intro bio called TUnE-Bio, and we’ve been holding a lot of qualitative interviews of intro bio faculty and and digging into some of this, they’re thinking about change and the constraints and barriers and facilitators of change and I’m getting excited about kind of expanding that network into the possible future.

Rebecca: That sounds like some interesting work.

Sarah: Thanks.

John: Well, it’s always great talking to you. Thanks for joining us. And we look forward to hearing more about that project as more information comes in, as well as anything else that you’re doing that you’d be willing to talk to us about.

Sarah: Great. Thanks for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

348. Active Learning Initiative at UGA

Posted in active learning, assessment, engagement, evidence-based practices, faculty learning community, higher education, inclusion, learning, pedagogy, professional development
July 3, 2024

While there is compelling evidence that active learning results in increased student learning, these initiatives often face resistance from students and faculty. In this episode, Megan Mittelstadt and Leah Carmichael join us to discuss the active learning initiative at the University of Georgia that provides professional development for faculty, active learning training for students, and for the redesign of classroom spaces. Meg is the Director of the Center for Teaching and Learning at the University of Georgia. Leah is the Director of Active Learning, also at the University of Georgia.

Show Notes

Transcript

xJohn: While there is compelling evidence that active learning results in increased student learning, these initiatives often face resistance from students and faculty. In this episode, we explore a large-scale active learning initiative that provides professional development for faculty, active learning training for students, and for the redesign of classroom spaces to better support this initiative.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Megan Mittelstadt and Leah Carmichael. Meg: is the Director of the Center for Teaching and Learning at the University of Georgia. Leah is the Director of Active Learning, also at the University of Georgia. Welcome Meg: and Leah.

Meg: Hi.

Leah: Good to be here.

Rebecca: Today’s teas are:… Meg, are you drinking tea?

Meg: I am, I’m drinking double chai. Tea is a big deal. In our center. All of the people who work here have favorites. And we provide tea service at every workshop and I checked this morning, the most popular tea at our workshops is lemon ginger.

Rebecca: Oh, that’s good to know. Another teaching center full of tea. [LAUGHTER]

Meg: Yeah, we even have the teas of folks who don’t work here anymore that we keep stashed, just in case. So, Leah, you know, Colleen. Colleen drank…

Leah: Yeah.

Meg: …caffeine free peppermint tea, and we still keep it stashed, even though she doesn’t work here anymore. [LAUGHTER] Just in case.

Rebecca: How about you, Leah?

Leah: Well, I am a diehard coffee drinker. And so I am enjoying a cup of coffee. But my best friend is a potter, so I have her beautiful mug here to enjoy it with.

Rebecca: Wonderful. How about you, John?

John: I’m drinking a black raspberry green tea today.

Rebecca: Oh, that’s nice. And I’m back to my blue sapphire black tea.

John: Black and blue?

Rebecca: It’s black and blue. [LAUGHTER] We’re recording this at the end of the semester. [LAUGHTER]

John: We read about the active learning initiative in a recent Chronicle article and we invited you here to talk a little bit about the active learning initiative at the University of Georgia. We were really impressed to see that the university had budgeted over a million dollars a year for five years as part of the university’s Quality Enhancement Plan. This is a really large investment in effective teaching practices, and it’s especially impressive that this is happening at an R1 university. How did this initiative come about?

Meg: Yeah, we’re really excited that this investment is happening. And it kind of stems from an initial pilot that happened in 2018. There was a presidential initiative on active learning that resulted from a faculty task force. And so in 2018, our president allocated a million dollars to active learning programs, as a pilot. I feel super lucky to have a president who cares so deeply about the teaching enterprise at UGA. It’s something that you don’t see at every institution. And so I really appreciate it, value it, and count my luck there. Our president allocated a million dollars in 2018 for an active learning course redesign institute and a pilot classroom infrastructure renovation, and also the creation of a one-credit course for our students called Becoming an Active Learner that’s deployed by our Student Success Center. And that first year, we thought, “Gosh, it might just be a one-year investment, so we really got to make this count.” And so all the folks in the center sort of got together and backwards designed the programming for our first Active Learning Summer Institute, which by the way, was one of the most gratifying and enjoyable experiences of my career to get five educational developers in a room and get to go totally meta and apply the backwards-design process to the creation of a Course Design Institute. We nerded out. [LAUGHTER] We had like the room where we had all of our notes on the walls, and it was sort of our Beautiful Mind room for the Course Redesign Institute. We loved it. And so we designed this institute for 2018, and it went quite well. The folks at our institution who applied were way more applications than we could accommodate in the Institute and the folks who got into the Institute had a good experience. And we were able to show a couple of courses that had been redesigned seeing some really significant positive outcomes for students, both in terms of their depth of understanding, but also in some of the metrics in courses like numbers of Ds, Fs and withdrawals from courses decreasing in a few really key courses like our introductory mathematics course sequence. And so we were able to go back to the President and the Provost every year after that and say, “Okay, here’s how the course redesign Institute went, could we get funding to do it again?” And so we continued to do it every year. And this continued for a couple more years. And when we got into the 2020 timeframe, UGA was preparing for our next reaffirmation through SACSCOC. And for SACSCOC reaffirmation, an important component of that reaccreditation proposal is the creation and proposal of a Quality Enhancement Plan (or QEP). And so a committee was assembled to come up with what that topic might be, and they looked at programs that existed on campus as well as the university strategic plan. And luckily, by this point, our strategic plan had been drafted with quite a few key performance indicators related to the adoption of active learning practices. And so they identified this opportunity to scale the Active Learning Initiative at UGA. And so they proposed this topic, it was selected and now the funding has been stabilized for some of the pilot programs that existed, but also some newer initiatives to0.

Rebecca: There’s a substantial body of work supporting the efficacy of active learning and increasing student learning and reducing equity gaps. And it’s so exciting to hear about such a big investment and the interest of faculty [LAUGHTER] having more interest than spots for example, But I know one of the things that sometimes happens when faculty decide to start implementing active learning in their classes is sometimes there’s some pushback from students, but I heard you say something about a one-credit course for students. Can you talk a little bit about that? And how that ties to preparing students for this endeavor?

Meg: Yeah, so because we had this course from the get go, there was already this idea that reaching out directly to students and building their capacities and dispositions as learners to amplify their success in an active learning environment is something we might ought to think about. Of course, we also think about it from the faculty side too. One of our most popular sessions in the Course Redesign Institute is a session on student resistance to active learning, and we rely heavily on Tolman and Kremling’s Why Students Resist Learning for that session and then talk about some of the instructor facilitation and explanation strategies that instructors can use to ensure that student resistance is not cropping up and to help prevent barriers to student engagement. I think what’s really cool about Tolman and Kremling’s book is that they reframe “student resistance” to “barriers to engaging” and so we work with faculty to reframe student resistance as “Where are the barriers to engaging that we need to address as instructors?” And then on the student side too, the philosophy is “Where are there barriers to engaging?” And then “When can we do some really important work to help students appreciate that engaging in active learning is a really actually durable practice for them?” It’s really going to benefit them long term. And since that idea was already in the mix, as we were coming up with the programming for the QEP, there was a really large committee, I think, a 33-person committee of faculty, students, and administrators who proposed the programming that would go into our Active Learning Initiative. And one of the things that they wanted to think about was the role that metacognition plays in active learning. That was identified really early on because we were already aware of the student piece, thinking about metacognition as a key component of active learning. The committee spent quite a bit of time unpacking: “What are the learning dispositions involved in student metacognition? And what are the learning dispositions involved in lifelong learning and other benefits of active learning? And where’s the overlap?” So we looked at things like the AAC&U value rubrics on lifelong learning. And I think the other one was, I want to say, it was like transferable learning, I’m getting the title wrong. But anyway, there were two AAC&U value rubrics that we looked at, as well as some of the literature on metacognition and literature on lifelong learning. And our assessment team identified four student learning dispositions that seemed to exist across all of the categories that we were looking at: students acting more curiously, students engaging with initiative, students becoming more reflective in their practice as learners, and students drawing connections between what they were learning in their active learning courses and developing other knowledge and skills. And so we use this framework of these four learning dispositions to really drive our student programming. And I’m going to pass it over to Leah, so she can share a little bit more about how we’ve expanded beyond this one credit hour course to a whole portfolio of student facing initiatives.

Leah: So, first of all, I would like to say with the instructor development piece, I was an instructor on campus, a lecturer, and was in that first cohort in 2018, an international affairs scholar. And it really was the first time that I had had such a professional development opportunity that felt like it truly revolutionized the way I taught, the way I thought about my students, the way I interacted with my students. And so whereas I was taught to transform one of my courses, I ended up transforming them all. And when I was able to be on the team for active learning, I signed up, it was really excited to do that. So it was an amazing experience. For the student piece, what I really love is that we have not just gone to the instructors and said, “Alright, it’s time to consider something different and then your job is to not only do all this hard work to change and rethink your courses, but then can you do us a favor and sell it to the students?” That’s just such a burden and very difficult. And so within CTL, they do a great job of training us in the Institute to go in first day and frame the course, to craft our syllabus language in a way that would potentially be able to introduce this course before even day one and to help decrease that student resistance. So that was a really important piece. But we also would have these teams of students, these just superstars on campus, who are also part of the programming for active learning. One of my absolute favorites is the peer learning assistants. This is an on-campus paid position for students. They apply and they are able to either return to a class or return to an instructor whom they have worked with before. And they attend a class where they facilitate active learning approaches and activities with the instructor. So the peer learner is responsible for helping with the planning of the course and preparation with the instructor implementing it and then also of course, following along with the course materials. And this is a key: this near-peer experience has been key in allowing the burden to be shared in introducing active learning within a classroom. And the students look at these peers, maybe a semester ahead of them sometimes, maybe years, and they start to see these mentors that are built in. And these students know and care about active learning. They see why it works in the class and they’re returning to the class. And we have found that those classrooms have been just the most wonderful place to see this cultural change among students. We also have active learning ambassadors. And they are trained to do outreach. They do demonstrations for faculty. One of my favorite moments is we have an active learning summit every year open to the public. But we have an active learning summit in which they run a showcase where the faculty walk in, and there are students behind tables with interactive material demonstrating different active learning techniques and how they see it in their class, how they use it for study skills, and how they would enjoy instructors being able to incorporate it in their classroom. The faculty love it. They stop, they ask questions, they think, “Well, what about this? Well, this would be difficult.” And then students give that experience from their perspective of how these techniques work for them. And we see that the faculty really crave that and really enjoy being the learner in that moment, and learning from a student about what techniques work best for them.

Meg: And Leah, I think one thing that’s really unique about our quality enhancement plan that I haven’t seen at other institutions yet, or if they have them, I haven’t seen them, so mea culpa. But one thing that’s really cool about our Quality Enhancement Plan is that we have a partnership with our Office of Student Affairs where they’ve designed some programming through our residential life curriculum to embed active learning in the co-curriculum too, so that students are experiencing and talking about the value of active learning not just in the classroom with each individual instructor, but experiencing active learning outside of the classroom in the co-curriculum. Students who are leaders on campus are trained in active learning facilitation techniques, so that when they go in to serve as leaders of student groups on campus, they have some of these active learning tools in their back pocket that they can deploy and experience on the other side as well. So we’re really seeing active learning proliferated, not just in the classroom, but outside of the classroom too.

John: It’s nice to see such a holistic approach. In so many cases where we’ve seen active learning initiatives, it’s usually just designed for faculty development, to go and fix all those issues and challenges. And by approaching it from all these angles, it seems to be a really effective way of addressing some of the challenges that faculty face when doing this. One of the challenges, though, that faculty sometimes face when they introduce active learning is they go into a classroom where the seats are either bolted down, or are very rigid. I think part of your plan involves some type of infrastructure changes to support that. Could you talk a little bit about how classrooms have been redesigned as part of this process?

Leah: It’s such a key aspect of it, I have the anecdotal experience where I graduated from the Active Learning Summer Institute, went into a classroom ready to be highly engaged and interactive and set up have a highly mobile class design only to find that I was in stadium seating, with fixed seats and I thought, “Oh, no, what am I possibly going to do?” And so we have three foci, within the initiative. And we’ve mentioned the instructor development piece. So key, but as you said, it can’t stand alone without support for the other two. We have the student piece, we also have the classroom piece. And so our goal is to every… usually it has to be in the summer, when it’s much quieter… every summer, we just push for classroom enhancements across campus. And we have three criteria in mind that we see as not necessarily perfect for every single classroom situation, but the baseline for which we want to capture for an active learning classroom. And so the first is: does this classroom have mobile furniture? Are we able to move students around? Could an English professor have a roundtable discussion? And then within the 20-minute transition time, could my students go into that same classroom, which happened last semester, in a highly mobile classroom and be able to set it up into a simulation of roleplay thing for my international politics course? So do we have classrooms with that mobile furniture is the first thing. The second is, do we have student collaboration tools? And these do not have to be high tech. In fact, we ran the survey, and students and faculty said just give us more dry erase boards. We don’t need the fancy, fanciest of smart screens. Please just give us the basics. But let us work on a project huddled over together and be implementing and creating something new. And then finally, it’s just general space, do we have enough space that we are able to not only move the furniture around, but move people around in an effective way. And so 25 square feet per student is kind of our ultimate goal. And so we looked around and I was actually just running the numbers recently. And I think on campus we have, depending on how you measure our classrooms, between 400 to 600 spaces, it depends on if you count labs, and these other learning spaces, about 82% have at least two of these criteria. And so we’re getting there. We’ve made a real push and we’re trying to say: “Yes, instructors, please change but also know that we’re going to meet you with an environment that facilitates this kind of approach to learning.

Meg: Well, I also have to interject and say that the stance of the CTL is that active learning can happen in any classroom no matter the features. And so of course, like a good backwards design course or re-design institute, we always start out with the situational factors of a course. And that might include what’s the likelihood that you’re going to be in a fixed-seat auditorium or fixed table, but moveable chairs, and think about the choreography of that, the active learning techniques that you’ll adopt based on that. But in Leah’s example, in particular, if I remember correctly, Leah, you had a student in a wheelchair in your course the first semester after you redesigned your course. And when you were in the tiered auditorium, that meant that there was a specific area where the student in the wheelchair could sit and they couldn’t move through all of the case study roles that you had available. And so you were able to advocate for a different classroom, you obtained one, thank goodness, and everyone could move through a flat classroom instead. And so sometimes things crop up, even if you have planned to be in a fixed-seat auditorium, that you have to be able to modulate on the fly based on the characteristics of your students each semester too.

Rebecca: Can you talk a little bit about whether or not your program has focused on specific programs or if it started with specific programs and its spread to different programs or what your strategy is for involving faculty across disciplines?

Leah: One of the most important parts that I really liked about this initiative is it’s not a mandate, we are not requiring any instructor to adopt active learning through a mandate. Instead, it really is a support, educate, encourage. And so we really have not been focused on okay, this unit by X year needs to have Y outcome in terms of an active learning goal. But in terms of strategy, what’s nice is we can start to see the people that walk in the door to find the active learning Summer Institute. And the high demand that first year was a nice little kind of bellwether that, okay, there is something going on here. When you have more people that can fit, and that’s consistently. The CTL needs to keep it kind of a quality curated experience with a cohort that goes through and has to do these really difficult re-education, understanding education differently, and then of course redesign. Such a dense set of tasks. So with this cohort like experience really requires so much of the faculty. What we started to notice, as we looked around and realized that instructors from different schools and colleges were coming to the Summer Institute, and we have seen that as the CTL has expanded beyond the Summer Institute and provided workshops, both standalone workshops, as well as a course redesign experience outside of the institute. We see hundreds of faculty each semester coming into the programming of active learning. And really we have, I believe, 19 schools and colleges that have undergraduate programs, and we have representatives from each that participate in this programming.

Rebecca: That’s wonderful.

John: What type of training do faculty receive as part of the institute and other training that you provide?

Meg: It was initially just that summer Course Redesign Institute. And what we started hearing from faculty as we would get these applications every year is the deadline would pass, and we’d hear from some faculty who would say things like, “Oh, I really, really want to do the institute. but summer is my field research season” or “I’m in the clinics in summer,” or “I have childcare duties that I can’t get away during that time of year.” And so there started to be some sort of advocacy for other on ramps into the programming, so that when we had the opportunity to stabilize the funding through the Quality Enhancement Plan for active learning initiatives, we started thinking about how can we make this program one that faculty could find the on ramp that is appropriate for their needs? Is it a faculty member who’s been using active learning for a long, long time and they really want to brush up on their skills in one particular area? And so that might look like engaging in a specific workshop on something like facilitating group discussions in a large lecture environment, or looking at student metacognition and how that plays into a particular student developmental level that they might teach in their courses? Or does somebody want to do the whole workshop series and Course Redesign Institute all the way through, but they can’t do it during the summer, or anywhere in between. And so what we ended up with is a workshop series, we have 12 standard workshops that we offer over time: four fundamental workshops, and eight special topics, workshops, and we rotate through those. We offer a couple of each every year. And then we have the active learning summer institute that happens every May. And then starting next academic year, over the course of the spring semester, we’ve designed a course redesign experience that takes the course redesign elements out of the active learning Summer Institute and decouples those elements from the active learning workshop components of the Course Redesign Institute that we currently have, so that faculty can upskill through the workshop series. And once they’ve taken eight of the 12 workshops, they can apply for the course redesign experience over the spring semester. Through that course redesign experience, they come and meet with us for a couple of hours every three weeks or so over the course of the spring semester. And the idea is that by the end of that course redesign experience and workshop series completion, they’ve recapitulated all of the experiences that you might have during the multi-week summer Course Redesign Institute. We also provide different communities for folks to engage with. We have several robust Faculty Learning Communities on campus. So we have multiple faculty learning communities related to active learning, one specifically coming up this year that Lea generously funded from the Active Learning Initiative budget for folks who are looking to engage in SOTL, Scholarship of Teaching and Learning, on their active learning endeavors, as well as specific faculty learning communities for the folks who have peer learning assistance in their courses, or who are engaged in the work after the course redesign experience, for the alumni of the Course Redesign Institute. We also have opportunities for faculty who are highly engaged and master teachers to serve as active learning mentors to their peer faculty. So we have three active learning mentors every year who work with the folks who’ve graduated from the course redesign institutes to give them opportunities for office hours with their faculty peers for micro mentoring sessions on specific topics of interest. They also create supportive resources. We have similar mentors for our PLA faculty, our peer learning assistant faculty. And for those who are serving in those mentoring roles, the Center for Teaching and Learning provides them with opportunities for additional coaching and support on instructional development, educational development, instructional coaching practices that they might bring to bear in those consultative sessions with their peer faculty, too. So the goal here is that we have some workshops that are entry level, some special topics, workshops, the course redesign experiences, some stuff for the master teachers, who are really wanting to be mentors and wanting to kind of serve as role models within their spheres of influence and for others who are moving through this programming, and we’re helping them build capacity in those areas, too. And then, of course, we have faculty on campus who have been working in active learning for years and years and years, and who published on active learning, who are real role models nationally. And so we really try our best to honor their expertise and bring them in to help frame the ways in which we deploy our programming, but also to serve as guest speakers and to engage with folks who are newer to active learning so that we can help provide opportunities for that cross-collaboration and multidisciplinary discussion.

Rebecca: I love the range of options you have available, and really how much you all have embraced the idea of flexibility for your teacher learners, just like we might want in our classrooms as well.

Meg: Yeah, yeah, that’s right.

Rebecca: Can you talk a little bit about some of the active learning techniques that faculty have adopted after going through the institute or other variations?

Meg: Yeah, so as you’ve heard now, our summer institute is three weeks long. And so not only are we giving sessions on active learning, but we’re modeling active learning throughout. And so folks see a lot of different techniques that they might not have seen before. And they’re trying to identify which things that they’re seeing, which instructor moves they see, that align with the needs of their students, of the needs of their course, and their teaching persona as well. So as you might imagine, folks to leave the institute, select different things that are authentic to their practice and their situational factors for their course. So we go into the Institute, which actually begins today for summer 2024. We’re recording on the first day of the Active Learning Summer Institute. So we’re getting the anxious energy of meeting the new faculty. We start out on the first day of helping faculty appreciate that we don’t see active learning as a homogenous set of teaching activities, we’re not going to give them the recipe. We want them to discover the recipe that works for their course. So our QEP is sort of predicated on the notion that the extent to which students are actively engaged in thinking and applying what they’re learning is of far greater importance than the particular active learning instructional strategies used. And so to that end, the course redesign process helps faculty first hone in on the key enduring understandings and learning outcomes at the course level they want their students to leave with and identifying key learning bottlenecks or threshold concepts that they’ve identified that students tend to either have a tough time with, or they’ve really had to work hard to help students grasp that concept in order for them to move from novice to expert in that field. And then we encourage faculty to design their active learning interventions in alignment with those key features of the course so that active learning is more like the trunk of the tree rather than the ornaments on the tree. And our philosophy there is that when pressed for time, the ornaments are easy to cut, but if the act of learning is baked into the backbone or the trunk of the course, that’s just inherent to the identity of the course and it’s not something that’s going to be cut for time. And also you’re putting your energy into where the biggest impact is going to be for your learners..

Leah: If I may add, this was the revolutionary part of being a participant in the Active Learning Summer Institute. I thought I was going for the tips and the tricks, I’d get my little suitcase of ideas and I would just kind of put it all in there through the Institute, and then graduate and use all those as those ornaments as Meg: described it. And instead, I found myself looking at my course and realizing that the backwards design principle had me throwing a lot of content at my students, and not really aligning the activities in the course towards the student learning outcomes. Once I was able to do that hard work, then the excitement of “Ooh, okay, well, which tips and tricks, which kind of active learning engaging styles would be most useful for this.” And that’s just a really exciting way to do it as an instructor.

John: Most instructors, as they come through their educational process, have not been exposed, particularly not in graduate school, to active learning techniques. How do you address issues of faculty resistance to completely changing the way in which they’re approaching the design of their classes.

Leah: It takes a while, [LAUGHTER] having been one of the resistant faculty, I’ll speak to it from this perspective, I think Meg: referenced the former colleague Colleen, and she was my mentor going through this institute. And I remember her asking me to write all the content that I felt was so essential to get the students to to learn throughout the semester, and I distinctly remember this moment where she said, “Okay, let’s put that in an envelope, all that stuff that you just wrote down, we’re just going to put it in an envelope, and we’re just going to set it over here, it’s still very much matters, don’t worry, it’s not going away. It’s all still important.” And then she said, “Let’s really think about the future student. Who are you teaching? What would be a success?” And I think that’s a moment where I could take a deep breath, as a faculty. At first, it was scary and fearful. But it’s a moment for the first time you take a deep breath and go, if I miss a case in international law, they are still going to thrive in this world. I’m focusing on what skills they actually need to thrive in this future career. And so it’s not about that specific case, it’s about the skills. And once I was able to realign that, and it’s so important to be sitting in a cohort, it’s not an isolating experience, you’re sitting in a cohort with other people doing the same reformulating of their whole profession, with mentors from the CTL, helping and encouraging them. Without that sort of ecosystem, it would be a very lonely and harder… just slow… process, maybe, even if you are willing to change. So I think the quality of that instructional development experience within the CTL cannot be replicated. And then I hope the other two kind of prongs of the initiative, once they leave the institute, they’ll still have the mentorship for the next year with CTL and with the faculty mentors. They then can leave and say: “We have the classroom engagement things.” We are also trying to change the environment to help you and we want your feedback. We are always asking those active learning instructors, what can we do differently within these classrooms? And then also the students… we’re engaging the students. So you’re going to walk into class, and they’re going to know what you’re trying to do, and that it’s recognized as a really good thing to do across campus. So it’s the ecosystem that will create more empathy and support and therefore reduce the resistance.

Meg: Yeah, so Leah mentioned, this idea of you have this mentor that exists for the year after the institute. And I don’t know that we explained that. That’s a pretty unique feature of our Course Redesign Institute. And it comes somewhat from our Center for Teaching and Learning has quite a bit of tradition around investment in longitudinal cohort-based programming. So we have a lot of one-year, two-year, multi-month cohorted faculty development experiences, because we find that that interdisciplinary conversation is so helpful for helping folks build connections and shared disciplinary content, and teaching practices with one another in ways that proliferate those great ideas across campus and seed innovation. And as we thought about the ways that our faculty fellows…we call them Faculty Fellows Programs at UGA… ways that our faculty fellows programs were successful, were in that sort of longitudinal support that was available. And we wanted to replicate that as best possible with our Course Redesign Institute. And when we thought about the outcomes of our institute, and how we wanted people to leave the institute, be prepared to deploy a redesigned course in the coming academic year. And we started thinking about what that experiences like as a teacher, and we thought, “Gosh, people are going to leave the institute, be all excited, go back to their department, where they might be one of few who’s engaging in this sort of robust active learning commitment, especially in the early years of the institute.” How can we support them in being successful when they go into their first class and leave the class and go: “Ooh, that didn’t feel like it usually felt I wonder if it went well?” or they do their first robust activity and maybe it doesn’t go quite as they were expecting? How can we be there for them in those moments? So as folks leave our Course Redesign Institute, they’re each paired with a mentor from the CTL. And that mentor is their touch point for the year following the institute, so that when they have that class session that throws them for a loop or they get some feedback from students, that they’re not quite sure how they’re To respond to it, they have a touch point that they can go to. And it’s someone who knows their course intimately, who’s worked with them through the redesign process, is aware of the ways in which their course has changed, and is there to provide them with that support. And also that mentor is the person who performs… we call them mid-semester formative evaluations… but at other CTLs they’re called small group instructional diagnoses. So at the midpoint of their course, someone from the CTL comes in and talks with their students to focus group them, to collect some feedback on how the course is going so far, and then works with the faculty member at closing the loop on that feedback to help students appreciate that they are such an important component of the redesign process too, because the instructor is engaging in this two-way exchange of information with the students both on the content and on the operational organization of the course and refining the course in response to that feedback. So we want the students to appreciate that their experience and the way that they’re learning and what they think could be happening to support their learning even better, how that could be incorporated into the course as it continues.

Rebecca: I really love the depth of the culture change components that are built into this initiative, from feeling like you’re part of something bigger, the fact that it’s voluntary, your support structures, it’s holistic, both Academic Affairs and Student Affairs, there’s opportunities for feedback loops. Change can be really difficult, but when you’re a part of that bigger network, it can be helpful. The timing of this was really interesting when it kicked off…so, a couple years before COVID, so you had some time to establish the roots, and then COVID erupts, AI erupts, there’s more first-gen students, we’ve got a lot going on, and there’s a lot of change happening. How do you help your faculty avoid burnout with so much change and maintain this momentum?

Meg: Yeah, great question. You’re taking me right back to the pandemic and that experience of everyone just doing the best emergency remote teaching you could do. And one of the things that I remember from that period is faculty who had gone through the Active Learning Summer Institute mentioning that they were so glad that they’d gone through the institute because it made pandemic teaching easier. They had recently conceptualized of their course learning outcomes that identified the most important content that they wanted to prioritize, they had a sense of some lesson plans for how to deploy these things. And even though the institute happens in person, because of the way we were talking about how you would make adjustment to different things in different settings, they could more easily conceptualize of how to engage online or asynchronously with students using active learning principles. So we had this early indication that the folks who’d experienced the course redesign process felt better prepared for emergency remote teaching and pivoting quickly. We also found active learning to be perfectly timed as an initiative as a response to post-pandemic disengagement. So when students were coming back into the classroom and members of our community were craving that deep intellectual engagement and community with one another, but then in some instances, were a little bit disappointed when it didn’t materialize in the ways that it might have before the pandemic, we found that faculty were coming to the institute, saying, “This is something that I’m hoping can meet this need that I have to engage students in rich conversation or to engage students in deep intellectual engagement in this learning community, and how can I address this through active learning?” And also, we’re lucky to have a large number of faculty champions for active learning on campus, folks who’ve been using active learning for a while, and in various departments across the institution, not just in specific pockets. And so they’re serving as role models and mentors to the faculty in their unit. And a message that those faculty champions were helping us reinforce is that teaching with active learning is a joyful way to teach. And so oftentimes, when these faculty champions, these mentors, were talking to folks who were saying things like “post-pandemic teaching is a little bit disappointing,” or “I’m feeling a little bit burned out because of everything that we’ve experienced through the pandemic, and through these various disruptions to higher education.” When their mentors were telling them, “Have you considered bringing some joy into your classroom and making teaching a more joyful experience for you and for your students? And guess what? You can do that through active learning. Let me tell you how I brought joy into my classroom through active learning.” We would see folks come to us and say, “I’m just really looking for some rejuvenation and some revitalization, and I’ve seen my colleagues just transition their course through active learning in a way that they’re just skipping down the hall after they leave their class, and I want that feeling. So I’m hoping that this will be that experience for me.” And so we had people coming to us asking for that, and we hoped that would be the case. And then I’ll just add that like many higher education institutions right now we’re in the midst of an evaluation of teaching culture change, too. And so we’re trying to adopt a more what we’re calling a three-voice culture of teaching evaluations that’s relying upon your individual reflection on your teaching some data from your students and the interpretation and contextualization of that data, as well as peer observation of teaching data and bringing that all together as sort of like triangulated data to provide some insights on our teaching, and then respond to that and have like a assessment loop on this. And so it’s this idea of a culture of continuous improvement with teaching where we’re able to proliferate this message of: “You’re not an award winning teacher necessarily on your first day of teaching, but this is a set of skills that you can develop over time. And also, you’re never done developing it.” And so this culture is helping to amplify the message that taking part in educational development is not just an expected part of being a teacher, but it’s an opportunity to continue your culture of continued mastery.

John: So one of the ways in which this is spread has been from faculty to faculty. One of the things I’ve been noticing a lot on my campus is that it also sometimes spreads from students to faculty. As more and more faculty move from faculty-centered teaching to student-centered teaching, students become a little bit more vocal sometimes about expressing what works for them and that will often result in faculty considering using new techniques that their students have recommended to them. Has that been happening as well at your institution?

Leah: That’s why we wanted to capitalize on this proactively and have channels of growth. And so again, we have little areas in which we have students really playing that intentional role. So rather than maybe a complaining, “I went into this class and I didn’t like it” kind of a customer satisfaction problem area, we really want to make sure it’s more that students, first of all, are trained in the active learning pedagogy. So my active learning ambassadors, as they are chosen, we then say, “Okay, here’s the wizard behind the curtain friends, come join us [LAUGHTER] and see what’s going on behind these intentional classrooms that you really enjoy and don’t quite know why you like it so much.” So we bring them into the understanding of what instructors are doing when they’re in an active learning classroom and why. And then we give them opportunities to then go and front face and tell other faculty, “this is what an active learning classroom feels like for a student.” This is why one of my favorite moments… we have, again, as I mentioned, an active learning summit every year in February, and there was an entire breakout session run by students who were peer learning assistants and active learning ambassadors. And they were talking about the study skills that they have seen through their training, or through their classes that really work in their discipline that they use outside of the classroom to kind of empower themselves to study and to learn. And what was wonderful is that the front of the room were students in a panel and most of the room or faculty and I found that the question and answer period, the instructors would raise their hand and say, “Okay, well, but I’m not in that discipline, but I’m thinking about bringing this into my class in X way, is this something that you would find effective?” And they were asking the students about designing their course to be student centered. And that’s when I think if we put those students that know what is going on, so it’s not a complaint section, it is more a reframing and a sharing out of we know what you’re trying to do and we really appreciate it, and here’s how it works really well, for us. It just starts to kind of change that conversation from being yet another instructor versus student dynamic, which I think we have plenty of those around campuses at any given time to becoming an instructor and student experience.

Rebecca: Having student ambassadors and student voices celebrating this work, is certainly a testament to the success of this work. And you’ve also mentioned some other moments of assessment. Can you talk a little bit about how you’re assessing this initiative?

Leah: There is a rich assessment team who is actually part of the larger Active Learning Initiative. And so I hope I do justice to their very nuanced framework that they have. We have macro level, mezzo level, and micro level indicators that we are trying to capture. At the most macro level demand is one of the earliest indicators of success for us. Are people applying to the Summer Institute? Are ambassadors applying for the program? How many peer learning assistants apply and how many instructors want them, just really kind of those basic quantifiable amounts for demand. And then we also are attempting to capture survey data across the university. We stop the students and say: “We don’t know if you’ve interacted with active learning in the classroom or not, we’re not selecting a given class, we’re just wondering what you would say if you observe a cultural change going on.” So we’re also trying to get these larger macro level indicators. Then at the mezzo level, we’re getting into those lifelong learning dispositions that Meg: referenced earlier, the curiosity, the initiative, the reflection and connection. We have artifacts that we pull from courses that have an instructor who’s either going through all the Active Learning Summer Institute, or who has gone through it already, so we do a pre- and post-analysis. And we have faculty scorers, who can go through these artifacts to see if we are capturing if the artifacts include a prompt about curiosity or a prompt about initiative. And then we are attempting to measure, and we’re working on different ways to do that, but measure if we’re seeing growth over time in these lifelong learning dispositions, and then finally, at kind of the micro level, we’re making sure that the disciplinary knowledge is being enhanced within this that are focused on active learning techniques. One thing we have yet to mention is yet another part of our holistic umbrella. This is why the programming is so large, but we have these active learning change grants. And this felt really good to hear for me as an instructor who went through the institute. And then it was about five years later that the next person in my department went through the institute. What these change grants do is it really allows those who go to the institute to go back to their department or their unit and say, “What if we change several classes at a time through some sort of infusion of deep learning and the CTO will support us, the assessment team will help us assess whether we’re doing what we’re saying we’re doing.” And so on this micro level, the assessment team is really starting to capture some pretty cool indicators, that when you kind of create a unit change, there’s a cascading norm it’s called in international affairs, where you change enough of the unit and enough of the actors in a given scenario, you can then really see that norm start to shift in a unit. And so at each of these levels we’re finding it’s still early. So we’re not saying “It’s a win, close up shop,” [LAUGHTER] but we are finding really robust indicators of early success.

John: Well, you’ve given us and our listeners a lot to think about, and it sounds like a model program that could be copied very productively at other institutions. But we always end by asking: “What’s next?”

Meg: We’ve started to hear from folks that are interested in proliferating initiatives. So, of course, we welcome conversations with folks who are interested in hearing about some of the elements of the program and brainstorming ways to deploy these on other campuses, lessons learned along the way, etc. I mentioned earlier that as folks crave these multiple on ramps, we’re deploying these additional on ramps over time. So starting in the fall, we’ve designed some of our foundational active learning workshops, we’ve taken those and created asynchronous online versions of those workshops that are going to be hosted in our learning management system. So they’ll be available on demand. So if a faculty member doesn’t happen to be available at the time that we’ve offered that workshop this semester, they can still engage, especially in those foundational workshops. And then the spring semester course redesign experience version of our Course Redesign Institute will be deploying for the first time in spring 2025. We’re really excited about that as well.

Leah: I would also say for your listeners, if there is an interest in kind of seeing what we’re up to and helping participate in engaging with this discussion, traditionally the active learning summit has been really focused on UGA community for the first two years, but in the third year, so February 2025, we are going to open up the summit to visitors and encourage them to be a part of these breakout sessions or propose different ideas that they would like to talk about. I know Rebecca mentioned earlier, the challenges of AI, we’re going to try to have a breakout session, though, that focuses on this: active learning best practices in the age of such change. We’ll send out a call for applications in the fall, and we’d love to have anybody who’s interested in learning more, I think it will be Thursday, February 27, and Friday, February 28, on UGA campus, and we welcome all those who are interested in attending.

Rebecca: That’s great. Thank you so much for joining us. There’s so much to think about and so much exciting work that you’ve all done.

Leah: Thank you.

Meg: Thank you so much for having us.

John: Thank you and we look forward to more conversations in the future.

Meg: Absolutely. Anytime.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

 

339. Industry to Faculty

Posted in active learning, assessment, engagement, equity, evidence-based practices, grading, higher education, learning, pedagogy, professional development, STEM education, teaching
May 1, 2024

Some faculty begin teaching as a second career, after working in industry. In this episode, Kevin McCullen and Michael Walters join us to discuss how their prior careers in industry helped prepare them to design authentic learning experiences for their students.

Kevin is an Associate Professor of Computer Science at SUNY Plattsburgh. Prior to joining the computer science department at Plattsburgh, Kevin worked for several years at IBM. Michael is an Associate Professor and Chair of the Physics Department at SUNY Plattsburgh. Prior to joining the Physics faculty, Michael was the CEO of EISWorks Technologies and a metrology engineer for Corning Inc.

Show Notes

  • Design Automation Conference (DAC)
  • Consortium for Computing Sciences in Colleges (CCSCNE)
  • Clark, D., & Talbert, R. (2023). Grading for growth: A guide to alternative grading practices that promote authentic learning and student engagement in higher education. Taylor & Francis.
  • Lang, J. M. (2008). On course: A week-by-week guide to your first semester of college teaching. Harvard University Press.
  • CircuitPython
  • R.P. Colwell, At random – Employee Performance Reviews, IEEE Computer, 35(9), 12-15, Sept 2002

Transcript

John: Some faculty begin teaching as a second career, after working in industry. In this episode, we discuss how prior careers can prepare faculty to design authentic learning experiences for their students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Kevin McCullen and Michael Walters. Kevin is an Associate Professor of Computer Science at SUNY Plattsburgh. Prior to joining the computer science department at Plattsburgh, Kevin worked for several years at IBM. Michael is an Associate Professor and Chair of the Physics Department at SUNY Plattsburgh. Prior to joining the Physics faculty, Michael was the CEO of EISWorks Technologies and a metrology engineer for Corning Inc. Welcome Kevin and Mike.

Mike: Thank you.

Kevin: Hello. I will just say 32 years is several.

John: It is. [LAUGHTER]

Rebecca: I think that’s the official definition. Well, today’s teas are:… Kevin, are you drinking any tea?

Kevin: I have Harney and Son’s Hot Cinnamon Sunset.

Rebecca: Oh, that’s a good one.

Kevin: It’s my go to tea.

Mike: And I accidentally left my tea in the car, but I do have a cinnamon tea that is by Tazo.

Rebecca: Yum..

John: Very nice.

Rebecca: Cinnamon’s are a favorite at my house.

John: I have that too, and I enjoy that one. Today, though, I have a spring cherry green tea.

Rebecca: And today I have Hunan Noir, which is very tasty.

John: Each of you received a PhD and worked in industry for a number of years before joining the professoriate. We’ve invited you here to discuss the transition from industry to academia. Many of us just went to school and never left. And we thought it would be interesting to hear the perspective of people who’ve been out there working and returned to academia. Could you tell us a little bit about your initial careers in industry?

Kevin: Well, actually, I took the roundabout path in that I joined IBM after my bachelor’s degree and IBM paid for my graduate work through programs that they don’t have so much anymore. And I always wanted to teach. And when I graduated as an undergraduate many years ago, I was kind of burned out on school. And then it gets hard to leave after a while because it becomes very comfortable. But eventually I kind of felt like it was time. I was becoming I would say more uncomfortable in the situation I was in in industry and opportunity presented itself. I had actually been looking for a number of years for a good local opportunity because I didn’t want to relocate. My wife still works for IBM. And Plattsburgh had an opening and I applied for it. And there’s a very funny story in there with the conversations with my management over that. But I applied for it, and they accepted me, and it’s been a lot of fun.

John: You mentioned a funny story, I think [LAUGHTER] that requires a follow up.

Kevin: Oh, it was just that I had submitted a presentation to an industry conference. So I’d gone through all the clearance work to get it approved, patent clearance and all those things, and it was not accepted. And then when the position opened up, and I applied for it, and Plattsburgh invited me to come over and give a presentation, I went to my manager, and I said, “So do you remember that presentation that we got approved for DAC? Can I go give it at SUNY Plattsburgh?” And she said, “Why?” And I said, “Well, they have an opening, and I’d like to apply for it.” And I said, “Don’t worry too much about it, because my guess is that they can’t afford me, even if they do give me an offer.” And she said, “Sure, go ahead and do it.” So I came back to her about a month later. And I said, “Good news or bad news. The bad news is they can’t afford me. The good news is I’m going to take the job anyway.” And we parted on good terms.

Mike: So I did get my PhD before starting in industry. And I sort of backed my way into industry. I was thinking about staying in teaching, but I was worried I wasn’t a good enough physicist to actually go for it. I have impostor syndrome very badly. And so I looked at going into industry because I was worried I was going to only teach freshman physics for the rest of my life, because that’s the only thing anybody would trust me to do. I went into industry for nine years, I did not have to approach anybody to see if it was time to leave, because they approached me and said, “Congratulations, you work your way into a research position at a production facility. That’s a pretty awesome job. And now we’re in a recession. So we don’t need you. Goodbye.” [LAUGHTER] I applied to a couple places in industry, had a position semi-offered to me at a solar panel facility in Oregon. And I could not say to the person who’s trying to hire, “Do you want this job?” He asked me three times. And after the third time, his shoulders just slumped and said, “You can just tell, okay, that was done.” And on my way home, my wife asked me, “Well, how did it go?” I’m like, “ah, I didn’t get that job.” I told her what happened. She’s like, “That’s lovely, dear. You have to figure out what you want to be when you grow up because we need a paycheck.” And at that time, I said, “Do you mind if I tried to go back into teaching? I think I really want to go back to it. Maybe I can not teach just freshman physics.” And so I adjuncted for a year at two institutions near my parents’ house. And SUNY Plattsburgh was looking for somebody who had a physics background but had engineering experience for their engineering three plus two program. I was a visiting professor for a year and then worked my way into a regular position.

Rebecca: I think there’s a lot of folks that are considering moving around and shifting careers at different points in their career. Can you talk a little bit about what that actual transition felt like that first year, maybe, back in academia?

Mike:Well, I can say three things that strike me when I was moving back into academia. Number one, I surprised myself finding out I can teach something besides freshman physics, so that was a great thing to figure out in getting some confidence there. The second thing was, and Kevin and I have joked around about this before, the pressures of academia in general, compared to the pressures in industry are at different scales. [LAUGHTER] And so we used to have fun when people would come up and talk about the pressures of doing stuff, we’d sort of chuckle a little bit, and go,”Yeah, this is pressure,” and I respect it, because there is definitely pressures in academia. But we can also talk about the times when you’re sitting around the office going, “Am I on the cutting block this time around?” …because you know, layoffs are coming this month, or even if you’re not on the cutting block, looking around at your friends and going, “Who’s not going to be here next month?” …is not a pleasant feeling. So being in academia made me feel more relaxed. I didn’t have tenure yet, but just the idea of knowing you belong, and you would know, hopefully, coming up if something was going to happen that way. And lastly, the camaraderie of the department … I am blessed to be in a department that was a very close-knit department. They were very supporting, they felt bad that initially, my office was in the middle of nowhere in a closet, basically, because that’s all the space they had. Kevin, I was on the fourth floor in one of those little side cubby holes. I only could see the sky from a skylight, literally everything else was cement.

Kevin: I thought you were talking about the office I first saw you in which also resembled a closet.

Mike: Yes, well, that was a different closet, [LAUGHTER] but it was a better closet. But anyhow, it was great having that camaraderie and having that trust, and that trust grew into a new major, because they trusted me to go ahead and take out the robotics major, and kick the wheels and let it go out for a spin.

Kevin: For me, it’s autonomy. I see a lot of people, I read the Chronicle and such, and it’s like making the transition to industry, and I’m thinking they’re going to have an interesting end of the year when they sit down with their manager to talk about how they did against their goals for that year, because it can be a little head spinning. My first year, I kind of felt a little lost, because it’s like if you had a guide, if everything you do is coming down from above, even if we get to a relatively high level in an organization like IBM, you’re driven by strategies that are sent to you and you are a piece of that strategy. And now you’re kind of an independent agent. And it took me a little while to get comfortable with that. But Mike and I both know people who really, really are struggling with the pressures of academia and all the things. I feel like I have a lot more ability to say “JNo,” and to say, “No, this is my limit of how much I am willing to take on and so I’m just going to steady on here.

John: I was talking to one of my advisees, who’s planning to go into a PhD program. And he said he was researching the market for a PhD economist, and he noted that jobs in industry paid a lot more than jobs in academia, and jobs in the government was somewhere in between academic jobs, and that, and he asked why there was such a big difference. And I used basically the argument that you just used, that you get a lot more autonomy when you’re in an academic setting, in terms of research, in terms of what you teach, in terms of what you focus on. And that’s something that I think we all value quite a bit. Was there anything that was surprising when you made the transition to an academic environment full time?

Mike: I’ll go with the autonomy. I mean, that was really surprising. Where all of a sudden nobody was breathing down my back. Where are the deliverables that you were supposed to get here? Other than getting classes together and doing your best that way? Yeah.

Kevin: I’m gonna go back to actually, it was a funny question I asked the Dean when I was hired, because I have kind of a long commute, as does Mike. And I was a little concerned about doing a commute from Vermont to New York five days a week on the ferry. And I asked the Dean, what are the expectations in terms of like, where you work, because even at IBM into the 2000s, we did a lot of working from home, pretty flexible work environment. And he said, one of the biggest changes when he moved out of teaching into administration was an expectation that he was in his office. And it’s the freedom and the flexibility. I’ll say one other thing, that was kind of weird, though. It’s the summers, and I’m a volunteer with several organizations. And immediately there was an assumption that I had nothing to do during the summer, and that I would immediately become available to them. And what I’ve taken to telling people is, the way that summers work is, I have a lot of things to do, but nowhere I have to be. And by the way, biggest surprise is that from what I have observed, we have the whole summer and everyone still does their syllabus three days before the semester starts.

Rebecca: We’re deadline driven, [LAUGHTER] just like our students. We are no different and that’s for sure. Can you talk a little bit about how your prior work experience has shaped your teaching practices.

Mike: So when I first came back into teaching, I taught in a very traditional sense. I would have the PowerPoint slides, I would have the graded homework, I would have the tests and exams in class so we can make sure everything is all above board and all that other fun stuff. And then as I go through, I think more and more back on what was important when I was working, and how that work environment operated. I now have moved as far away from trying to test as minimally as possible, if not never, and do much more project-based learning because that’s what I did. And even when I give a test, I would give a two-fold piece of the test. The first bullet piece is the closed book portion, which never ever had a calculation in it. And I told my students, when I was at a meeting, I was there as a knowledge source. And so I’d have to know somethings off the top of my head. That’s what they paid me the money for . And then if they asked me a question about X, Y, or Z, that had to have some calculations based into it, we’ve never had to do it right there on the table. That was not something you did. You could go back to your office, you could use all your resources, you could pull together the things you needed for that project. And so I tried now to model my classroom along those points, I will tell them, these are the things you need to know off the top of your head, this is the stuff that they’re going to pay you for that boom, boom, boom, you need to know. But everything else, Google is your friend, having your textbook there is your friend. And so on any of that stuff it’s open book, open notes. It’s a little harder with AI now. I’m not going to lie. But still, in general, if I was in industry, that would probably be where I would start as well. Oh, I need to do this. Okay, “Hey Google blah, blah, blah, blah, blah.” There’s my starting point. Now let’s move on forward.

Kevin: I think two things. So I spent the 32 years that I was at IBM, I spent one year doing microprocessor design and the other 31 years, I was in software development. And I’m a big believer in soft skills. And some of that comes from also being an ex-Scoutmaster, and heavily involved in scouting. But I tell my students, I try to instill in them the idea that your communication skills are vital to your advancement in your career. Everybody can code, but as you advance in your career, as you go further up the ladder, you’ll have larger responsibilities and you have more communication responsibilities, till you reach a certain point in which you live from being as technical toward being more in terms of communication. But one of the last things I worked on was I was basically project managing the, what they call the enablement for a test site, which is to say, just getting all the pieces together to build an experimental chip and a new technology. And that was primarily a communication job, it was a project management kind of thing. The other thing is the importance of group work and project work. And being able to work with other people that those, I think, as well as for my computer science students, algorithms really do matter. And anybody can write code that slow. But it’s important that you understand enough of the algorithms. And that’s because I entered industry as an electrical engineer, I had no background in any of those things. And I wrote some spectacularly bad code, when I was back in my 20s, before I started taking some graduate courses and going, “Oh, wow, there’s a better way to do this.” And so those are kind of the things that I think are important.

Mike: I want to piggyback on the group work part, I do a lot of group work as well. And one of the things I tell my students because I will always get the “Well, I’m the strongest person on this team, I’m doing all the work.” And I go to a “Yeah, congratulations. I said there was never a project that was less than $20,000 I worked on by myself. You’re always part of a team, you’re never a loan problem solver, if there’s money involved, because they want the others to be part of it and keep track of it and make sure it happens as fast as possible.” So group work where sometimes we let them off because of the whole stigma of “I’m going to be the only person working on it. Everybody is going to be a hanger on.” Well, when you get paid, that’s going to happen too and all you have to do is smile and say, “Yay, thank you,” and keep going [LAUGHTER] because you’re getting paid to do the thing.

Kevin: But I’ll just add that there’s no industry group I’ve ever worked on, where there weren’t checkpoints and communication of where things were. And so I know Mike does a lot of these kinds of things as well, which is the idea that what happened when we were undergrads where they basically say, “alright, you’re in a group, go do this, and then deliver the product.” So even in student groups, the importance of having checkpoints, and I basically have them write progress reports like you would do in industry along the way. It’s not “Here’s the project, go do it. Now here’s an end result.” It’s more like “Okay, you’ve gone for a week. What has each person done? What’s your next step?” Those kinds of things.

Mike: And while we’re trying to develop these skills for students going into industry, we think that there… well, I say, and probably Kevin will agree, they’re important no matter where you’re going to go into, including academia or any of the other spots. So I don’t want to make it sound like we’re trying to educate students just to go into a particular engineering job. But this is more of a realistic facet of life in general.

Rebecca: Do you think you both would have found this same path in teaching had you not had the industry experience and kind of underscoring and value some of the same things that you do now?

Mike: Absolutely not. You like what you’ve seen, and so we would have seen “sage on the stage,” and that is what we’d do. And to a point, that’s where we all start, right? I mean, there’s that sage on the stage piece, and then you start saying, “Okay, there’s got to be something better, because I’m not reaching all my students.” So how can I do that with the knowledge that you never will reach all of your students, but you got to give it the best bet you can.

Kevin: I think when I started, one of the big differences was the deference that people showed, it’s like, “Oh, you’re a faculty member, oh…”, and I worry, actually, if I had been 28, and started teaching at that point, that that would have gone to my head, and I would have started to believe my own PR. I don’t see people around me with that problem, but kind of a stereotype of academia. And you do read stories about actual faculty who are like that. But working in industry can give you a good strong sense of humility, because you have successes, and you have failures along the way.

Mike: So I will say in industry, there was still some of that out there. I mean, I used to get emails from my PhD colleagues down in Painted post, who would talk down to me because I was at a plant saying, “Oh, let me explain this to you in very simple terms.” In the very next email, I would just respond. “Well, thank you so much for your guided steps. Dr. Michael Walters, PhD da da da da da.” it was amazing. The next email was much more as an equal instead of I will talk down to you because you are way out there in the hinterlands. Do you know how to rub two sticks together to get fired, because you might need that you’re in the cold part of the country.

John: That reminds me of a time when I was coaching soccer for my sons’ teams. And there was another child who was on the same team three or four times over a space of four or five years. And there was an article that mentioned me in a local newspaper, and this mother came up to me and said, “Well, I knew you worked at the college, but I assumed you were a janitor there.” It was an interesting experience, because she always saw me out there in sweatpants and t-shirts and just assumed “Well, that’s not how a professor looks.” I think maybe that attitude is more common in elite private institutions, Ivy League, and so forth. I don’t think I’ve seen it at Oswego, which is very much like Plattsburgh, in terms of being a four-year comprehensive college.

Kevin: Well, if you wanted to, you could have coached in the tweed jacket with the elbow patches, and then they’d have all known.

Mike: And a pipe, you gotta have the pipe. You don’t have to put anything in the pipe, because that’s bad. But just the pipe itself.

John: I actually had one of those sports jackets. And it was a running joke with some of my students back in the 1980s, [LAUGHTER] because I just had that one sports jacket, and I wore it whenever there were honors, awards, and so forth. And they still ask me about it 35 years later. [LAUGHTER] But you know, one of the things I’m hearing from both of you is that you’re both doing some authentic assessments, which is something that we’re trying to encourage faculty to do in general, to move beyond the traditional types of teaching. How do you find that students react to these types of group projects or the teamwork or other things that you’re having them do?

Kevin: I’ve had, I think, a lot of success with it. I can’t think of any really bad experiences. Mike’s nodding his head. But I have a poster actually coming up at CCSCNE on our tech startup class, that actually, Dr. Del Hart, our department chair, the person who kind of came up with this idea, and then he came to me and he said, “So you’ve worked in industry, and you’ve been a Scoutmaster, how would you like to try leading this class, and it’s a multi-level multi-semester class where students basically can take on different roles based on whether they’re 400, 300, or 200 level in the class, and projects can run for multiple semesters?” And it’s probably one of the most fun classes I’ve ever taught. Because a lot of times I found myself, I don’t teach it now, but I found myself actually telling them “Slow down a little bit. Let’s establish some realistic goals. I know you’re very excited about what you want to do here.” But I think generally, I find the project work goes really well. But I do try really hard to make sure that I’m aware of what they think is going on with their team through various assignments that I collect that are like reflections or statuses.

Mike: I’ve had overall success. There have been times where there’s been failures, one or two semi spectacularly. We do a intermediate robotics lab, which is a project-based group class and at the beginning of the semester I give them what I call a 90% google-able project and say “Okay, from here on out, your group is now doing this for the rest of the semester. If you get done with part A, there is a part B you can have as a stretch goal if you want, but this is what you need to do.” And I try to emphasize that failure still means you can get an A, it just means that you failed at trying to get the project to work. The 10% is the hardest part of the whole project for any of these projects. Working for Corning, I was working on the bleeding edge of technology. And so I was always a horrible person till then in saying how long it’s gonna take to do something because I’ll say “it takes two weeks,” because it looks simple. And at the end of the two weeks, my boss would come back to me, “How’s that going?”” I got the first step kind of done.”” What do you mean kind of done?” “We’re still inventing something brand new, it didn’t happen as fast as I was hoping.” So when the project does work, this 90% thing, it’s great to see the students get excited, they see progress, they keep moving. And then what’s funny is then the cockiness will come out and say we can get this done in that three weeks, I’m like, “Yep, you’re just like me, we’ll see you at the end of the semester.” And sure enough, [LAUGHTER] that’s when it’ll happen, because they’ll run into wall after wall after wall because that last 10% is hard. But I do get sometimes, like this past semester, where the project I picked ended up being a failure. By the end of the semester, it did not work the way it was supposed to work, they could not get the robot to do the thing. And you can tell that towards the end, they were getting really down. I mean, really, really down. And at our final session where we basically talked about what worked and what didn’t work for the project, basically, I told them, “This is what happens probably 25 to 50% of the time in industry. You’re going to work on a project, work on a project, and it’s not going to work.” And there’s times when you got to sit down and say why doesn’t this work? And then go to your boss and say, “Well, you either a didn’t give us enough information, or enough resources or a combination of the two.” And I told them, “I didn’t give them enough resources to make it happen.” But we got to this point. So I pumped them up like, “Here’s the teamwork that you did, this is all great.” And they all got their “A” because every single person contributed and worked towards it. But those failures can sometimes be problematic. On the flip side of it, doing so much non-assessing means I go ahead and give them a lot of homework assignments that once again, are graded basically, “did you hand it in?” Did it kind of do the thing was supposed to do? Yay, you get all the points. And what’s funny is I thought this semester where I cut up all my tests. I have no test this semester whatsoever. Everything is you have homework, classwork, and a project. That’s it. And the beginning of semester, I was nervous. I was like, how am I going to differentiate? Not that I care. If all the students get As I’m happy. But I just want to know if I had to differentiate, how am I going to do the differentiation based on the system? Well, my students have proven to me that they can self select, because they stopped doing assignments. So they stopped doing whatever. And it’s like, “okay, this is how self selection will work.” It’s not that I don’t even come to them, say, “by the way, you haven’t handled this in…” “yeah, I’ll get to it.” We’re human, they’re not going to get to it. And that’s fine, too. But it ends up being almost more psychological of “You didn’t prove to me, not only that you didn’t know the material, but you also showed that you didn’t have the discipline to get that piece done.” And whether that is the focus of my class or not, I don’t know. That’s the negative to this type of assessment. I can’t tell if it’s because they didn’t know the material. So therefore, they didn’t feel comfortable trying to finish the assignment, or whether life happened. And that’s the only negative I’ve seen so far about using this assessment, because you see them not be so stressed, because there’s a test coming up and not be so focused “I’ve got to cram this thing together,” then five minutes later don’t know a single thing when you go talk to me in the hallway about what I put on the piece of paper.

Kevin: It’s funny in that class that I started by actually did something that would be called ungrading. And I used something I’d found in an article years ago, I could find the reference if I had to, but it was how Intel assessed people is managers would come in initially with a sheaf of overhead slides, then they’d have “this person should be promoted.” And you look at it’s here slide 1 of 60 kind of thing. And they went to this thing, I think they call it three up, three down, where you had to give three positive and three negative for the person and then summarize. And I actually had my students in that class for their midterm and their final grade, they had to do those. They had to do a self assessment, they had to tell me three things they were proud of, three things they could have done better or they didn’t do well, what they thought their final grade shouldn’t be and why. That class, though, that was a pretty much a highly selective class, that it was very, very seldom that a student checked out. And part of that was because the person in charge of their project was the 400 level student who they had agreed to work on that project with. And so there was a lot of peer interaction that led to just generally, I think, really good outcomes. Dr. Lecky’s teaching the class now, and from what I hear, he’s having a very similar experience with that course sequence.

John: One of the things we’re doing right now, and I know Kevin, you’ve already been participating in this, is a reading group on Grading for Growth by Robert Talbert and David Clark. And one of the points they make in their book is that, while we’re so used to traditional grading because we grew up with it, and that’s what we’ve always done in some way, one issue is the only time people tend to be graded this way is in academia, when they’re working in industry, they’re reviewed, but they don’t get grades, they either meet the requirements or they don’t, or if they exceed them, there are chances of promotion and so forth. But in pretty much everything else, including our own careers, we get renewed, we get tenure at some point, and there may be some other additional pay increases based on how well you’ve done. But we don’t get letter grades. And we don’t get numeric grades every year. And one of the things I’m noticing is you’re both doing some types of grading that are closer to the types of experience that you’d find in industry. Do you think that made you perhaps more willing to try these types of things that a lot of faculty have been really reluctant to experiment with?

Mike: I think so. When we talk about the industry thing it brought back a story, I one time was in a review, and I was rather well paid at this plant. And so they had high expectations for the salary they were paying me. I remember, I was talking to the plant managing engineer and they even brought the plant manager in during my review. So either this was gonna be a really good thing or a really bad thing. When I walk into the room. It’s like, okay, which way is this gonna go? I thought it was a bad thing. But what’s funny was they gave me a “met expectations.” But the plant manager turned to me he and he goes “I expected you to “exceed expectations.” I’m like, “Well, how am I supposed to exceed expectations? By default, you’re saying that is your expectation. So therefore, I either met the expectation or I didn’t.” And we got into a heated verbal argument for a little while. And eventually, I said, “Look, you either give me met expectations, or I didn’t meet expectations, I don’t care. But what you just said makes no sense.” And she threw up her hands and left, and I got “met expectations.” But it was just one of those things. It’s like even using these types of methodologies, sometimes you get conflicting viewpoints that goes forward. As to what your question asks, yeah, I think with the industry, this is why I’m gravitated towards that. I teach a class called fundamentals of engineering design. And when I tell the students why this class is there, I said, it’s everything I wish I knew as an engineer, before I became one. I was a physicist, I didn’t go through any of these preparatory courses, and then boom, here’s an engineering role, go run with it. And so being able to help my students be in a better situation when they go out into industry, which I assume most of my students are going to do, is a good thing. And any of the skills they pick up because of this isn’t bad if they stay in academia. Maybe this helps break the cycle of, we only can test and do homework for grades. By the way, I’m just gonna throw out my pet peeve for all of you out there listening. My biggest pet peeve is if you grade homework like a test, and count it, where are the students supposed to make the mistakes? That’s the thing that always used to tick me off. Where do I practice and get legitimate feedback on my practice, to know where I’m making a mistake such that when I go ahead and then perform, I know I’m performing at a higher level. That part always bothered me when I was an undergraduate. That was from day one, when I started adjuncting, I said, I’m not going to do that. Homework is going to be: did you try it or did you not try it, period, That’s it.

Kevin: I’m gonna make just one little thing. IBM really, really tried very hard for a long time to just grade people. We had a system where you had an appraisal or a personal business commitment. And at the end of the year, your manager basically gave you a score from one to five. The system changed many times over my years, but: one was, you’re on track, we’re going to promote you, you’re doing awesome, and a five was it was nice knowing you, but you generally knew where you were going to fall. And I won’t get into all of the arguments over the years as to whether there were ratios or stacked ranking, and at what level in the organization, there was a quota on how many could be at each thing. Actually, those kinds of systems caused a lot of the negativity that made me think that maybe I wanted to go to teach. I think, for me, one of the really big things has been, as soon as I started teaching, I started trying to learn how to teach, because I walked in the door, I adjuncted for one class at the University of Vermont, it was a graduate-level course and I really enjoyed it, and it was great. But I realized how much I didn’t know about teaching. I had my teaching statement when I applied for the job, actually leaned really, really heavily on stuff I’d learned as a boy scout leader, and then like a leadership course called wood badge, a six day leadership program. And I had learned a lot of what was basically active learning as part of that program, something called the edge method: Explain, Demonstrate, Guide, Enable sort of thing. And so I had kind of this foundation that came from a very non-traditional place in terms of pedagogy. And I came in and I started out doing exactly what I had experienced, which was you stand up, you lecture, you use PowerPoint, it’s really cool, you give homeworks but at the same time I started reading stuff and a lot of it was James Lang’s books. I think I’ve read all of his books, because the first one I read was On Course, and I think that they should give every new faculty member a copy of that book during orientation, because it got me through my first year. I have a copy in my office that I think has got a thousand post it notes stuck in it. And then I started reading his other books. And then I started reading the books he was talking about. And these days, I love to teach my classes in our programming lab, because when I can, it takes more prep time, but it makes actually the class easier for me in a lot of ways to do a little bit of lecturing and then say, “Okay, now you get the computer in front of you, go make JavaScript to do this.” And I’m going to walk around the room and help you do it. And so you’re kind of turning even lecture sections into labs, in that sense. And I find that’s a lot of fun for me. And I think the students find it more interesting, because some of them kind of struggle. But the thing is, there’s me, and there’s the other students around them. Like when I give quizzes, now I hand out a quiz, I say, “Take about 10 minutes and work on this.” And I say, “Okay, now compare your notes with your neighbors.” And then I say, “alright, let’s discuss it.” And at the end, I collect it, and basically say, “There you go, you get credit for being here, because you did this.” And I have a couple of students and Mike would know their names, I won’t mention them, they sit there, and when I give him one of these things, they’re way beyond anything I’ve even envisioned, they’ve gone off. But other students, I have to sit with them and help them. And I just find that’s a lot more satisfying for me as a teacher than to stand at a whiteboard and scribble on it for 50 minutes, or mine are all an hour and 15 minutes this way I set my schedule up.

Mike: I’ll agree with that, and I’m actually moving more and more of my courses over to, I will give you the bare minimum that you need to do what I’m about to ask you to do, and then go do this thing. And practice it there. And I’ll walk around, I’ll help out, we’ll talk to everybody, make it much more of a laboratory class, even though it’s not a laboratory class, just because the interaction between their peers, their interaction with me, becomes more of a one on one instead of one on many. Way back when, I used to start off my class, when I would teach introductory physics with saying, “You could learn from the book everything I’m about to show you, and that’d be great. And if you never show up and just show up for the test, you could do that. Awesome, congratulations, we’ll see these dates and have a great time. I know. But if you come here and just listen to me, that gives you the book and me. But here’s the more important thing, look around the room, there’s 40 other people here. So that’s me, 39 other people and you. All of a sudden you have lots of teachers that could work with you. And as long as I help facilitate you being able to use that resource, you have many, many more options, because I might not have the analogy you need. But your friend next to you or the one over two rows from you might have that analogy you need to make that connection.”

John: Could you each share some of your favorite teaching activities that you use in your classes?

Mike: I’m trying to teach them brainstorming methods, especially technical brainstorming. I have 1 2 3 4 5 6 whiteboards in my room. And I’m going to ask in front of each whiteboard, “I want three technologies, just random three technologies.” I’ll keep writing them down without telling them what’s happening, three technologies on every whiteboard. And then I break them up into groups and give them each one color markers. They always use their color marker and say, “Okay, now the six groups go to each of your boards, you now have two minutes to write every single technology product you can come up with, that’s a combination of some sort, matter, or form of those three technologies.” And sometimes like it could be weird stuff like ice cream, rocketry, and such and such, and I’m like, “Yep, now you got to figure out how you’re going to combine those three.” And I tell them, “Okay, so if you’re the first person on the board, you’re gonna put a one next to your thing, you only get one point for every one you come up with. Then you’re gonna draw a line and the next group comes up, they get two points per thing they come up with, they can’t come up with something that you already have, all the way up to that final group, where now five of the groups that brainstormed, but everything you come up with now is worth six points.” So it’s like you get one or two things you just made up for your first board and second board without a problem. But it really shows them, sometimes those six pointers at the end, when you’re really just grasping at straws, become the things that you want to move forward on. It goes to writing even. So I know when I do my writing, it’s like that first draft, you did all the easy stuff. Now throw out all the easy stuff, and I’ll start putting the second and third order stuff that might make it much more interesting.

Rebecca: That’s a great exercise. It’s really hard to get students to get to that further step. They want to stop at step one or two.

Mike: Oh exactly, and making it a competition of which team can get the most points, get some of them, their juices flowing. And it’ll be funny because at the end, we go through everybody’s stuff like “is this a legitimate idea?” No, cross it off. Then it doesn’t count. That’s a group response of rating, basically, of the thing. And it’s funny because at the end, most of the time, I can’t get through all six boards by the end, and they’ll be sitting there going: “We got to finish this. I want to know who won.”

Kevin: …favorite teaching activities. I really do like and I do it a lot now, which is to give the students little coding challenges in C++ or JavaScript class, which is to basically hand out something that looks like a quiz, but it’s basically: here’s some coding, let’s do do this simple coding thing in class, because if I handed out as a homework, well, they could try using ChatGPT. Although I have to at some point talk with Mike about the fact that one of my students tried that and ChatGPT hallucinated a library for circuit Python that doesn’t exist. And as I said, on social media to my friends, I’m happy to set up a PayPal account, if ChatGPT wants to hire me as a consultant to debug their program’s mistakes, because the poor student had no idea what to do, because the code he was given was incomprehensible. But I really like giving the students an exercise to do in class. The second place I would say, was, when I taught our ethics course, I did a lot of debates in class. And sometimes I would let them self select what side they wanted to be on the issue like drone warfare things, other times, I would kind of assign them. And when they balk at that, I’d kind of lean hard into my own experience and say, “Well, you know, I debated in high school, and you didn’t get to choose what side you were on. And it helps sharpen your brain to argue something that you don’t necessarily agree with, to try to logically follow it.” I would say for in classroom, those would be for out of classroom, like project kind of things, the posters, having students do a poster session, at the end of the semester, actually printing the posters, and putting them up in the lobby of our building. And then having kind of a little session where people can walk around, just like a conference poster session.

Rebecca: it’s really great to hear the different ways that both of you have incorporated your industry experience into the classroom from changing grading systems to thinking about consistent feedback throughout the semester so that students can learn and always being on that verge of not quite knowing what’s coming next, like you are often when you’re in a tech field and having to constantly learn or have professional development or try things that are brand new to you. But we always wrap up by asking: “what’s next?”

Kevin: I am, as the phrase goes, playing with house money. I was at IBM for 32 years, my wife still works there. I continue to do this because I really enjoy it. And I love all three aspects of the job in academia, I love the research, which we do research, I think I would say with a small r,, not like R1 with a capital R. I love the research, I really liked the service. All that time I did at IBM, I can do committees like you wouldn’t believe. I know how to do that kind of stuff. And I know how to be helpful in that way. And I love the teaching. I mean, I’m teaching an honors seminar on algorithmic bias this fall that I’ve taught once before. I really enjoy teaching a new class every once in a while or creating a class. It’s a very intellectually stimulating field. And I’ll just do it until my wife and I decide it’s time to be able to go to Disney World in September or something. And my guess is that a few years down the road, I’ll probably retire. And I will continue to adjunct because the great thing about adjuncting is you can keep an email address and you can also still get discounts on software. [LAUGHTER] I kid you not. It’s kind of like “Hey, I’ve got all of the Adobe tools and I was playing with Dreamweaver for my JavaScript class and generative AI creating images the other day.” I’m so curiosity driven that continuing to teach as long as I’m enjoying it.

Mike: My “what’s next” is to try to become a better evangelizer for the robotics major here at Plattsburgh. We’ve been growing it slowly, and I firmly believe in what opportunities it allows people to have it going towards the future. I run a not for profit here in Plattsburgh that works with middle schoolers and high schoolers to give them robotics opportunities on Saturdays. I do this major, I’m chair just so I can make sure I have the time to devote to doing things. It is fun, starting something and watching it grow up. And I’m just having a ball with it. All the stresses and anxiety that come with that as well. But yeah, that’s my what’s next.

John: And going back to Kevin’s comment, before you make the decision to go to Disney World in September. If you start teaching online or doing some more work teaching online, there is the Online Learning Consortium, which has a conference at Disney World in November, which is a really great time to leave the Northeast. And it’s a four-day conference, but you get the special conference rates at the hotel for a couple of days before and a couple of days after. So you might be able to combine those things.

Kevin: I will have to look into that because the best conference I ever went to is Design Automation Conference, it was held at the San Diego Convention Center and I went to that a couple times for IBM for CAD software and it’s one of those 6000-person trade show conferences, top end technical conference and on the trade show floor, everybody’s collecting polo shirts and desk-y gadgets, that kind of thing. It can be a lot of fun. And I have stories about that that are actually very funny as well, because when you’re at one of those and the part of the company you work for actually competes with many of the companies you’re visiting, eventually they kind of figure out that you’re asking questions that an ordinary customer would not ask. [LAUGHTER] And they’re kind of like, what exactly do you do for them?

Rebecca: Well, thank you so much for joining us. It was great to get a chance to talk to you in more depth.

Kevin: This was fun.

Mike: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

338. Diversifying the Education Pipeline

Posted in access, classroom community, diversity, engagement, equity, evidence-based practices, feedback, first-generation, graduate training, higher education, inclusion, instructional communication, learning, pedagogy, professional development, social justice, teacher training, teaching
April 24, 2024

Diversifying various fields and disciplines requires intentional work to create and support a pipeline of practitioners. In this episode, Laura Spenceley joins us to discuss specific initiatives to increase inclusion in the PK-12 sector. Laura is the Dean of the School of Education here at SUNY Oswego. She is an Impact Academy Fellow through the national non-profit organization Deans for Impact which works to strengthen and diversify the educator workforce.

Transcript

John: Diversifying various fields and disciplines requires intentional work to create and support a pipeline of practitioners. In this episode, we talk about specific initiatives to increase inclusion in the PK-12 sector

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Laura Spenceley. Laura is the Dean of the School of Education here at SUNY Oswego. She is an Impact Academy Fellow through the national non-profit organization Deans for Impact which works to strengthen and diversify the educator workforce. Welcome, Laura.

Laura: So great to be with you two today. I’m really excited for our conversation.

John: It’s overdue. We’re really happy to be talking to you today. And we’re talking to you from a new room which the School of Education has graciously loaned to us because we lost our old recording space, and we’re now able to keep the podcast going. So thank you.

Laura: You’re most welcome. And truly, I can’t think of a more symbiotic relationship between Tea for Teaching, the opportunity to host you here in the School of Education, and continue this great work and digging into pedagogy issues across the education continuum. And having you right here in-house makes a lot of sense.

John: Our teas today are:… Laura, are you drinking any tea?

Laura: I am not a tea drinker. But I do have a robust French roast in my mug today.

Rebecca: A popular flavor of tea, as we’ve noted multiple times here on the podcast. I have Hunan Jig today, John, I thought it was a good celebratory sounding tea.

John: And I have a spring cherry green tea in a brand new School of Education mug, which I just received today. Thank you.

Laura: You’re welcome.

Rebecca: In your role as the Dean of the School of Education, you’re involved in the entire P-20 pathway. The School of Education has received two large grants to help support students pursuing careers in education. Could you tell us a little bit about these grants?

Laura: I’m thrilled to talk more about these opportunities. And these two projects are our teacher opportunity core grant, that’s our TOC grant and our Cultivating Representation in School Psychology grant. Both of these grants are, as their name may imply, designed to help us recruit, support, retain and move through into careers, teachers and school psychologists from a wide range of backgrounds. It’s a well known fact across most educational spaces that folks who are delivering instruction or providing support services don’t always represent the constituent student populations with whom they work. And each of these programs are designed to 1. help ensure that we have a representative and diverse education workforce. But more importantly than that, they’re also designed to ensure that as we recruit and matriculate students into our teacher preparation programs and school psychology program, that we’re investing in those candidates along the way, recognizing that students from historically marginalized groups are often more likely to have challenges in persisting through their academic programs, both undergraduate and graduate. So each of these programs has some similar foundational elements. That includes financial support along the continuum of their educational program, again, undergraduate or graduate, support for things like transportation, professional development opportunities. And those professional development opportunities are really designed to empower our students from those historically marginalized or underrepresented spaces in their fields with contemporary activities that will help them address systemic discrepancies in schools. So making classrooms more responsive to the needs of the multicultural communities in which they reside, helping ensure that our future psychologists and educators have opportunities to work with people along the way that look like them, that share their lived experiences, and build that really continuum of care around their success. It’s also a great opportunity for us as educators, leaders, teachers, to be invested in those spaces with them and hearing from their mouths, what their challenges are, what their opportunities are in districts and really working to empower them for a career in education.

John: And the School of Education has always been engaged in working with students from all these marginalized groups that you’ve mentioned. What other types of support, though, have you been working on developing in the School of Ed for students who are first-gen students, Pell-eligible students, or from historically minoritized groups?

Laura: It’s a great question, John, and I’m so thankful to have the opportunity to chat about it because I think sometimes, as is human nature, we want to create responses or programming from a perspective of “all first gen-students might need this thing” or “all Pell-eligible students might need this thing” and we’ve been certainly sensitive to the realities that those populations experience, but also have tried to take a step back and take a more nuanced approach to that work. I’ve had the opportunity to be in the Dean’s position for about three years now. And when I came in the door, it was the year after the global pandemic. And so, so much of what we knew here on a college campus and in our P-12 spaces in the community had just been turned on its head. So when I came into the dean’s office, one of the things that was really important was to hear from our students, our candidates pursuing their degrees, what are the challenges that you’re experiencing? And I need to give a shout out to Dr. Nicole Brown, our director of our clinical practice and partnerships office, she really had this vision of surveying students, engaging in some listening sessions with those students. And we took the opportunity to deploy a survey about how students were experiencing their programs, but also the challenges that they were experiencing. And those data were humbling, to say the least. We learned back in 2021-22, about one in five of our students was having to make the choice between eating and paying for gas to get to their placements. About one in four of our students had to go without a meal at a certain point so that they would be able to engage. We recognize, and this was not a new problem, but just how challenging transportation to and from their clinical sites was becoming particularly in a time where we were very sensitive to being in enclosed spaces. And so we took that opportunity to reflect on the ways we weren’t just to use quotes, “utilizing the resources,” like our teacher opportunity core program, but to really recalibrate the way we were engaging with all students. And so what that looks like right now is we’ve really taken an approach to engage with our students from the time they’re prospective students all the way through graduation and career placement. And so what that has looked like is we’ve had active faculty and administrator presence at recruitment events, helping prospective students and their families understand not just the phenomenal financial aid resources available to them, but demystifying grant programs like the Teach Grant program, like public service loan forgiveness, which is available to many of our graduates after a period of time, it’s also about helping those families understand what contemporary classrooms and contemporary educational careers look like. There’s so much animus in public media and in the dialogue around what’s going on in schools and giving our prospective students the opportunity to hear from our current students and our partners, what they can actually expect, not just out of their degree program, but the field, has been really impactful. On a more nuanced level, as a result of that survey, we have utilized a range of resources to help offset some of those specific concerns regarding transportation, costs related to certification, and food instability. I do need to call out our amazing student affairs team on campus that has made enormous investments in this space as well. And I think always connecting students to the most local resources has been a really effective way to ensure they can take advantage of those supports in a way that makes sense to them. For us, I was lucky enough to inherit a 12-passenger van that had been purchased right before the pandemic, and we’ve taken the opportunity to hire student drivers. We offer daily routes to and from the Syracuse area at no cost to our students, leveraging the generosity of our alumni and donors who have invested in the School of Education, we have made a commitment for any candidate that needs to get to or from their site and can’t use the van, that we’ve created an Uber voucher system. And so we’ve been able to put those funds directly in front of students. We’ve expanded our supports for certification exams and fingerprinting through really simply asking students: “Are these barriers?” and creating a process again, through our foundation funds, that we’ve been able to offset those costs in time of need. And then lastly, I think another part of this is helping our faculty understand the realities that our candidates are experiencing as they move through their degree programs. Bringing those data back from not just the survey but listening sessions we continue to offer as an open forum for students to share their concerns, bringing that back to our faculty and helping them understand in both a formative and accurate sort of way, what the experiences of our students are in looking to make changes, whether it’s to our core sequences, the way that we administer fees, encouraging open-access resources, building resource libraries within our departments, so that we can continue to wrap our arms around students, give them a great experience, and also ensure that something like a meal isn’t getting in the way of their ability to get to and from their site.

John: And one thing we should note is that Syracuse is about 40 miles or so from here. So the commute time for a student would probably be about 45 minutes to 55 minutes, depending on where the school is located. So transportation would be relatively expensive for students.

Laura: Yes, and it’s such a good point to make, and I know we’re likely to talk about partnerships later, being where we are here in Oswego, we are lucky enough to have amazing partners along the I-81 corridor from Syracuse to Oswego, almost to Utica, almost to Rochester. And so it is not unusual that our candidates are driving up to an hour each way. So absolutely.

Rebecca: And it’s a problem that campuses that are in a more rural context experience when they’re doing any kind of placements, whether it’s internships, student teaching, or service learning, or anything like that. It’s very different than an urban context.

Laura: Absolutely. And I think we’re really committed, we embrace the idea that we are a rural educator preparation program, while also embracing the fact we have large urban partners just down the road. And we see that as an opportunity to diversify our candidates’ experience and give them a broad sampling of school structures, organizations, what’s possible, both in large districts, small and the in-betweens. And so that’s been a really critical part of our preparation framework for many, many years here in the School of Education.

Rebecca: It’s always great to hear stories, and we’ve heard more of these since COVID, of schools really paying attention to the actual needs of students and what barriers they’re facing, and really being responsive to them and finding ways to solve the barriers that they’re facing. Because often, the barriers are great, but often there are solutions that we can put into place that aren’t actually that complicated when we get down to what they are.

Laura: Rebecca, I love so much how you framed that, because one of the driving mantras that I often remind myself is we can solve problems if we choose to. And I think to solve problems, we need to be able to describe the problem, we need to understand it from a compassionate perspective. Recognizing every candidate that declares a School of Ed major sees themselves as a future educator in one form or another, and partnering with them, to actualize that vision is the best part of my job.

Rebecca: Well, and we would hope they want to do the same thing for their students. So modeling what we’re hoping to bring into the land of education is certainly a good place to be. We talked before COVID, with the School of Education about some of the really great partnerships you all have both locally and regionally, both as school districts and community organizations. Can you talk a little bit more about some of the outreach you’re continuing to do in those spaces?

Laura: Absolutely. And frankly, we would not be a school of education without partners. Every program housed within the School of Ed requires an applied learning experience. And so even our non-teacher preparation programs in counseling psychology and management programs require some sort of applied experience. And so strong relationships and a strong understanding of the context for our partners is really important to our success and our candidates’ success. So I’ll talk about a few examples since there are many day to day here in the School of Education. For context, in 2020-23, for example, we placed more than 1000 candidates across I believe 246 different settings, kindergarten all the way up through high school. Don’t quote me on the number, but I think I’m close. But we have made a couple of purposeful efforts over the last few years. And in particular around our professional development schools. These are schools where we have an agreement to focus professional development activities for our pre-service teachers, in-service teachers, so folks who are classroom teachers in that district, and align that with our faculty areas of expertise to create what are called that third space for learning, where we were bringing in a range of expertise and perspectives. Here in Oswego County, we have professional development school partnerships with six of our nine local districts. And those partnerships have focused on strengthening support for technology and STEM-oriented initiatives at the junior high and high school level. In another district, it’s included a dedicated focus on early elementary school literacy outcomes and improving the way we’re preparing kids to read and write. Other districts have focused on more systems-level change with strengthening the way that we identify students who are struggling before that becomes a significant impediment to their learning, and organizing teacher and leader responses to ensure that those processes are unbiased, that we’re making decisions that help us put resources around what kids need to thrive. But that’s just a sampling of the types of partnerships that we have with local districts. I think, given the fact that education continues to be a domain where the public dialogue is not always framed as positive, partnerships with our local practitioners and districts, give us opportunities to 1. change that narrative and help people in our communities understand the value that schools can bring to that community. And Rebecca, you mentioned the way that, coming through the pandemic, schools are really regarded as kind of community centers. And I think we’ve seen those types of initiatives be really effective, whether that is including opportunities for families to access medical care, dental care, social services and supports when they’re in district, we have something to offer to that conversation because of the range of our preparation programs here on campus. And so we really tried to engage with our partners in the P-12 sector to ensure that we are operating as a true continuum of care for our communities.

John: During the pandemic, there were some fairly substantial learning losses that disproportionately affected students from low-income households, and low-income school districts. And a study was just released by WICHE in February of 2024, which found that those learning losses are still persisting, which means the students who are teaching in schools are going to face students with a fairly wide variance in their prior preparation. What types of things do you do to help prepare students to address the diversity of student backgrounds in terms of how much they come into the classes with?

Laura: Coming through the pandemic, one of the things we were reminded of was the power of a direct positive relationship between a student and an adult in the building, that can look like a wide range of folks, coaches, teachers, mentors, administrators, volunteers. But that’s something we have continued to embed in our curriculum and helping our candidates understand that day to day, the way that we make students feel is often the thing that will be really critical and helping ensure that they walk through the door the next morning, the next day, or ask for help when they’re having a tough situation. And so empowering our students to respect the dignity of people’s lived experiences, we embed before they move into their clinical experiences in schools, quite a bit of reflection, self-reflection on their own lived experiences, what education has meant for them, the spaces where they have been privileged or perhaps oppressed through educational structures, protocols or policies, and building that capacity of our students to both identify and address bias or embedded bias in the spaces where they work. Getting down into the curricular focus, it is frankly staggering to see how children have been impacted, particularly in the early elementary levels and the older adolescence ages when we think about both academic skills, but socioemotional skills, wellbeing, and I think one of the things that I try to celebrate is the resilience that’s been built into this generation of learners. On the other hand, that comes with a certain amount of exhaustion, anxiety, uncertainty about their future, and so coming back to that element of relationship building, it’s so key. At the curricular level, there is an enormous focus here in New York on the ways that we can do a better job in preparing children to be literate, contributing members of a just society. So right now in New York, there’s a lot of focus on the science of reading. And I want to be really clear to listeners, the science of reading isn’t about a curriculum, it is about embracing the empirical evidence towards a systemic approach to the instruction of reading, writing, and literacy across grades. There’s an enormous amount of energy around this work right now recognizing that we can do better in preparing kids to be literate. And I’m starting to see an equal attention to a focus on math and frankly, STEM preparation. Here in our community in central New York industry has returned with a bang. And there are wonderful opportunities for employment for students with a wide range of educational experiences starting right out of high school for gainful and lifelong employment in these sectors. But what we’ve heard is that math proficiency is often a barrier for those students. And so trying to incorporate elements of curiosity and creativity, bringing science, technology, and math into creative spaces, like art, design, dance, helping students understand that math and science, quite frankly, isn’t just about memorizing equations, or knowing your multiplication tables. But there are a range of application of math, science in our day-to-day lives. When I think about when I was a kid, we had a great lesson on fractals, and we went outside and we found fractals in nature. I remember it to this day, almost like it was yesterday, trying to engage students in their learning, that incorporates their lived experiences, having access to textbooks, readings that reflect their family structures, their linguistic background, the realities of their lives. And so our students ,when they move into those first teaching positions, carrying with them that, I’ll go back to respect for the dignity of all persons, that recognition that kids carry much more with them into a classroom than a backpack, and finding ways that we can empower their learning not just through curriculum, but through curriculum that is delivered at their level, that celebrates individual students growth, that isn’t anchored in some specific outcome, but recognizes individual student’ s growth as meaningful. And I’m really proud to say our data suggests our students do move into their first teaching careers with those skills. And I think the next phase of the conversation starts to focus on how can we, as an educator preparation program, be part of the community in delivering professional development? How can we bring the great work that’s going on in schools to our faculty and have that kind of recursive flow of information, and also ensuring that we are preparing school leaders to create conditions that help us achieve all of these goals. And with our programming here in the School of Education, we’re uniquely positioned because we offer those range of preparation programs and have the luxury of great relationships with partners.

Rebecca: We’ve talked a little bit about learning losses in K-12. But obviously, that has an impact on our college students as well. They also survived the pandemic and are here. What are some of the things that your faculty are doing to help these candidates feel like they have what they need to go into the community having maybe had some experiences of remote learning or challenges with some of these subjects too?

Laura: This is a really important question. And I think it’s one I’ll both answer from the school of education perspective, but also the broader campus perspective, frankly, because we don’t do this work in a vacuum or in a silo. And so I’ll pick up on some of the elements of the last answer, which is making sure that we on campus are framing the opportunity to serve our students as a positive one. We have an opportunity to meet students where they are and that will look different for different students. On the other hand, I think our faculty have been eager to try to crack this code, so to speak, on bridging our lived experiences in K through 12 and transition to college through the framework of the contemporary student experience and those are not parallel for many of us or really even comparable. I have been trying to, as dean of the school, make sure that we are approaching this from a perspective of strength. What students do come to us with us is a resilience that students of prior generations may not have. They come to us with skills and knowledge of technology and how to leverage it in a way that students didn’t in the past. And frankly, it was because they had to learn how to use that technology. Students coming to our campus today have access to more information at their fingertips than any generation of student has ever had in the past. And where we have opportunities is to help students both identify where they may have gaps, whether in their knowledge or experience, put those experiences in front of our candidates, and also help them navigate this world where it’s not always easy to identify what is good information and bad information. Now, let’s drill into some more specific examples there. I think one of the most critical things that we have done in the last few years and again, I’ll give kudos where they’re due to our student orientation groups. We have been embedding ourselves into some of those opening week sessions, where we’ve had the opportunity to talk to our future majors and undeclared students around college- level expectations. And while that might sound silly, I think when students can hear from us, from me, from my faculty colleagues, our stories that many of us are first-gen, were Pell-eligible, have student loans, worked while we were in college, struggled to purchase textbooks, come from different backgrounds, are new to the state or the country. I think that goes a long way as a formative first step in building that trust in the relationship with our candidates. It also gives us the opportunity to begin to plant those seeds around what supports are available to them. Whether that is to help them address their mental and physical health needs, their academic needs through our tutoring center, our office of learning services, the embedded tutoring services, and also come to see us as people that are here to help support them and what their vision for their success looks like. I had a young person come up at the end of our first orientation and said, “I’ve never had someone like you say, you walked in the same path as I have.” And it was really a moment where I understood that coming to the table and being authentic with our students is so critical at this point. The other piece of it, John and Rebecca, is recalibrating the way that we orient our classrooms to continue to give students frameworks for utilizing technology and AI and the wide range of information, also helping them become critical consumers of that information, providing formative feedback to their assignments in real time, flipping our classrooms and allowing students to lead the conversation. I think providing choice as much as possible around what literature students choose to present in classes, for example, have been ways that we have said to students, both overtly and more discreetly, you matter, your story matters and you and my classroom matters. I’ll also share we have been taking a look at the way we offer our curriculum to try to identify, and again as part of a larger campus focus on retention and success for our first-year students, where are the pain points, so to speak, in their degree program, particularly for transfer students, we’ve heard that there isn’t always as seamless of a transition to campus as we may assume. And so really starting to tackle some changes in our course sequence to ensure that our academic programs are set up in a structured sequential way that also gives enough flexibility for students to pursue some of their areas of interest, be student athletes ,be club leaders, or frankly, support their family, which many of our students need to do.

John: Recently, we’ve seen a lot of attacks on education at all levels, with books being banned in the classroom, removed from libraries, and restrictions on the types of topics and the types of history that can be taught in classes. What are some of the things that you do in the School of Education to help prepare students for a world in which they may be facing more challenges from parents and political pressures?

Laura: This is a critically important question and will be critically important action. I hope I could say in the next year, but I have a feeling this will be a long road. Too often I hear colleagues, neighbors, friends say, “Well, Laura, at least we’re in New York, you don’t have to deal with this in the same way.” And while it is true, I am proud to be in a state that has been very overt about people’s lived experience, respecting the civil rights of all persons here in the state. It is also the case that that can change very quickly in an election cycle. And frankly, we have seen that across the country. I’m particularly concerned right now about the way that civil rights discussions have been framed. And there are some cases in litigation right now around bullying initiatives and carving out exclusions or exceptions in those bullying protocols or policies that would allow, for example, trans students to be targeted without recourse. We see that replicated in both sports spaces, but also classroom spaces. I’m deeply troubled that this is a place that we’re going in a country that was frankly built upon public education and access. And we haven’t always gotten that access piece right. Many of our school policies and procedures have been built on an exclusionary clause rather than an inclusionary mission. On the other hand, we’ve made such great gains over the last 10 to 20 years in ensuring that schools are more responsive, supportive, dignifying places. I’m really troubled by this work. For our students, we have a commitment to social justice embedded in the framework of the School of Education. And that is more than transactional or it goes beyond the words social justice. Our curriculum prior to students going into schools is focused on those ways that schools have been exclusive spaces, have not always dignified the realities of the people who walk through the doors. And so for our students, what we tried to do is help them understand the threats or attacks on people’s civil rights or their histories is not new. On the other hand, it’s taken a very different tone in the way that it’s being implemented. So 1., it’s giving them the tools in knowledge and understanding of history to understand schools don’t have to be exclusive places, schools can be responsive, welcoming, encouraging spaces for all students from all backgrounds. It’s also about helping students understand that I want to be a teacher, I don’t want to be a politician is not a reality in which they will live as an educator. We try to empower them without encouraging them to engage in any particular behavior, to stay attuned to the realities of how legislative efforts, school board votes, and other sorts of community oriented book bans and dialogues might impact their ability to teach a particular lesson, utilize a particular academic resource. So you may have seen coming out of Florida, among other things in the last few weeks, a school district teacher had thought that it was prudent to send home a permission slip to alert the parents, that their children were going to be exposed to a black author. It wasn’t about the content of the book, it was simply the fact that they were going to be reading a book from an African- American author. I can’t think of a better example to highlight the importance that an apolitical stance is not one that classroom teachers can take, that school leaders can take. And that’s not about voting for a particular person or a particular party. But it is standing up to ensure as educational experts who have preparation, expertise, and insights that the community may not, to share them and be vocal about the ways that diverse learning materials, accurate historical representations of the world are critical towards ensuring that we do not replicate them. And so I am both incredibly disheartened at the turn that this discourse has taken. I’m also incredibly optimistic when I look at how young people, how educators, how experts have shown up and linked arms to say: “This is not right. We are not going to correct the wrongs of our history by limiting access to that history or by building smaller tables with fewer voices.” And so I tried to stay in that headspace as much as possible.

Rebecca: Definitely some good reminders and things for all of us to be thinking about in our role as educators. So we always ramp up by asking what’s next?

Laura: I love that question, because I could probably read you a laundry list of the things that we’re thinking about in our School of Education. Right now, here in New York, there’s a lot of robust conversation around, what does a high-quality teacher look like? What are they ready to do in their classrooms? And there’s a lot of engagement there. And again, this relates to helping the public understand not just what we do day to day or what the outcomes are, but the opportunity space towards preparing children for lifetimes of success. Here in the School of Education, we’ve got a range of new programs that we’re lifting in the fall of 2024, including our new master’s degree in behavioral health and wellness. And frankly, I think we have a range of new programs. As we talked about industry thriving here in central New York. There’s also a lot of energy in the school of education around our career and technical educator preparation programs, and of course, our technology education programs, some great investments in our lab spaces, looking at opportunities to do more in partnership in and with local districts to ensure, as we were talking about kids across Oswego County, have great opportunities to engage with science and technology and math in ways that bring them out of the classroom and into the world and creative spaces. So I’m really excited to see what the year ahead is going to look like for us.

Rebecca: Well, thank you so much for joining us, Dean Spenceley and for our wonderful space.

John: …and we’ll be talking to you much more in the future.

Laura: I look forward to the opportunity. Thank you both.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

333. High Structure STEM Classes

Posted in active learning, authentic learning, clickers, edtech, equity, evidence-based practices, higher education, inclusion, learning, pedagogy, scholarship of teaching and learning, self-regulated learning, STEM, teaching, two-stage exam
March 20, 2024

Multiple studies have found that increasing course structure reduces equity gaps and provides benefits to all students. In this episode, Justin Shaffer joins us to discuss several ways to increase structure in STEM classes.

Justin is the Associate Dean for Undergraduate Studies and a Teaching Professor in Chemical and Biological Engineering and in Quantitative Biosciences and Engineering at the Colorado School of Mines. He has taught a variety of both small and large STEM classes in multiple modalities using evidence-based approaches and has won multiple teaching awards as a result of this work. Justin is also an active researcher with 16 peer-reviewed publications and serves as the editor for four STEM education journals. He is the author of a forthcoming book on high-structure course design coming in late 2024 or early 2025 from Macmillan.

Show Notes

Transcript

John: Multiple studies have found that increasing course structure reduces equity gaps and provides benefits to all students. In this episode, we examine several ways to increase structure in STEM classes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca:Our guest today is Justin Shaffer. Justin is the Associate Dean for Undergraduate Studies and a Teaching Professor in Chemical and Biological Engineering and in Quantitative Biosciences and Engineering at the Colorado School of Mines. He has taught a variety of both small and large STEM classes in multiple modalities using evidence-based approaches and has won multiple teaching awards as a result of this work. Justin is also an active researcher with 16 peer-reviewed publications and serves as the editor for four STEM education journals. He is the author of a forthcoming book on high-structure course design coming in late 2024 or early 2025 from Macmillan. Welcome, Justin.

Justin: Thanks for having me. I’m really excited to talk to you today.

John: We met at the POD conference back in November, and we talked about doing this but we finally got around to actually scheduling it. So thank you for joining us.

Justin: Thank you, John. Yeah, I saw you sitting there at a table by yourself. I knew your face from LinkedIn and I built up the courage to introduce myself and I’m glad I did.

John: Today’s teas are:… Justin, are you drinking any tea?

Justin: I am drinking some tea. I got prepared. It’s from a small little coffee/tea shop in West Middlesex, Pennsylvania. I’m from the near that area in western Pennsylvania. We call it the Shenango Valley and it’s called O’Neill’s and they make lovely Western PA coffees and teas and I just have some nice black tea here. It is that time of day for me that I need some.

Rebecca: Today I have a jasmine green tea, John.

John: Very good, and today I have a Prince of Wales tea, and a peppermint tea because I didn’t have very much to drink today, and my throat was getting really dry in my econometrics class. So I wanted to be well hydrated. [LAUGHTER]

Rebecca: Lots of variety, too. So we’ve invited you here today, Justin, to talk about how you’ve implemented high-structure course design. And for listeners who might be new to the concept, can you talk a little bit about what you mean by high-structure course design?

Justin: Yeah, so just like I’ll tell you all like I tell my students, I didn’t come up with this term. I wish I did. But I’m building off the work of other giants in the field. So as far as I know, the term really came out in the mid-2000s, 5-, 6-, 7-ish range from the University of Washington in Seattle. There’s some pioneering folks, Scott Freeman and Mary Pat Wenderoth in biology and they use this term structure to think about, “Well, how do we scaffold student learning before class, during class, after class, and we’re continuously building as we go up through the three levels.” So pre-class involves some kind of content acquisition, that can be reading your textbook, reading journal articles, watching Khan Academy videos, other sources like that. And then some kind of formative assessment before class because we all know students need a little bit of a carrot there to get some of that work done sometimes, which is totally great, because you give them a lot of chances, sometimes even unlimited attempts… again, formative assessment before class, make sure everyone has the same solid foundation, then you come to class, high-structure course design, just like any good class nowadays should be really rooted in active learning. So we got a lot of activities going on: group work, problem solving, case studies, real-world connections, any kind of flavor you want to do there with your active learning as long as students are engaged. And the big thing here though, is that you’re aligned to what happens before a class. We’re building off what happens before a class, I’m not in class just repeating what happens ahead of time. Rather, I’m using the foundational material students acquire before class, we build that up with another level in class, and then we keep going. And then after class, there’s a bucket there, where students are going to be practicing those skill sets they’ve been developing through the before class and in-class buckets, applying their skills to new contexts, solving new problems, doing projects, essays, more authentic assignments, whatever you want it to be. And then there’s typically a frequent assessment piece with a high-structure course design too, whether that’s a multiple midterm model, weekly quizzes, every other weekly quizzes, things like that, in addition to some more authentic assessments, which is something I’ve tried to get more into this past year, by adding more realistic projects and course assignments in my high-structure courses.

John: Could you give us some examples of some of the authentic assessments that you use in your classes?

Justin: Yeah, thanks, John. So this is really new to me still, this past year. I know I’m behind the curve. I know this is a term has been around there for a while. I always like to think I’ve had some authenticity in my classes for a long time. I integrate news stories, I use case studies, I have this new thing I called pod cases where I take podcasts and combine them with case studies and we use those in the classroom. But I thought I could try to be a little bit more authentic, even still… getting that advice from my wife for my personal life. She always says that can be more authentic. So, maybe I can make my courses and my assessments more authentic too. One example I’m doing right now. I’m teaching introduction to biomedical engineering, and I’m having the students write the actual legit National Science Foundation Graduate Research Fellowship proposal. Those of you familiar with that, there’s a personal statement part and there’s a technical kind of research proposal part. So we’re just doing that. It’s a two-page proposal, and we’re working on this in a structured way throughout the entire semester. Last week, students had to turn in the research topic and research question. This week, they’re turning in their hypothesis, in a couple weeks they’re turning in the literature review and experimental plan. We’re gonna have two rounds of a formal peer review of the entire proposal. And then we’re going to conclude with a mock panel discussion, just like you might have in the real NSF or the real NIH where students are going to be reviewing each other’s proposals and basically pitching them to each other and seeing if they should get, quote, unquote, funded or not. I don’t have any funds for them. Maybe we’ll bring candy on the last couple of days. But that’s kind of the idea of having a real thing that students could actually use some day if they’re going to graduate school. But even if they’re not, being able to write technically and persuasively in two pages on a proposal is a really important skill that they should be working on, in my opinion.

Rebecca: Sounds like you’re going to need some chocolate coins for that day.

Justin: That’s a great idea. I will get chocolate coins. [LAUGHTER] I didn’t think of that. [LAUGHTER]

Rebecca: Can you talk a little bit about what motivated you to attempt and move in the direction of high-structured course design.

Justin: I kind of started out this way, I kind of started in the deep end, if you will. So my postdoctoral program was called spire, S-P-I-R-E, and it’s one of these bigger programs called IRACDA through the NIH. They’re postdocs where you still do research. So I was a molecular biologist, biochemist at the time. I have a weird background in chemical engineering, bioengineering, molecular biology, but at the time, I was studying muscle proteins in squid, super fun stuff. And then I also got training, though, on how to teach evidence-based college science and engineering courses. In my postdoc, through great people like Ed Neil, Brian Ray Bozic, Leslie Lauria, and then also Kelly Hogan was someone that took me under her wing at the time, she was able to, along with the rest of the SPIRE team, helped me be introduced to using evidence-based course design practices, which included with high structure. So this was around 2010-11, right when these papers are starting to come out from Scott Freeman and Mary Pat Wenderoth. I started with high structure because the evidence shows it works. And that’s as a trained scientist, as a trained engineer, we like to go with evidence ideally to guide our decisions. And it’s no different with course design and teaching as it would be with thermodynamics or reactor design or anything else. The literature on high structure is really strong, even then it’s been growing over the years, some of my own data and a lot of other papers. The main gist, though, is that by having the scaffolding before class, in class, after class, students are able to really monitor their learning, develop self-regulated learning skills, be more metacognitive. And the data in a lot of papers shows all students are doing better in the class. But it’s not just everyone, using high structure and moving up in structure will start to close or reduce achievement gaps or performance gaps between historically underperforming students and their majority counterparts. So for example, female students and male students in physics, there has traditionally been a gap there, add some more structure to the class, those gaps tend to get smaller. And I’ve been participating in that kind of research here over the last many years with undergraduate students and my research team. And we found some similar results across different STEM disciplines. It’s really the evidence guiding me in these decisions to do it. I also think it’s a lot more fun. Once I have in mind one of my very first faculty interviews, oh a long time ago, now, I had an interviewer say to me, “Well, hey, I’ve looked at all the papers on active learning, and they’re all wrong. [LAUGHTER] All the statistics are wrong. What do you have to say about that?” And I paused for a second I said, “Well, even if that’s true, it’s a heck of a lot more fun to teach actively with active learning than not, I think the high-structure design model is more of a fun way to teach. And again, you’re helping the students with those study skills and metacognitive skills that they can then take forward to their future classes.”

John: There was a recent paper that did argue that many of the studies on active learning were flawed in some way in terms of a very strict methodological approach. However, there are so many of them, and the results are so remarkably consistent. I don’t really buy that argument in terms of the questions about methodology.

Justin: You’re right, it was a big deal, I think it was either Beckie Supiano or Beth McMurtrie in the Chronicle had articles on this going around for sure. And that’s the thing though, with us, as DBER folks… I want to introduce a new term, DBER – discipline-based education researchers… which I consider myself to be in. So I’m in the discipline. I’m in chemical engineering, I’m in biology, I’m in Biomedical Engineering, and I do the education research from that lens, but also from that training background. I’ve never been formally trained in education research methods in psychology or social sciences or psychometrics. I am, totally honest, I fake it. I do what I can. I know my boundaries. I know when I need a collaborator in the true social science disciplines that helped me with my studies. And so I think that paper was getting some of that. Sometimes the folks like us that are doing this work, whether we call ourselves DBER people, or Scholarship of Teaching and Learning (SOTL) people, we might not have that true background, and therefore the rigor of the paper, the methodologies, aren’t as advanced as maybe they should be. But like you said, there’s so many studies showing benefits of this, whether it’s simple correlational studies… sometimes they get to the mixed methods, even quasi-experimental studies, but the evidence just keeps mounting and mounting and mounting. And that key paper there was the 2014 one from Scott Freeman that PNAS meta-analysis showing that hundreds of papers, hundreds of studies… again, they have their flaws, don’t get me wrong… but you start to see these themes everywhere and you think, well, there must be an effect, something’s going on.

John: And we will include a link to that paper in the show notes, if anyone would like to follow up. But the fact that there may be these flaws in some of the research methodology suggests that maybe we need to see more people doing work in this area, and that there’s a lot of areas where there’s potential growth.

Justin: Totally agree.

John: And you’ve done quite a bit of research already.

Justin: I’ve tried, as a teaching faculty member now for this is going on year 12-ish, but I’ve always made that a part of my career. Before I was at Mines, I was at University of California, Irvine, where we did have a scholarship expectation or to earn tenure as teaching faculty. The UC system is one of the few in the country where you have actually tenure-stream teaching faculty, which was really cool. So I earned that, but I only had tenure about a month because then I left and came to Colorado, which… no regrets, I love it here at Mines… but I don’t have tenure here. But there’s still some expectations for scholarship, but not as high as it was in the California system. But I continue to make that part of my career, because, again, as a scientist and engineer, I think going based on data, based on evidence to guide your decisions, moving forward is a really important thing to improve what you have, because that’s all it comes down to really whether you call it SOTL or DBER or whatever, we’re trying to improve our courses, we’re trying to improve our students’ experiences. And yeah, so I’ve tried to do what I can over the years. And now I’m at the point where I want to help other people too. So I work with faculty individually, I do workshops via my side business, and even at Mines this semester, starting next week, I have a three-part getting started in DBER series kicking off for about 25 faculty on campus. I’m going to help them get started, get their feet wet in this field, and then we’ll see where it goes. Because I think a lot of folks, like you said, John, want to do this work, they’re interested in it, they might not have the time, they might not know how to, but you kind of give them the start, hold their hand a little bit and then see where it goes, and we’ll get even more evidence to support these claims.

Rebecca: I think a lot of faculty have an inclination to have a reflective practice and continuous improvement. They’re wanting to do SOTL work, but just don’t know how to formalize that practice. And we can certainly lean into helping people move in that direction.

Justin: 100% Rebecca, even when I started, I didn’t know what I was doing. If it wasn’t for Kelly Hogan and a few other folks at Carolina and then moving on from there at UC Irvine folks like Brian Sato, Diane O’Dowd, Adrianne Williams, they really helped me understand the field more. And then you start going to conferences and going to meetings and learning more about the tricks of the trades.

Rebecca: Can you talk a little bit about ways that a high-structure synchronous course might be different from a high-structure asynchronous course?

Justin: So I think the high structure while what I described earlier, you might be thinking of okay, a Monday, Wednesday, Friday class meeting at two o’clock in a lecture hall works great. I think it works even better in online for some cases, because you think about an online situation, whether again, it’s synchronous or asynchronous, your students you’re not really seeing and there’s that human connection of having eyeball to eyeball being able to understand, “Okay, I’m here I’m motivated, I’m ready to learn.” But whether it’s the recording, that’s the worst case scenario, when it’s totally up to you, or if it’s going to be on that live Zoom remote, where you can make your screen black, and who knows what else you’re doing. I’m guilty of that in meetings for sure. But having the structure though, having that scaffolding, having the clear deliverables of what is due when and seeing that step up from pre-class, in-class, after class, whether it’s the in-class is out again, synchronous Zoom session, or the recorded video you do on your own time, there’s that scaffolding helps students stay more connected with material, they understand more of what’s going on, when it’s due, what is due, how I’m being assessed, because that’s the nice thing about with structure. And just to be clear, I’m from the STEM disciplines, but you could do this any field. The research on high structure, as far as I know, is only been published in the STEM fields. I bet it’s being done in art history, I bet it’s being done in the languages, I just don’t know about it. So if anyone listening does this kind of teaching or research, I’d love to hear from you. But you can use that scaffolding again to help students be more transparent with the course. And that’s a word that I think comes through with high structure and is really true in that online environment, especially as transparency. I always tell my students, I want to know what you know, not what you think I want you to know. And by being uber clear with the learning objectives, starting with backwards design, just a clear Canvas website for the online cases can be really helpful for that. One distinction, though, with the async online versus synchronous online, is that kind of active learning piece, because we’re not there together. So one tip I like to share with faculty is that okay, if you’re recording a video for the in class, mimic, because you’re not actually meeting in class. What I always say is “okay, well make the video, but if there’s a point where you’d want them to answer a clicker question… I’m a big clicker guy… or if you want them to do an activity, just say, Okay, everyone, pause the video here. Take a couple minutes, think about this problem, what’s the answer or work on this, whatever it might be. When you’re ready, unpause the video and keep going. That’s a way that you can add the active learning to an online asynchronous class as part of the bigger high-structure model. I find when I have to record an online video, if I have traveling for a conference because I have to miss a day of class. I’ll have a 50 minute in-person class, when I make the video recording it’s only about 20 to 25 minutes, because all the active learning I don’t do in the video recording, but I give that tip… I tell my students, hey, pause the video here, answer this quick clicker question, solve this problem, come back when you’re ready. And so that’s a way to build the active learning into the asynchronous part, which again, is part of the overall high-structure piece.

John: And also, if you want to take it a little bit beyond that, you could use one of the tools where you can embed the questions directly in the video, and have them auto graded. I use PlayPosit for that, and it’s possible in Panopto, and EdPuzzle is another alternative that many people use. PlayPosit works really well. And I’ve been using it for a couple of years.

Justin: I just learned about PlayPosit maybe three weeks ago, I was doing a workshop at CU Boulder on clickers. And I kept hearing PlayPosit, it does seem like a very powerful tool to automate that active learning in the video piece, which I really like. I just haven’t had a chance to play with it.

Rebecca: So you did just mention that you’re a clicker guy, I believe. So do you use a flipped classroom approach in all of your synchronous classes? And how are you using clickers in your classes?

Justin: Yeah, I think that’s the technical term these days, clicker guy. So, I’ll go with that. I’ve been using them for as long as I’ve been teaching in various platforms. You name the platform, I’ve used it. I’ve primarily been using iClicker, just because that was the platform we had at UC Irvine and the platform we have here at Mines as well. Yes, I love using them. I think it’s really important. And you mentioned the flipped classroom design there, too, Rebecca. So when you think of high structure, you might think, “Oh, it sounds like a flipped classroom.” And it kind of is because we’re having students do stuff ahead of time. In class, they’re very active, but there’s still that structure piece even within class. I guess maybe I haven’t seen a truly flipped class because I think flipped classes, I think in class is like purely work time, purely activities, purely worksheets, homework type assignments, whereas I do a series of clicker questions, discussions before or after numerical problem solving, if it’s more of my engineering classes that I teach in thermodynamics, or material and energy balances, more conceptual sometimes if it’s biology or anatomy and physiology, but I’m breaking up a lot of activities and any talking with clickers. And so the clickers… I love all the modern bells and whistles that they have as that’s the another technical term I use, moving beyond multiple choice. And when I do workshops for faculty with clickers, that’s what I triy to get them to do. Okay, yes, you know, you can do A B C D E, but can you do multiple answers? Can you do target questions or hotspot where you touch the screen? Can you do ranking or ordering? numerical answers? …and I’m trying to get this traction on my own campus too. So even this semester, my first five questions of the semester, which I didn’t do intentionally, but after I wrote them, I realized, oh, I should do this every class now… my first five questions in my introduction to biomedical engineering class, each one was a different question type. So we did multiple choice, we did multiple answers, we did short answer, text entry, we did numerical, and we did target or touch the screen. And when I was done, I asked my students, “Hey, who’s ever used those other question types before?” And maybe four hands went up, I said, “Cool. What class did you do them in?” and they said, “Oh, your class last semester.” I seem to be the one and only that I know of on my campus that’s trying out these other tools, although like I said, I’m trying to change that. So yeah, if you’re using any kind of device like this, any type of ed tech, for that matter, make sure you justify it, justify the cost, and you make use of those bells and whistles in class. Or if it’s out of class, courseware, whatever it might be, just make good use of the tool, and you’ll find your students that are going to be able to buy into it and use it more effectively, and hopefully learn more from it.

John: And one option that we have here is a campus license, which actually is much less costly per student. And that reduces that barrier of students having to pay for it, which has led to a pretty rapid expansion in the use of it on campus.

Rebecca: One of the things that I really like about your example of the five questions is that extra level of scaffolding, of introducing all of the question types right away, and giving everybody an opportunity to practice those question types that you’re going to be using all semester. And so I just wanted to make explicit that other layer of scaffolding that you have built in there.

Justin: Yes, teaching students how to use the tools is very important. Thanks, Rebecca, for mentioning that, because whether it’s the app now on your phone for these clicker devices, or if it was the old physical device, there’s a learning curve with them. And same thing, even if it’s computational tool, or like Desmos, or something for a calc class. Yeah, you got to be able to teach students how to use the technology properly first, before you expect them to use it well. I will say though, with the learning curve of the clickers, when I was at Irvine, the app wasn’t really there yet. So everyone had the physical iClicker remote, and this was the iClicker w, so you could do text entry, and you could do number entry, but you had these up and down arrows, which you had to click to go through the alphabet. So I do the type of clicker question where I like to give four statements on the screen, and I say, “Okay, well tell me if they’re true or false, and you have to click in all four.” So you would click in a string of T’s and F, so it’s like TTFT, or FFFT, whatever it might be. And so when I would do this in intro bio there with 440 students, it sounds like a swarm of locusts, all the clicking to get to the T’s and then someone says, oh, you should change it. Just make A be true and B be false? I said “Great idea, I’ll do that.” But now with the app, we lose the sound effects and iyou can just type in T’s and F’s and it’s a little bit easier to use.

John: One thing I haven’t tried with iClicker, and I do plan to do it this semester is allowing students to rate their confidence in their answers. Have you tried that yet?

Justin: I did just last week. So I just found out that was available this semester, and I gave it a shot. And it was kind of fun. Like the first day I turned it on, they’re like, Oh, look at this, this question was 80% confident, and now it’s down to 40% confident and it kinda reflected the percent answers that were correct. So like the one that was 80% confident it was a single multiple choice answer. And it was pretty much most people got it right. The one that was lower confidence, like 40%, was a multiple select answer. So they had five options, but two or three could have been correct. And so the confidence goes way down through a different question types even. So, I think it was really fun for a day or two, and then kind of the novelty wore off, unfortunately. [LAUGHTER] So I’m going to try to revisit it maybe more in targeted times and targeted questions rather than doing it for every single one. But it was fun to try it. I’d encourage you to do it, too.

John: One of the reasons I was considering it is that it might help students improve their metacognition a little bit by reflecting on how well they thought they knew something and then getting fairly immediate feedback on whether their expectations about how well they knew it actually matched the response, [LAUGHTER] or the feedback to the response.

Justin: Absolutely.

Rebecca: When you’re using clickers, are you also using strategies like think-pair-share as part of your implementation?

Justin: Yeah, absolutely. And there’s that sweet spot of clicker responses that lead to that. So the best practices, I’d say, and a lot of this work came from Carl Wieman Science Education Institute, which I believe is still housed through Vancouver, British Columbia, but he’s at Stanford now. But there’s this great flyer they made like this trifold flyer that kind of walks you through how to use clickers properly, or best practices suggested, and then what are the possible outcomes? Typically, you’d start off with that silence. Okay, click in on your own at first, I tell them I want to hear crickets, just click in by yourself, and then I’m able to see on the app the distribution and so at that point, there’s typically kind of three outcomes. So one is maybe 80-90% have the correct answer. At that point I’ll say “Okay, let’s move on.” But I make sure to take time to have students respond and say, “Okay, well, why is it C?” or to have a little discussion about why it’s not the other answers. Because even though you get 90% correct, there’s still 10% of students who got it wrong. So we want to make sure that everyone’s on the same board. The next option is usually that kind of random distribution. Let’s say if it’s a multiple choice question, you get ABCD each are in the 20 to 30% range. At that point, I call that Hmmm, what’s going on? Was it a really hard question? Did I goof the question? Is there a chemistry exam this evening, so there’s some reason why maybe it didn’t click? And then what I call the Goldilocks case is when you get the kind of 50-50. So maybe you’re split between two answers. And I love actually, when it’s exactly 50-50, I’ll show that on the screen. But sometimes I don’t, and I’ll say “Okay, at this point, now, you’ve all had a chance to try on your own, now talk to each other.” So that’s kind of the think-pair-share that’s built into the clicker there. So this is peer instruction at its best. This was pioneered by Eric Mazur at Harvard, people like Jenny Knight at CU Boulder have studied this. And she actually has a Science paper with Carl and a few others that showed when students are doing this… So you give them some quick information on their own. Don’t show them the answer, have them talk to each other. They put a bunch of microphones around the lecture halls at CU Boulder, and they listened into their conversations. And they actually found students are legitimately teaching each other and debriefing the topic, discussing it, tearing it apart. It’s not just the one student says, “Oh, it’s A ,everyone else, click A.” They’re actually debating it and talking about it, so that you get that true peer teaching and peer instruction going on. And then typically, you see everyone moving more towards the positive correct response that you’re looking for, at which point you can say, “Hey, look, you don’t need me as your instructor, you’re teaching each other. You’re doing great. Kudos, keep up the good work.” Yeah, there’s so many ways to riff on clickers and use them in different respects. Whether it’s the more traditional question types, or now the more advanced question types, and no matter how you do it, you’re getting students more engaged, they’re having more fun with it. And fun actually is a word they use with clickers. I have a paper a few years ago from the HABS educator where I looked at clickers vs. Kahoot!s, and students rated Kahoot! and clickers both equally fun, where Kahoot! being the more gamified version with trophies and a podium and music and sound and stuff like that.

John: And both Eric Mazur and Carl Wieman and some co-authors had done some experiments on the use of clickers with that methodology with answering it individually and then discussing it and voting again. And they found some really substantial learning gains in both of those studies. And we’ll include links to those in the show notes. In all areas, we’ve been seeing some significant equity gaps, but I think since COVID, we’ve been seeing much worse equity gaps in terms of students because learning losses were pretty pronounced everywhere. And that’s especially in math and in the STEM fields, but they were especially large in lower-income communities where schools were not as well funded and where students did not have the equipment and so forth, needed to thrive as effectively in remote instruction. One of the things that some institutions have been using increasingly is co-requisites in place of prerequisites to help students who want to pursue a major in a STEM field, but who would otherwise fall way behind or take many years just to get up to the basic level of math needed for the entry level courses in the discipline. In 2021, though, you had done a study that did a comparison between the effects of prerequisites and co-requisites. Could you talk a little bit about that study and what you found?

Justin: Yeah, absolutely. We saw the same thing probably everyone else did with students struggling, returning from online instruction in high school and having online instruction here. I taught remote synchronous for three full semesters, even though at Mines we stayed open for business pretty much the whole time with in-person classes, just lower capacity. I had prior online instructional experience in my career, so I volunteered to do those online sections for our department. But with the co-rec/prereq issue, I think that’s really important to consider for a lot of reasons. And this work really started when I was at Irvine with Brian Sato and Pavan Kadandale and a few others, looking more in the biological sciences and “do prereqs matter?” It was kind of our big research question. And then I carried that over to Mines and applied that to chemical engineering too. The short answer is: it depends. So first of all, and kind of why do we even have prereqs or co-reqs? A lot of it seems to be more of a management thing. So if we want to control class sizes, moving from first year to second year, and so on, and so forth. But there’s definitely an idea of well conceptual understanding and carry forward for if you have Bio I, well, those skills you develop could be used and count for something when you get to Bio II or same thing with the calc series or whatever it might be. However, we don’t know. It’s unfortunate that I think programs don’t sometimes look at those connections, that vertical alignment, if you will, between courses in terms of prereq to the following class, or with the co-req issues. At Mines, what I did with the paper you’re referencing, John, in chemical engineering, we have these two courses. One is called material and energy balances, MEB, I’ll call it for short. And that’s kind of like our intro bio version for chemical engineering. It’s kind of like a survey course that covers a lot of different topics, gives you some foundational skills that you’re going to need as a chemical engineer. We also have introductory thermodynamics, and an intro thermo, it’s related to MEB, covers some same conceptual understanding and problem-solving skill development, but it’s more on the nature of energy, and that transformation understanding. So typically in chemical engineering curricula, and I’m getting in the weeds here, so apologies for anyone who’s tuning out cuz you’re not a chemie, but MEB comes first, that’s typically either the fall of sophomore or spring of sophomore, you take that. First, it’s kind of like your gateway class, and then that’s a prereq to intro thermo and everything else you take. At Mines though, we do a little bit different. Mines, we do intro thermo first, fall of sophomore year, and then in the spring of sophomore year, you take MEB and coming from that background myself as a student at Penn State, I wish I would have had that way because intro thermo is the more… I use the word gentle… it’s a gentler introduction to chemical engineering than material energy balances is. But the idea here is that you can take thermo as a prereq in the fall or as a co-req in the spring with MEB. So the natural question came up… it’s a nicely well organic experiment to test is, well, if you have the thermo as the prereq do you do better than if you take thermo as a co-req with MEB. And this has implications for advising and its implications for progression through our major through repeating classes, all sorts of things. So it’s a really worthy study and my student, Jordan Lopez, at the time, he was an undergraduate student in mechanical engineering, and I love working with undergraduates on these SOTL and DBER projects. Jordan came along with me for this we did a quantitative assessment to figure out okay, well if students in MEB, did you have thermo as a prereq or co-req? How’d you do in MEB? And then we also asked the students qualitatively “Okay, well, do you think having thermo as a prereq or co-req matters? And if so, what do you think? And what we found was there was actually a big deal. Students who took thermo as the prereq did significantly better in their overall course performance in MEB than if they took thermo as a co-req, and that’s using linear regression models controlling for incoming GPA and things like that. Students also felt the same way. They felt that if you have thermo as a prereq, you would do better than having thermo as the co-req with MEB, and again, the quantitative data bore that out. The reasons they gave for that was because they get some opportunity to get more familiar with the discipline, with how we approach chemical engineering, how we do problem-solving skills, and having that foundation in the prior class allows them to do better in the follow up MEB course. In this case, it mattered, prereq vs. co-req. However, in our studies at Irvine, with Brian and Pavan and others, we found it didn’t really matter so much whether you took for example, microbiology lecture before after microbiology lab, you would think those two would be highly correlated one as a prereq to the other, it didn’t really matter in the result. Same thing with anatomy and physiology sequences, it didn’t seem to matter too much, in other papers in which we cite in these studies looking at organic chemistry sequences, economic sequences, it’s kind of hit or miss with whether prereqs matter or not. Long-winded segment here, but I think we as programs should be taking careful look at our prereq sequences and core sequences in general to see what, if anything, our students carrying over from one course to the other, and maybe we should loosen that up a little bit, because if they don’t matter, that can help students be a little bit more flexible with their degree planning, and time to graduation and things like that.

Rebecca: Speaking of prerequisites, faculty are often complaining that students don’t necessarily recall [LAUGHTER] information from prerequisites as they go into a sequence like we might expect. In 2018, you conducted a study that investigated the effect of physiology prerequisites on the future anatomy and physiology courses. Can you talk a little bit about that study and what you found there?

Justin: Yeah, so that was what I was alluding to there, Rebecca, with some of the work I did with Brian and Pavan. And in those cases, there wasn’t too strong of an effect, if anything, with the sequencing and the ordering that students took these classes. At Irvine, we did have human physiology as a prereq to human anatomy. I know that sounds a little funny to those of you in the A&P space. But the reason we did that was because of kind of a controlling the size of things. Our physiology course there was a lecture-based course, it would have 3-4 hundred students in it every quarter… talking about over 1000 students a year taking this class, versus the anatomy class, which had a lab, we can only handle about 144 per quarter, so availability of seats in the human anatomy was much, much lower than physiology, and also not as many students wanted to take anatomy versus physiology. So that’s kind of why that ordering existed. And so we tested that, though, looking at with what Brian and Pavan and I came up with, we call it a familiarity scale. How do you do on questions that are based on how familiar you are with the content based on the prereq? What we did was we had the prereq instructors look at the follow up course exam questions and write them as very familiar or not familiar based on basically whether it was covered or not. And then we looked at student performance on those questions based on having the prereq or not, and we didn’t see much of a difference based on the very familiar versus not familiar questions. The way we designed those studies, we didn’t come up with any strong conclusions, again, on the biological sciences side saying that, yeah, these prereq sequences matter. But on the chemical engineering side, which I just described, that seemed to totally make a difference, the ordering of the classes.

Rebecca: It seems like it’s really important to think about why we’re structuring the curriculum in the way that we are. And is it just traffic control? Or is there actually a legitimate reason why we’re hoping that they are scaffolded in the discipline?

Justin: Yeah, I think it’s reality, a little bit of both that happens, which makes sense, kind of have that traffic control, but also making sure that that alignment occurs through our new programs or existing programs, and just taking the time as a program or department every so often to say, hey, let’s look back at this and see, why are we doing it this way? Things need to change in this modern day.

John: One thing I wonder a little bit, though, when instructors are complaining about how little students remember from the prerequisite courses that they’ve taken earlier, is how much of that may be due to the incentives provided to students. The way many courses are structured, not in the sense of the structure you were talking about, encourages students to do a lot of cramming before the high-stakes exams. And we know that that results in very little long-term recall. Might some of these issues be resolved if we could encourage students to adopt better learning strategies, and if we could design some of our courses at the lower level, perhaps, to build that in, using the type of structure that you’ve been talking about? Because at least from what I’ve seen, that’s not so common? You picked up this stuff fairly early, but I don’t think that’s part of the training of most of the faculty in STEM fields or in most disciplines.

Justin: Yeah, I agree with that. And there was even in 2018, I believe, in Science, a paper, I think they call it “The Anatomy of STEM Education in North America,” it was something along those lines, where they looked at a big survey a big swath of courses and trying to see who’s using Active Learning who’s not, they use the COPUS protocol, the observation tool to characterize what’s happening in the classroom, and they broke it down in these seven different buckets. And even at that time, a very small percentage of the courses surveyed in STEM were taught using these student-centered strategies, not necessarily high structure, but the active learning, student-centered ideas. But I completely agree with you, John, if we can get more faculty on board with using these types of evidence-based practices, which yes, it takes some time, takes some energy, takes some foresight to develop these courses. But you do the same thing with your research, it takes your time and energy and careful thought to put in your research experiments. We should put the same energy into our classes too. And if we can do that at the early stage, which again, there’s such a huge issue with loss of learning. Not only that, but also lots of students from the early stage. There’s a paper after paper for these bigger courses, taking them as quote unquote gateway classes, we get a lot of attrition at these levels especially of students who might not be as well represented in STEM. We can add high structure. We know that works. We know that helps students do better and succeed. It might not work the first time, don’t get frustrated everyone. There’s a great paper from Anne Casper and Scott Freeman about having True Grit with being able to try and try again. So in her case, she tried adopting high structure at a smaller school in Michigan, it took her about three tries to get the right formula for it to work for her students to see success. And that paper documents that trial and error over a few semesters. But if we can use these types of methods in our classes, ideally, our students are going to be better prepared then for future classes. I do these things called Reading guides, which again, another shout out to Kelly, I got a lot of inspiration from her at Carolina for these, these are just Word documents that I make. And I have a bunch of these on my professional website. If anyone’s interested to download them for free, go for it. They’re just Word documents that help students read the textbook and again be more transparent with what I want them to know. So it has them define terms, fill out tables, make little drawings, they answer checkpoint questions in the text, summarizing things in their own words. And with the reading guides, we’re able to help students again acquire that content and then use it in the class as needed. I’ve had students in follow-up classes say to their instructor, “Well, where the reading guides at?” And they’ll come to me like, “Hey, what are these things? What are they talking about?” So the point being with that story, though, is that I tell students that “Hey, even if you go to a follow-up class, where it’s not highly structured, you’ve developed so many great metacognitive skills, you’ve developed so many great self-regulated learning skills through this class that you can then apply it and add your own structure to follow-up classes, if needed. And these are the kinds of tips that I love working with faculty about to try to help them transform their classes when I visit a campus and do a workshop or work with faculty one on one or on Zoom or whatever, trying to take these practices, which again, are rooted in the evidence, they’re rooted in the literature, but we have to tweak them to your own personal situation, your own student body, your own courses and demographics and everything. You all know your students way better than I do. What I try to do is combine what I know from the literature and the evidence. And that says a lot of this is in the book I’m writing on high structure, it’s going to guide you through the course-design process, a lot of practical outcomes and deliverables. But you got to tweak it and match it with again, your unique circumstances to achieve those positive outcomes.

Rebecca: I think one of the nice things about introducing students to some strategies for high-structured learning and retaining information is that if you have things like your reading guide, and they show it to another instructor, now they might request that of their instructor and the other instructor might decide, “Hey, this is a good idea. This is a strategy I want to adopt too.” And that’s some of the best ways to get some of these techniques to spread.

Justin: It really is. It’s that camaraderie building through seeing each other’s practices, seeing what’s going on. I’ve been part of a project here at Mines since I got here almost six years ago now where we’ve been working on kind of revising how we evaluate, and what we consider effective instruction to be. So we’re trying to take a more holistic review of how we take into account the best practices of teaching and student learning. And one of the big pieces there is peer observation. So we’re actually rolling that out, as of this week and last week with doing departmental trainings and helping faculty see how we can do peer observation to learn from each other. I learned so much from going into another person’s classroom or looking at their Canvas site, or their syllabi or their materials. And I really got this from Malcolm Campbell, who was at Davidson College in North Carolina, when I was a postdoc, this is going back 14 years or so. And Malcolm told me once on the phone, he said, “Find someone who you consider to be a master teacher, and go watch him teach.” But he said “don’t go just once, go a lot. Once a week, if you can for several weeks, and you’ll get to see them handle different situations, you get to see the ups and downs of the semester, you get to see what it’s like on the day where the midterm gets passed back. Or you get to see what it’s like when they’re getting to a really difficult topic or there’s some kind of disruption that class and how they handle it.” That’s the one great thing about peer observation is that if I go to your classes, sure, I’ll be able to give you some comments or feedback on what I saw, but I’m going to learn probably more from seeing you two teach than whatever I could give you and then I’m able to incorporate that into some of my lessons and my teaching style. If we can just change that culture, make it a little bit more okay sometimes to talk about teaching, share teaching strategies, see each other teach and have fun with it really and learn from each other, I think that can go a long ways towards, again, improving student outcomes and helping them do better in all aspects of college and life.

Rebecca: I found one of the most fun opportunities for observations is doing observations outside of your own discipline. It’s really informative to see how people in wildly different disciplines approach teaching and the way that they operate their classroom. I remember visiting a dance instruction classroom when I was at Marymount, at my prior institution, and it was really interesting, it was fascinating to me about how specific and transparent every little move was and how to correct things and the anatomy that was being discussed and just the level of depth that you might not realize how something transpires in a classroom unless you’ve actually had the opportunity to observe it.

Justin: I’m going to play that some for my faculty Rebecca. I thank you, because that’s what we’re trying to do at Mines is not intra-departmental, but rather inter-departmental observation. So getting across the disciplines for sure. Because yeah, you learn so many different things. And even if you’re just going from mechanical to electrical engineering, there’s still a lot of differences between those types of courses, you might see what’s going on. And I’ve learned that myself, I’ve watched some economics instructors teach at Mines, no idea. I feel like I maybe should know this stuff to be a better financially literate person myself, but I learned so many cool ways about how they’re teaching, but I also learned some cool concepts.

John: We do have an open classroom project here on our campus, we keep hoping more people will open their classes and more people will attend. But the people who have participated have found it universally to be valuable. So we’re going to keep doing it, and we just hope to see it become more a part of the culture.

Rebecca: Sometimes just indicating to a faculty member that you’re interested in their content, and you just want to sit in because you’re curious is a great way to learn new content, and maybe get an opportunity to observe someone teaching. [LAUGHTER]

Justin: And yeah, I definitely pitch it that way. Like I want to learn from you. And you get some new ideas. And I feel like in that case, faculty are more than willing to have you in. Well, yeah, Best of luck with that open classroom project. I’ve seen those types of programs in the country kind of sometimes hit sometimes miss. But best of luck with that.

John: One of the things you’ve done is just some research or some experiments involving two-stage exams. Could you talk a little bit about how you implement them, and what you found in general? And how did students react to the two-stage exams?

Justin: Yes, so I’ll try to be brief on these because I could spend probably 45 minutes on these myself. But yeah, two-stage collaborative exams, or more affectionately called group quizzes or group exams for short. There’s been a ton of literature on this Jim Cook, UC San Diego, he really helped me get started in this about five years or so now. But it’s all disciplines. You look it up, and the one paper I have, you mentioned in the intro, there’s a lot of references to different STEM disciplines that use them and some non-STEM too. The idea, though, is that we push active learning, we push group work, we push collaboration, but then we assess you by yourself with a pencil and paper and a calculator on a piece of paper, so why not change the dynamic there a little bit. And the two-stage idea is that first you have students take an individual quiz or individual exam, and I personally have gone to a weekly quiz model with my high-structure courses. I don’t do midterms anymore, I don’t do finals sometimes even. I know that’s taboo sometimes, but it depends on the class and I might not even do that. My colleague in Irvine Adrienne Williams says with high structure by going to the weekly quiz model, instead of having these giant peaks of anxiety with the two or three midterms, we just have a smooth level of anxiety the entire semester, and spread it out a little bit. But, for me, group quizzes then, what they look like is students take the individual quiz, turn it in, then they get in a group, this is a predetermined group, they know who their group is going to be. And then I give them a group quiz. That group quiz in the paper that you referenced was an identical one. So they took the identical two problems. And this was a MEB material energy balance class. So it was just two numerical problems, problem solving. They did that again as a group. However, my other courses now I do a different quiz. It’s either an isomorphic question, so slightly similar, but a little bit different, maybe in intro thermo or if it’s biology or anatomy and physiology, completely different set of questions. Then they do it as a group. The reason we do this, again, was trying to get this collaborative piece in the group setting. But also, when do you find students are most excited to talk about a quiz or an exam? It’s as soon as it’s over, they go walk out, the hallways abuzz, everyone’s all excited, good or bad excited, but they’re talking. So let’s capture that energy and do it in the classroom through that group assignment. Now, the group quiz is usually a fraction of the weight. In the literature, you’ll see, it’s usually let’s say you call the whole experience 100%, the individual quiz, maybe 80%, the group quiz is 20%. But that can vary from 75 up to 95, however you want to do it. And again, they’re covering the same kind of concepts just in that group dynamic. And so when I did this the first time, I got some consultations from my co-instructors, Rachel Morrish, Dave Marr at Mines, and I was a little bit nervous to see how it would go. But it was a hit. It was an absolute blast. The energy in the room is one thing I cannot describe it strongly enough here through audio, it’s just you got to see it. The students are energized, they’re talking, they’re debating. They’re working through the problems together. They’re trying to figure out, I will use the F word here. It is fun. It is a very fun experience to see. I’ve had students tell me that after the quiz is over, like that was a lot of fun. Talking about spreading things on campus. This is one thing that has spread a little bit at least within my department of chemical engineering, a few other classes have been using them now. And I collected some data on this, which I published, showing that… which makes sense, there was no shock… but the group quiz score is higher than the individual score because you’re tackling this together right away. And then also though, students strongly preferred it. The affect, the positive attitude response was through the roof. Both of those results, these are replicated from other disciplines, they see the same kind of things. I think it’s a really nice way to promote that community in the class, show that you’re working together, add that active part to the assessment. However, I’ll end with saying there can be some resistance to this, especially from students who might feel like “hey, I can do it all on my own, I’m fine.” And then they have to work together with maybe some students, they don’t know as well. You might get some resistance from students saying that I don’t want to work with someone else, I wanted this on my own. So be prepared for that. Also, accommodations with testing, if you have students who get extra time, that can be difficult to manage. I will say though, from doing this now about five years and four different classes at Mines, I’ve never had a single issue with students and extra time. What I typically do is I’ll have students get the extra time for the individual part, then they come join us for the group part and have the normal amount of time because in that group setting the students with extra time, they’ve always told me and I always ask them, I say, “are you okay with this?” And they’re like, “Yeah, it’s fine to have the normal time because I’m in a group.” So just some things to be aware of, if you’re going to try them out. And again, I could go on and on with these, so if anyone wants to learn more, hit me up. I’m happy to chat with ya.

Rebecca: Sounds like a good day to visit a classroom, huh?

Justin: It’s an energetic day for sure. Although you might not see much teaching going on if it’s just the assessment part. But it’s fun to drop in and see what’s happening.

John: There is a lot of teaching going on in their second stage, though.

Justin: Absolutely.

John: In fact, when I first did it, I was so excited about it, I made a short video recording and sent it to Rebecca, so she could see the dynamics of that interaction.

Rebecca: And hear it.

John: It was just really, really different than going over a quiz, where some of them would be really happy, but not that focused. And the students who did poorly were generally pretty disengaged when we went over a quiz ift they didn’t do so well. But when they were explaining it to each other, it was so much more positive.

Justin: Yeah. And you’re right, the teaching is happening. It’s coming from the students to each other. Not for me necessarily, but it’s coming from them. And it’s so powerful to see… absolutely to hear that live.

Rebecca: But you’ve set up the structure to facilitate it. So therefore, you’ve done the teaching.

Justin: [LAUGHTER] Thank you. Yeah.

Rebecca: So we always wrap up by asking: “what’s next?”

Justin: Yeah. So for me, what’s next? You mentioned at the top, John, I have this new position, I’m Associate Dean of Undergraduate Studies. And I mentioned in passing, I have a side business I called Recombinant Education. I do workshops for faculty. I’m writing a book. For me, I don’t know what happened, but last year, I had a birthday the ends in zero. Things have been a little wonky in my brain lately. And I listened to Taylor Swift now, sometimes. I never used to, but I think a lot of people do that. So it’s not just me turning that age. But the point being is that I’ve been doing this enough time now that I feel like I have things to share. And I’m trying to do more of that. So I’m trying to, and I’m still nervous to do things like this kind of stick my neck out and even talk to folks like yourselves, but I want to share best practices with teaching and learning, specifically with STEM education, because I still hear from students and I still see it from time to time, students are just not getting the best classroom experience they should be getting. College keeps getting more and more expensive. Students don’t deserve to have less than excellent experiences, in my opinion. I’m doing what I can on my own to share these best practices. Someone called me a one man teaching and learning center, it’s probably a fair way to say it. One of my students said, “Oh, you’re teaching teachers how to teach,” and that’s, I guess, another fair way to say it. But again, I got the energy for it now. I kinda have this midlife crisis going on too where I want to make some bigger impacts on the world outside of my own bubble, whether it’s at the administrative level at Mines now or across the country, or even the world. So I love working with other faculty on all these different areas. And that’s what I’m trying to make a big part of my day-to-day life now. But I’m still in the trenches. I’m still teaching classes every semester, because that really keeps me rooted, lets me try some new things out, still doing the DBER work and the SOTL work, working with students and just trying to have a blast and do what I can here in the time I have left.

John: Well, thank you for joining us. And we’re really looking forward to that book. And I hope you’ll come back and talk to us again when the book is close to coming out. Or sooner.

Justin: I would be honored to come back anytime John and Rebecca and if both of you are ever in the Denver golden area, please let me know. I’d be happy to show you around out here.

Rebecca: Well, thank you so much for joining us. It was fun to talk to you and to hear about the fun happening in your class.

John: More F words.

Rebecca: Yeah, [LAUGHTER] a lot of F words.

Justin: Yes. [LAUGHTER] Thank you both. I appreciate it. It was fun.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

GANESH: Editing assistance by Ganesh.

[MUSIC]