Category: scholarship of teaching and learning

82. Geeky Pedagogy

Posted in evidence-based practices, geeky pedagogy, higher education, learning, pedagogy, scholarship of teaching and learning
May 22, 2019

When they were students, most faculty members were not the “average student.” They generally enjoyed learning and were willing to spend long hours independently studying topics that others may not care much about. In this episode, Dr. Jessamyn Neuhaus joins us to examine how geeks and nerds can successfully teach our more “normal” students.

Jessamyn is a professor in the history department at SUNY Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessamyn is the recipient of the State University of New York’s Chancellor’s Award for Teaching Excellence. She’s also the author of Geeky Pedagogy: A Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers, which is scheduled for release in September 2019.

Show Notes

Transcript

John: When they were students, most faculty members were not the “average student.” They generally enjoyed learning and were willing to spend long hours independently studying topics that others may not care much about. In this episode, we examine how geeks and nerds can successfully teach our more “normal” students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: Our guest today is Dr. Jessamyn Neuhaus. Jessamyn is a professor in the history department at SUNY Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessamyn is the recipient of the State University of New York’s Chancellor’s Award for Teaching Excellence. She’s also the author of Geeky Pedagogy: A Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers, which is scheduled for release in September 2019. Welcome, Jessamyn.

Jessamyn: Thank you. Nice to be here.

John: Welcome. Our teas today are…

Jessamyn: I am drinking Lemon Zinger in my book nerd mug. [LAUGHTER]

Rebecca: That sounds perfect.

John: You’ll have to bring that to book signings too. [LAUGHTER]

Rebecca: I’m drinking Rose Garden today.

John: I’m drinking Twinings Enchanted Forest Fruits Black tea, which I picked up in Epcot last year.

Rebecca: Yeah, my Rose Garden’s from there too.

John: I was there for the OLC conference and you were there actually for a vacation.

Rebecca: Yep. [LAUGHTER]

Jessamyn: Enchanted garden sounds super nerdy. [LAUGHTER]

John: It is, and it tastes very good, too. We’ve invited you here to talk about your new book. What inspired you to write this?

Jessamyn: For as long as I could remember, I’ve loved to read and write and think. I’ve always been an introvert, I need a lot of solitude in order to function. I’ve always done well academically. My son, on the other hand, he’s an off-the-charts extrovert and a different student—let’s call it that, different—watching him grow up and go through the education system made me realize how differently he and I experienced school. And like I say in the acknowledgments section of my book, living with him and with his father—my significant other—is a master class in the difference between nerds and normals. [LAUGHTER] They are the normals, I’m the nerd. This was a big part of the inspiration for the book. Nobody was saying what I think is pretty self-evident: that people who earn advanced degrees—by and large—are pretty nerdy, which is as it should be—we’re the experts. Introverts are also disproportionately represented in academia, we take pretty easily to those long hours of isolated study that’s required to earn an advanced degree. Another inspiration was I really wanted to inspire teaching self-efficacy and helping faculty become effective teachers. And I use that word really, really deliberately—I understand why SoTL folks and professional developers use terms like best teachers, excellent teachers, even good teachers—but I think those terms really feed into some disempowering myths about teaching, myths like “good teachers are born, not made,” or myths like “only the most astounding super teachers affect student learning.” And those highly idealized impossible standards, I think, can really undermine teaching self-efficacy… feed into doubts and insecurities… So that was another inspiration. And similarly, throughout the book, I use us and we, when I’m talking about teaching, trying to create a sense of shared undertaking. Like we do in our classes when we talk about our class, our learning, our discussion, trying to help students become aware of their own responsibilities for their learning. And similarly, a lot of SoTL authors who I know want to invite readers to join the Teaching Commons, inadvertently undermining this goal by handing down these rigid dictates from above. “You should do this,” “Don’t forget to do that,” “You do this… that…” as if the person writing is not also in the teaching trenches trying to learn and relearn how to be an effective teacher. And I guess along with that, I also really wanted to bridge the major gulf between SoTL converts and faculty who are new to—or even resistant to— professional development. I see so much conversation about college teaching that is really divided along these two extreme positions. On one side: pro-student SoTL experts, they’re practically perfect, they never get frustrated by students… [LAUGHTER]… they’re 100% compassionate, and they’re totally on board with professional development. And on the other extreme, faculty who are totally burnt out, or completely cynical, and they’re always sniping at each other like, “You should be more compassionate to students.” “No, students are always terrible.” [LAUGHTER] And I want something in the middle saying “We can learn how to be effective teachers, we can be compassionate, we can be understanding, but also, sometimes students are irritating. It’s frustrating.” And I think finally, the most important inspiration for me was, I saw a need for a teaching book that strongly and repeatedly acknowledges the importance of our individual teaching context. And what works for one instructor just plain might not work for another. I mean, even what works for you in one class may not work in another class. And this isn’t like a brand new concept—it’s widely acknowledged—but I don’t think it’s acknowledged consistently enough. I think, especially for new instructors, I think you can read a lot of SoTL that seems to be suggesting, “If you just do this, you’ll be an effective teacher,” and that’s not nuanced enough.

Rebecca: I know that I was really excited to hear about your book, because it includes words like introvert and geeky in the title [LAUGHTER] and I identify that way. And I know that the first chapter in your book is on identity. Can you talk a little bit about why it’s important to think about identity and why that’s not often included in professional development?

Jessamyn: Sure. I’m so glad you asked that question, and the short answer to why I included it is because embodied identity is an important reality in human interactions. And I’m not sure why it’s not more fully acknowledged in professional development. It does seem like a lot of advice about teaching and scholarship on teaching and learning seems to imagine that we’re teaching in some sort of enchanted bubble that’s floating above the dreary workaday world, this wondrous place of true equality. There’s no racism or sexism and students and teachers are purely intellectual beings, and we gather—totally free of our biases—just to learn together every morning [LAUGHTER] the sky is full rainbows, and we skip down lollipop lane to another glorious day of tenure, but…

Rebecca: Right after we walk by the unicorns, right? [LAUGHTER]

Jessamyn: That’s right, yeah. When we enter the classroom, we don’t close the door, and presto change-o, there’s no race and ethnicity and gender expression, and speaking voice and physical abilities, sexual identity, they don’t exist. We bring all those assumptions, and stereotypes, and biases, and unconscious biases with us. But a lot of otherwise excellent scholarship on teaching and learning just does not fully acknowledge this. There’s a widespread assumption about what a professor looks like, and it’s a white guy, probably wearing a tweed jacket with elbow patches, and he’s lecturing so brilliantly in front of these mesmerized students that they learn without effort. And to be clear, I’m not suggesting that white guys don’t have to work hard to be effective teachers—they do—and I’m also not suggesting anyone who’s not a white guy can’t teach effectively—of course that’s not true—but any teaching advice is not going to apply in exactly the same way in every classroom and embodied identity is one—it’s only one—but it is one important aspect of our individual and unique teaching context. So, just to give an example, it’s pretty clear from the scholarship that effective teachers build rapport and demonstrate immediacy with students. But what I have to do to achieve that as a white gender-normative woman is different than what my white gender-normative male colleague would have to do. And it’s different from what all faculty of color have to do, especially because their expertise is not assumed in the same way. It will be challenged in a different way than many white faculty members. To take an even more specific example, I’ve seen teaching advice that talks about how professors need to be friendly and approachable, and that that would include smiling to students. But, you know that saying you should smile more means something different to women than it does to men, and we will hear it differently. It doesn’t mean we shouldn’t smile with students, but it means we have to utilize that scholarship in a different way. I think in every aspect of our world, white male privilege is often the default center of SoTL, and I would add maybe tenure too—this kind of default privileging—and I think we just have to start more regularly acknowledging that effective teaching and learning is shaped by embodied identity just like everything else we do as human beings. And one last point on this, it was very, very important to me that I not reinforce gendered and racialized stereotypes about geeks and nerds. Historically, those terms have been gendered male and raced as white. That’s changing, and you can see it in some contemporary popular representations. You can see it in people’s lived experiences. Those stereotypes aren’t gone—they still have an impact on people’s lives—and in fact, geek gatekeeping where white male geeks say, “You can’t play this, you can’t do that,” is still a factor, especially in fan cultures and gaming communities, it’s still a problem. But there’s so many of us nerds and geeks who fully embrace and celebrate Spock’s view of the universe, [LAUGHTER] as a place of infinite diversity in infinite combinations.

John: You also in that first chapter include a section called “Learning is Hard” and that’s probably a useful thing for faculty to remember because many of us have either found it to be easy along the way or it was so long ago that we were at that position that we’ve forgotten how difficult it is to learn new things. Could you address that just a little bit?

Jessamyn: That section draws on all the great science out there about the brain and learning. And you’re absolutely right. It’s especially important for us nerdy experts because we know our topic so well—our brains are so skilled and practiced at it— that’s the expert blind spot that gets in our way. It’s something that we do so automatically, it’s hard to remember what it’s like for a new learner. It’s hard to remember how long it takes to learn. It’s hard to remember how emotional learning can be, especially when—like I was saying with my son—we’ve had really different academic experiences than most people. Even our most brilliant students at our most elite institutions, most of them don’t take to academia the way we did. Most people want to be done with school. We said, “No, I like school so much, I’m going to stay in school forever.” So the science of learning is an important way for us to keep reminding ourselves that learning is hard and we have to do it over and over… and that does apply to us too. One of my main points in this book is that learning how to be an effective teacher never stops. We are always learning and relearning because students change, we change, curriculum changes, we’re always having to relearn. And yet faculty will often throw up their hands at the first obstacle they get to and say, “I’m not a good teacher.” I’ve lectured my students, I don’t know how many times on,“You got to have a growth mindset. Don’t tell yourself you’re bad at something.” But then when I was trying to teach myself how to do Twitter, after two weeks I was like, “Oh, I’m terrible at this. Everybody else is so good. I just can’t do it.” It’s really hard to learn, and it’s easy to forget in our subjects because we’re so skilled at them.

Rebecca: Can you talk a little bit about what you mean by geeky pedagogy in general?

Jessamyn: Sure. So this is the first college teaching guide addressed to geeks, introverts, and nerds, which is pretty amazing because, damn. Like, look around you, pal. [LAUGHTER] I mean, academia is jam packed with us but it’s more than a gimmick. So at the heart of my argument is that geeks, introverts, and nerds as a group—and I’m generalizing, this isn’t every single person—but as a group, we face certain obstacles to effective teaching and learning. Obstacles like effective communication, building rapport, productive professional and social interaction. We’re highly successful academically and most of us who have not taught much before somehow believe that that’s going to magically translate into helping other people be successful academically, but it doesn’t, it’s not necessarily. That being said though, geeks, introverts, and nerds as a group bring important and necessary skills to learning and relearning how to teach effectively. We’re passionate about our subjects and we can draw on that passion to create what I call a geek culture of sharing pedagogy. So that is creating classrooms and instruction that invites everyone into the study of whatever crazy, arcane, esoteric topic we love with all our dorky hearts. Instead of acting as a geek gatekeeper… keep people out, prove how smart we are… geek culture of sharing says, “Come in! This cool thing we are learning about… I can’t wait to share with you.”

John: You have a chapter in your book on preparing for class, could you tell us some of the things you focus on in there?

Jessamyn: Okay, so my first recommendation for preparation is you’re going to read Geeky Pedagogy from cover to cover. [LAUGHTER] You’re going to follow me on Twitter @geekypedagogy. You’re going to visit my website, geekypedagogy.com, going live July 1st. But seriously, the first thing I would emphasize—my top recommendation for preparation—the thing I most want to share with my tribe of nerdy eggheads—is to think about teaching effectively as an intellectual activity. We have to use our big fat brains for effective teaching. All those geeky study skills we have, we have to apply it to teaching preparation. So, we have to do some research on pedagogical content knowledge, how to teach your subject. We have to think carefully about our syllabi, and prepare them in a timely way—do not procrastinate. That’s my one nag, [LAUGHTER] don’t procrastinate—we have to think through our experts’ blind spots, we have to read up on the science of the brain, how people learn, and we have to take into account that over and over again in our individual and unique teaching context, we’re going to be learning and relearning about effective teaching. Now in the book, it’s a narrative guide. So I don’t offer extensive checklists or step-by-step… do this do that, and then, “Tada! You’re an effective teacher.” Those can be helpful at times, I think maybe especially if it’s your very first class if you’re really, really nervous, but I want faculty drawing on—a lot of this may be specific advice I have in the book—but I want faculty to feel empowered to research their own specific teaching context to become what Stephen Brookfield called experts on our own teaching. Keep figuring out what worked, what didn’t work, reflect on it, repeat.

John: One of the things you mentioned is preparing for confrontation and conflict. What advice do you give faculty? or what types of confrontation and conflict do you address?

Jessamyn: We were just talking about the emotional aspects of teaching and learning. Teaching and learning include a lot of extreme emotions. I’m not the first. Stephen Brookfield, Maryellen Weimer both mentioned that teaching is a roller coaster, there’s a lot of ups and downs. Same is true for learning, there’s a lot of ups and downs. For different kinds of assessment and creating there can be conflict. Student incivility, which is a very polite term for things that sometimes are pretty egregious, it can happen. And this is all—it’s a tiny part of teaching—but it happens and it sucks up a disproportionate amount of our time and energy. Another example might be plagiarism. One issue I talked about is academic entitlement, a new and growing issue. The most important thing—and this is across all teaching contexts, even taking into account what Roxanna Harlow called disparate teaching reality—so even taking those into account, preparation is the one thing that can help mitigate any kind of conflict. Clearly conveying and communicating—not easy for introverts and nerds—but clearly conveying your expectations, being as transparent as possible, that’s the number one thing, preparing for those. And then the other thing I guess in that section, the most important point is that understanding as introverts… and not every introvert and nerd is socially awkward…. I am. I’m definitely socially awkward, smarty pants, that’s my persona. [LAUGHTER] Understanding that, for me, high-levels of emotion are hard to handle. And it’s definitely not easy for anybody in any workplace to deal with conflict and anger and strong emotion. But I think it can be especially daunting for introverts, for people who aren’t extra socially skilled. It’s hard to be right there in the face of extreme student emotion. Preparing for it with some scripts in your mind, not like endlessly rehearsing, “I’m going to say this to so-and-so,” but just having a kind of standard for plagiarism, this is kind of my standard script. For someone upset about their grade, this is some steps I do. Being as mindful and as present as possible in that moment tends to help as well, which is actually a strength for introverts is listening. So the research seems to be suggesting that the best way to defuse any kind of student conflict is for students to feel like they’ve been listened to, and that seems to matter more than what you actually do to resolve the situation. So there was one study that showed, for example, students might view the offer to do makeup work for something as either a positive or a negative resolution, fully depending on if they felt like the professor had been listening to their concerns. That’s great news for introverts and nerds, because it means it has everything to do with our communication skills, which we can do.

Rebecca: You mentioned reflective practice a little bit ago, can you talk a little bit about some techniques or ways that we can build that into our practice and then actually use the time that we reflect effectively?

Jessamyn: Right. Well, I’m glad you mentioned, those are two very different things, and neither one of them is easy but the second one is definitely harder. So applying the knowledge you’ve gained from reflection to your actual classroom practice is a lot harder. The number one thing to do and not do is don’t limit your pedagogical reflection to student evaluations. Too often, that’s the only feedback, and the only reflection faculty do about their teaching, and it’s insufficient. It can tell us some important things, but it’s not adequate on its own. I have some specific suggestions in the book. Things as simple as keeping ongoing notes throughout the term on your syllabus about things that are working or not working well. But I would say generally, I really want to encourage people to find reflective practices that engage you as part of your actual work of teaching and make sense for you. I was really aware that for some of us, mindfulness practices, yoga, we’re onboard. And then for some of us, even anything slightly new agey is not going to work at all. So my main recommendation is find reflective practices that help you reflect on what’s working, what’s not working, and then apply to your practice. An important part of that is thinking about reflection as something we do as individuals, but also part of a community of practice. We have to talk to other people about teaching. And sorry, introverts, you can’t do it just on your own. Academia doesn’t encourage it at all, you’re often going against the grain to try to talk about teaching, especially if you want to talk about our teaching mistakes—which are the most important way we learn just like our students—it’s the most important way we learn. But there’s so few opportunities, we have to really go out of our way to make those opportunities happen. And the one last plug I put in is for adding a gratitude practice to pedagogical reflection. And here I’m drawing on Kerry Howells’ book. It’s called Gratitude and Education: A Radical View. I want to emphasize this is not just positive thinking, it doesn’t mean ignoring the toxic aspects of your workplace—not that academia has any toxic aspects whatsoever, right? [LAUGHTER] Or injustice, or inequality, or anything going wrong—that’s not what gratitude practice means. It does mean being fully aware of and paying attention to every aspect of your teaching context. And Howells argues that our teaching context in her gift paradigm of education… as opposed to the consumerization model. In the gift paradigm, our teaching context always includes gifts, things we get no strings attached from students, from colleagues, and staff. I know people listening might be thinking, “Gifts? Give me a break. Like, I get jack squat every day from my frustrating students.” But I would counter with—pardon me while I super nerd out the wise words of Thorin Oakenshield from the Hobbit—he said, “There is nothing like looking if you want to find something, you certainly usually find something if you look, but it’s not always quite this something you were after.” So gratitude practice by opening up our view of our teaching context, we will find a gift.

John: Early in the book, you start off with, “Learning is hard,” and at the end, you conclude with, “Teaching is hard.” Could you tell us a little bit about that?

Jessamyn: Sure. So the last chapter is the shortest one. It’s called “Practice.” It’s the best news and the worst news you’re ever going to hear about teaching, is that you can get better with practice. So that’s great news, because it means we’re always learning how to do it better. It’s bad news because nothing can replace it. Fellow bookworms, there’s nothing you can read that will replace it. And if you don’t have employment security—like most of us teaching college, the majority of us teaching college are doing so on a contingent basis—if you don’t have employment security, that’s hard news. And it’s a vicious irony, that teaching effectively is so key to our employment and yet, the thing that will help us the most—being able to do it over and over—is dependent on our employment status. That very last section in chapter five is just an acknowledgement that when you’re working hard to be an effective teacher, it’s tiring, it can be daunting. There’s some real highs and lows. And to guard against burnout—to be aware of what you can and can’t do, and to really—I circle back and say again—fight that super teacher myth. Get that damn Robin Williams in Dead Poet’s Society, “Captain, my captain.” Get out of my head, get out of my students’ head. That’s not how teaching goes. Every once in a while you have a magical moment, but there’s a lot of grinding hard work. Most of us become effective teachers the same way our students learn how to do something, which is just slogging away at it day after ever loving day. That’s how you get better. That’s how you become effective. And it’s hard.

Rebecca: Earlier you talked about inviting students into our geeky spaces. Can you talk a little bit about how you invite students into your geeky spaces?

Jessamyn: Well, I’ll never forget the time—I described this in chapter five—I just stumbled on it. I made a joke in passing about my own geekiness about an article we were reading and I got this big student laugh. So student laughs, always good. But later, reflecting on it, I realized what made that so pedagogically effective. So first, there’s a lot of debate and discussion about what exactly a nerd is, what exactly a geek is, but the one thing we all agree on is they’re super smart. So when I said, “I’m a big nerd,” to my students, it was joking and yet it was also reinforcing my expertise and my knowledge. I know a lot about this, I am a big nerd. It also, I think one thing I talked about in the book is the importance of enthusiasm and that’s a difficult term for someone like me who’s pretty reserved—in many ways, an introvert—I’m not going to be a cheerleader. I’m not extra warm and fuzzy, I’m pretty intellectual. But I am passionate and I love the things I’m teaching, and when I position myself as the big geek in the room who can’t get enough of this topic, it helps me convey that enthusiasm to students. One of the studies that I cite in the book says a massive survey of students who said they perceived a teacher as authentic when the teacher was happy that class begins. That’s a tough one for introverts because part of me is always going to be back alone in my office doing my research or my scholarship, whatever gets me going as a scholar. But by embracing my nerdy love for my subject, I’m able to convey to students, “I am happy when class begins.”

Rebecca: Before we started recording, you were talking about zombies in your class. [LAUGHTER]

Jessamyn: Yeah.

Rebecca: Could you talk a little bit about some of the things you’d like to nerd out on?

Jessamyn: Sure.

John: Or about that class in general.

Jessamyn: I would say—let’s see—I’m going to put it in the teaching context. So the thing that I most love about teaching my subjects is getting students to rethink something they thought they knew about popular culture. And everybody arrives in my classes, knowing a lot about popular culture in many ways. They know how to watch a movie, they know how to play a video game. And so getting them to rethink those things is what is most rewarding to me as a teacher, I sort of joke about it but sometimes when students will say, “Professor Neuhaus, you ruined such and such for me,” like some movie or some TV show like, “Now I can’t stop thinking about it.” That’s so rewarding for me as a professor. Being able to have students apply some pretty abstract cultural studies work to their real lives, that’s what makes me happiest as a professor. Every once in a while a student will say, “That documentary we saw, it was so interesting, I forced my roommate to watch it,” or, “We looked it up online, I wanted my dad to see it.” That’s like a microphone drop for a professor. I figure if they’re talking about it for no reason except they were interested outside of class, then I’ve definitely done my job. For me personally nerding out, a big chunk of it has been on the history of gender and prescriptive gender norms, prescriptive literature. So my first book was about cookbooks and gender, my second book was about advertising and housework. And I’ve written articles about sex-manuals, classroom films, instructional films, high-school instructional films, and I could talk about those things for hours and hours.

Rebecca: Will that be geeky pedagogy as well, right?

Jessamyn: Yes, that’s right. [LAUGHTER]

John: What strategies do you recommend for faculty trying to improve their teaching and where can they find assistance?

Jessamyn: You got to nerd out about teaching and learning. We already know how to nerd out about our topics, ask anyone teaching a college class “Tell me about this subject” and watch their little faces light up, hear their voice get animated, no matter how arcane or obscure. Donald Glover said “Strange, specific stuff; That’s what makes a nerd a nerd.” And that’s what makes academics, academics. And I’ll just use as an example, my good friend and colleague—he’s a historian of industrialization, Jeff Hornibrook, at SUNY Plattsburgh—he spent almost 20 years studying a single coal mine in China. A hole. A hole in the ground. But as he explains in his book, A Great Undertaking, this hole can tell you so many interesting things. That’s an academic nerd for you. We can apply that same focus and ability to studying, teaching, and learning. And just like we’re always learning about our topic, we’re always going to keep learning about teaching. So resources for that definitely SoTL—scholarship on teaching and learning—keeping in mind anything you read you have to apply in your specific, unique, individual context. I also think, probably the most immediately effective resource is your campus teaching and learning center. I’ve yet to speak to a single person who’s ever had anything negative to say about their experiences at a teaching and learning center. If you have access to one, if there’s one on your campus, it’s the very first step you should do for any kind of support and for resources. I’ve also learned a ton about teaching from teaching conferences, which I think are a totally different world than academic conferences. Academic conferences, you’re supposedly there to share knowledge but really it’s about proving you’re smarter than other smart people, in my humble opinion. Teaching conferences, I really do see people trying to share knowledge, and like in the book I say is like, “The mothership come home,” you’re surrounded by other nerdy people who want to learn about teaching. And I’m also going to put—going to say—academic Twitter. I’m going to say that. I only joined Twitter for recently. Thanks, marketing team West Virginia University Press, [LAUGHTER] they really said, “You should think about doing this,” and I did, hoping to get the word out about my book. But, just very surprisingly, I found that it has significantly expanded my pedagogical community of practice. Of course, it has significant limitations. 280 characters, that doesn’t leave a lot of room for citations as citations nerd like me. But it does provide a key component of reflection and improvement for teaching practice, which is talking to other people about teaching. And especially for me—as an introvert—I’ve always been terrible at networking. And I teach at a very small, very rural, and isolated university, and Twitter has really expanded my ability to hear what other people are doing in the classroom. And also, it really does, in many ways, work to de-center privilege in discussions about teaching and learning. It really is a platform for voices from across different employment status, teaching contacts, identity, and so on.

John: One of the barriers I think that a lot of people have with academic Twitter is when they first sign up for it, it doesn’t seem to offer much benefit until they start following enough group of people. So it takes time to develop that personal learning network there to make it more useful. And it’s worth taking the time to do that, but it doesn’t have that immediate feedback that many other types of social networks perhaps may.

Rebecca: But you can take your time to compose your interactions and not be caught on the spot. So in that way, it’s really wonderful. [LAUGHTER]

Jessamyn: Yeah, it is. It’s kind of a boon for introverts.

Rebecca: What are some ways that your book differs from other books on teaching and learning?

Jessamyn: Well, it’s fun to read. [LAUGHTER] That already sets it apart. I would call it SoTL with a side of snark. So in other words, it’s real scholarship. I mean, it’s so packed with citations, I had to cut out a bunch and I’m going to have to make that available as supplemental bibliographies online. So it’s real research, there’s all kinds of scholarly resources in there. But I’m allergic to jargon, and pomposity, and I really like to make people laugh so the book is highly readable. It’s multidisciplinary. I’m a multidisciplinary scholar so it was easy for me to move beyond the rigid dictates of studying history—that’s my discipline—and it takes into mind the real variety of faculty who are looking to become effective teachers with all kinds of different individual teaching contexts, employment status, embodied identity. It’s highly readable in a narrative style and it’s written by someone who who doesn’t take herself unduly seriously, and someone who can acknowledge the roller coaster of teaching and learning.

Rebecca: I’m really looking forward to checking out when it comes out in September, right?

Jessamyn: Yes, September 1st.

John: And we will include links to everything we’ve referred to here in the show notes.

Jessamyn: Great, thank you.

John: We always end our podcast with the question. What are you doing next?

Jessamyn: I guess you mean after I’m done with my worldwide speaking tour…

John: Right.

Rebecca: Clearly.

Jessamyn: ….when Geeky Pedagogy becomes an international bestseller. [LAUGHTER]

Rebecca: So it’ll be years from now.

Jessamyn: Yeah, years from now. After I convince all intellectuals everywhere to nerd out about teaching and learning. What I’d like to next focus on is de-centering privilege in SoTL. I’m not sure what form this will take yet. I’m considering maybe like an edited collection. I’m thinking specifically of practical pedagogical strategies for underrepresented and marginalized faculty. There are a lot of excellent books and articles, anthologies, scholarship, and reflection about disparate teaching realities—that’s Roxanna Harlow’s term—but what’s needed now, I think, is building on that for some practical suggestions and guidance for increasing pedagogical content knowledge when you don’t look like a professor.

Rebecca: I look forward to seeing what form that takes, it sounds really interesting.

Jessamyn: I’ll be back to talk about it on Tea for Teaching.

Rebecca: Yeah. [LAUGHTER]

John: Well, thank you very much.We’re both very much looking forward to your book and hearing more about it.

Jessamyn: Thank you. Thanks for having me.

Rebecca: Yeah, we appreciate the time you spent with us today.

Jessamyn: Thank you.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson. Brittany Jones, and Gabriella Perez.

54. SOTL

Posted in assessment, evidence-based practices, higher education, learning, pedagogy, reflective practice, scholarly teaching, scholarship of teaching and learning, teaching, teaching centers
November 7, 2018

As faculty, we face a tradeoff between spending time on  teaching and on research activities. In this episode, Dr. Regan Gurung joins us to explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.  Regan is the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research.

Show Notes

Show Notes

John: As faculty, we face a tradeoff between spending time on teaching and on research activities. In this episode, we explore how engaging in research on teaching and learning can help us become more productive as scholars and as educators while also improving student learning outcomes.

[MUSIC]

Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Regan Gurung, the Ben J. and Joyce Rosenberg Professor of Human Development in Psychology at the University of Wisconsin at Green Bay; President-Elect of the Psi Chi International Honor Society in Psychology; co-editor of Scholarship of Teaching and Learning in Psychology; co-chair of the American Psychological Association Introductory Psychology Initiative and the Director of the Hub for Intro Psych and Pedagogical Research. Welcome.

John: Welcome.

Regan: Thanks a lot, Rebecca and John.

John: Our teas today are…

Rebecca: I’m drinking Prince of Wales today.

Regan: Alright.

John: I’m drinking ginger tea.

Regan: Ooh, now you’re making me want to. [LAUGHTER]

John: We’ve invited you here today to talk about research in the scholarship of teaching and learning, or SOTL. You’ve conducted a lot of research on teaching and learning as well as research within your discipline. In most disciplines there has been an increase in the journals devoted to teaching and learning and an increase in research in teaching and learning, but it hasn’t reached everywhere yet. SOTL research is often not discussed in graduate programs and is sometimes devalued by campus colleagues. Why does that occur?

Regan: So. I think there are multiple reasons why the—and I’m going to start with the devaluing. I think there’s a lot of uncertainty about what it exactly it is, so on one hand, when people say a scholarship of teaching and learning… very often if it’s somebody who hasn’t really read up on it recently the sense is, oh, you know, that’s research on teaching; that’s not as good as your regular research. Now, I think that’s a misperception and once upon a time, and here I mean maybe even 15 years ago, there was some scholarship of teaching and learning that wasn’t done very well and I think people have heard about that in the past and that’s why there’s that knee-jerk reaction. Far too often it’s seen as something where it’s not as rigorous, perhaps, or it’s not done in the same way and most of that is wrong. What I like to tell folks who see that is, if you think the scholarship of teaching and learning is not rigorous, well, you haven’t tried to submit something to a journal recently. I co-edit a journal on the scholarship of teaching and learning in psychology and I can actually see some people submit poor work and I send it right back; I do the classic desk rejection and I say, look, this is just not good enough. So my favorite tip for “How do you write for a scholarship of teaching journal?” is very simple: just like you write anything else. There’s a lot of baggage, but I think that as you alluded to, John, it has changed more recently and I think part of what you notice now or what I’ve been seeing is that this kind of work, this kind of examination is being called different things. For example, a term that I’m hearing more and more often is DBER: disciplinary based educational research. And I’m hearing this come out of medical schools and engineering schools and social work schools and many professional programs where they’re doing DBER, which is essentially what the scholarship of teaching and learning is. So, I think because of that baggage with the term, people are calling it different things but in general the work is getting much more rigorous.

John: Excellent, and if changing the name is sufficient to do that, it’s a valuable step.

Regan: I think that’s why, when I talk about it I like to talk about it as: “Do you want to know if your students are learning? Do you want to know if your teaching is effective?” Well, then you should do some research on it. You can call it what you want. I started really calling it pedagogical research because that’s what it was, but it’s truly a rose by any name.

John: And that’s something that Carl Wieman has emphasized.

Regan: Absolutely, yup.

John: In the sciences, you test hypotheses and there’s no reason we couldn’t do the same thing in our teaching.

Regan: Exactly.

John: And that’s starting to happen, or it’s happening more and more.

Rebecca: In some disciplines, the scholarship of teaching and learning is not accepted as being part of their tenure and promotion file, for example. What would you recommend faculty do in a department like that if they really want to get started in SOTL?

Regan: Well, so, Rebecca, let me take you a half step back.

Rebecca: Yeah.

Regan: When you say “in some disciplines it isn’t as accepted.” What has surprised me is that most disciplines have actually been doing the scholarship of teaching and learning and publishing it for the longest time. I mean, if you take a look at chemistry, it goes back, gosh, seventy years or so. Almost every discipline out there has a journal that publishes the scholarship of teaching and learning, but, and here’s the big but: most of us in our normal training never run into it. So, I’ll take my own case. In psychology, the Teaching of Psychology Journal has been around for 46 years, yet all through grad school, all through my post-doc I never even knew the journal existed. Why? Because the programs that I went through weren’t focused on teaching the individuals—wonderful as they may be—who I worked with didn’t do that kind of work, so they didn’t know about it. So I think that’s a really important fine-tune there: there is a journal in almost every discipline—almost every discipline—for the scholarship of teaching and learning. So, it’s just a question of discovering it… it’s a question of finding it. Now, that said, where can they start? I think I can answer your question from a conceptual level and from a practical level, so I’ll start with the practical. The easiest place to start, there are lots of compilations of how to do it. For example, I think both of you have my website. On my website I have a simple tab called SOTL. On that tab is a list of places to get going, and I’ve organized it so that there’s a brief introduction to SOTL, there are journals, there are resources, there are little handouts. So, if a faculty member has even ten minutes, go to my website, hit SOTL, scroll through. That’s the more practical, that’s the easiest way to get started. From a conceptual standpoint it really starts with the question, what aspect of your teaching or your student learning are you curious about? John, I know you do some work in large-class instruction in economics. Why is this assignment not working? Can I get my students to remember certain concepts better if I change how I present information? It starts with a question. And you don’t have to read anything, you don’t have to look at any manual. If you look at your class and you go, “Hmmm, why isn’t this working, or why isn’t that working?” That’s where it begins, and from there you follow the same route that we always do: go look at what’s been published in it, fine-tune your question, design, think about what do you want to change and so on and so forth. I think it’ll help if I give you my working definition of the scholarship of teaching and learning, and when I think about it I think of SOTL as encompassing those theoretical underpinnings of how we learn. And more specifically, I see it as the intentional and systematic modifications of pedagogy and here’s the important part: the assessment of the resulting changes in learning. So that’s the key: you intentionally, you systematically, modify what you’re doing and then you measure whether it worked or not. That’s it. I could say that nonchalantly. There’s a technique , there’s a robustness to it, but at the heart, where do you start? You start by asking the question.

Rebecca: I think one of the things that I hear you saying is not much different than someone has a really reflective teaching practice—they’re doing it but not in that systematic way?

Regan:Yeah, absolutely right. There’s a term called scholarly teaching and in this kind of literature there’s a distinction made between scholarly teaching and the scholarship of teaching and learning, and all the distinction is is that scholarly teacher is reflecting on their work and then you’re right, you’re absolutely right; making those intentional systemic changes. That’s scholarly teaching. When it becomes the scholarship of teaching and learning is when you present it or you publish it, preferably through peer-reviewed ways, but you’re absolutely right; at the heart of it it’s scholarly teaching. It’s reflective intentional systematic changes.

John: One of the barriers, that people who are considering doing research in the scholarship of teaching and learning, is going through IRB approval, and in many disciplines that’s something they haven’t experienced before. It’s common in psychology. It’s less common in economics and perhaps in art.

Rebecca: It doesn’t exist in design. [LAUGHTER]

John: Could you tell us a little bit about that process?

Regan: Sure. Every university has an institutional review board and essentially what that board does is it’s in place to make sure that any research that’s being done isn’t harmful. Now, normally when we think about harmful we think about a drug or a food substance being tested, but here it just means any research that’s being done, and so when you do the scholarship of teaching and learning or when you’re examining your classes, yes, you could just look at your exams and see if exam scores are changing, but, if you do want to publish that, if you do want to share that, you really should go through institutional review board review. Now, the key thing here: it does sound like this whole new world, and it is, but at the heart of it is a very simple process. Now, there are three levels of review and I think knowing about the levels helps. For example, the first level is called an exempt review. The next level is called an expedited review, and the third level is called a full board review. I don’t think I’ve run into scholarship of teaching and learning that has gone through a full board review, because we’re not doing things that are more than minimum level of stress. Now when you say, hey, hang on, I didn’t know they were stress involved. Well, anytime you ask anybody to fill out a survey, there’s a minimal level of stress. And when you’re asking your students to reflect on their learning, well that’s a minimum level of stress. Every university has its own procedure. SUNY Oswego probably has a forum online. It’s a short forum; you’re basically telling this board what you plan on doing, what you plan on doing with the information, and most importantly, in these kind of cases, you are letting the board know whether or not students will be put under duress. What the IRB is going to look for is are you the instructor in some way forcing your students to do things that normally wouldn’t be done in the normal course of the educational process. But at the heart of it, all you’re doing is you’re sharing with this board whether or not you can do it and most scholarship of teaching and learning is at that exempt level. That exempt level essentially translates to exempt from further review. It doesn’t mean exempt from being reviewed; it just means this is mundane and low stress enough that it’s exempt from further review. Now that second level, expedited. If you do want to measure or keep track of names, if you want to look at how certain names relate to scores down the line—and that’s actually some really key research—well that’s expedited review. Now, even there it’s reviewed by one person. Both the expedited and the exempt review are reviewed by one person, often the chair. It often takes no longer than a week, and by doing that you just know that all your t’s are crossed and your i’s are dotted and it’s the ethical thing to do. So, whenever people say: “Oh, this is really mundane and I’m not really doing much more than just measuring student learning,” I still sa y if there’s any chance you want to present it or publish it make sure you go through the IRB.

John: And many journals will require evidence of completion of the IRB process.

Regan: Oh, absolutely. The moment you want to publish it you have to sign off saying that you got IRB review..

John: We do use an expedited review process on our campus. I was going to say, though, that we’re recording this a bit early because we’ve recorded a few things in advance, so we’re recording this in late October, but just yesterday I read that Rice University has introduced a streamlined expedited review process or IRB and apparently that’s something that’s been happening at more and more campuses. Are you familiar with that?

Regan: You know, not as much, because right now there’s so much up in the air with the IRB because national guidelines are changing. They were supposed to have changed in January, then it was moved to July. The latest I heard is it’s moved to next January. So, for the most part actual regulations are changing. Even on our own campus we switch from one form of human subjects training to another form, but this so called short-form expedited process will definitely help. That said, even the regular expedited, it’s a very easy process and I think the neat thing about this—and I tell students this when I’m teaching research methods, too—as the instructor or the researcher, just going through that IRB form really reminds you of some key things that you may have otherwise forgotten about, so, yes.

Rebecca: Do you talk a little bit about your own research to give people an overview of what project might look like from the beginning to the end?

Regan: Sure. What really got me interested in this is I teach large introductory psychology classes, the class is 250 individuals and I was struck by how when publisher reps come into my office and try to convince me to adopt one book over the other they would talk about the pedagogical aids in the textbook; “oh, look, our book has this and our book has that.” And that really got me started studying textbooks and how students use textbooks. So the umbrella under which I do research is student studying: What’s the optimal way for students to study? …and I use both a social psychology and a cognitive psychology lens or approach to it and it really started with looking at how they use textbook pedagogical aids. So, for example, in one of my really first studies I measured which of the different aids in a textbook the student uses and then I used their usage to predict their exam scores. Now, what I found, and this is what really surprised me and got me doing this even more, is that even those students were using and focusing on key terms a lot. Now, mind you, I’ll take a half step back—you may not be surprised to know that students use bold terms, they use italics, that’s what they focus a lot on. But students in my study also said that they use key terms a lot. Now if you’re studying key terms that should be good. If you’re making flashcards and studying those key terms that should be good, but what I found is that the more students use key terms the worse their exam scores. There was this negative correlation and that’s completely counterintuitive. Why would they go the opposite direction? So, I dug into it some more and I realized that students spend so much time on key terms or so much time on flashcards that they’re not studying in any other way. So even though they’re using flashcards, they’re so intent on memorizing and surface-level processing that they’re not doing deeper level processing. So, that was some years ago and I’ve been building on that, trying to unpack how students study. My most recent study… that’s actually under review right now… a colleague, Kate Burns, and I took two of the most recommended cognitive psychology study techniques, which is repeated practice or testing yourself frequently and spacing out your practice or spacing out your studying, and we took both of these and across nine different campuses divided up classes such that the students in those classes were either using high or low levels of each of these. So, in one study across multiple campuses we tested is there a main effect of one of these types of studying or is there an interaction? And what we found is that there is an interaction and the critical component seems to be spacing out your studying. Not so much even repeating your studying, but really spacing out your studying, and I think what’s interesting here is the reason this is happening is the students who said that they were testing themselves repeatedly, that sounds great, and if you’re a cognitive psychologist you say, hey, the lab says repeat testing is great; the problem is in the classroom a lot of students who were repeatedly testing themselves were repeatedly testing themselves during a really short period of time.

John: Right, I’ve seen that myself.

Regan: And I think that’s the issue, but because we had both these factors in the study, we could actually tease that out. So that’s the kind of work that I do… is take a look at what the cognitive lab says is important; let’s see how it works in the actual classroom.

John: Now was this a controlled experiment? Or was this based on the students’ behavior?

Regan: So, yes and no, okay. [LAUGHTER] I love this study because of a number of reasons. Number one, we tested two different techniques in the same thing. Number two, we did it at multiple institutions, so it’s not just my classroom. A lot of SOTL is one class. So, here we went beyond to try and generalize. But, to get to your question, we actually used a true experimental design. So we recruited these different campuses and we assigned a classroom. So, for example, I’d say, “Hey John, thanks for taking part. If you can have your students do high repetition and high spacing?” “Hey Rebecca, thanks for taking part. Could you have your students do high repetition and low spacing?” And that’s how we spread it out. We had about two campuses in each of these cells. That’s the true experiment on paper. To get to the other part of what you said… in reality, that’s not exactly what students always did. And you know students; we can tell them to do something but a whole bunch of things gets in the way. Fortunately, of course, we measured self reports of what students said they actually did and it was relatively close to the study cells, but even though it varied a little bit we could still control for it. So, yes, it was close to a controlled study as much as you could control something in the real world across nine campuses.

John: That brings us to the general question of how you construct controls. Suppose that you make a change in your class; how do you get the counterfactual?

Regan: Right.

John: What would be some examples for people designing an experiment?

Regan: The word control, especially in research, has the true connotation of the word control group and that’s controlling for factors as different from having a control group. Optimally we’d love a control group. The problem with the control group is that it means no treatment. So, very often a true control group means this group of students is not getting something. From a philosophical and an ethical standpoint, I don’t like the notion of one group not getting something. So, the word I like to use is comparison group. So, your question still holds, but what’s the comparison group? I think here’s where if you’re fortunate enough to teach multiple sections, well one of the sections can be the comparison group. If you’re not fortunate enough to have multiple sections, you compare the students this semester with the students the last semester when you weren’t doing that new, funky innovation. So, there are a bunch of different ways to gather the comparison group, but you’re absolutely right: having a comparison group is important. Most commonly in scholarship of teaching and learning, the comparison is the students before that intervention, so it’s a classic pre- and post- measure. I’ll give you this quiz before I’ve introduced the material, I give you an equivalent quiz after, let’s see if there are changes in learning. And that’s the most common comparison; you’re comparing them with them before but optimally again you want a different section, you want a group of students, a different semester, or so on, and so on.

John: And it’s best if you have some other controls…

Regan: Absolutely.

John: for student ability and characteristics.

Regan: You nailed one of the key—my two favorite are effort and ability. As much as possible, measure their GPA. If they’re first-year students, measure their high school ACT scores or their high school GPA and then you have to measure ability, and I think those two are probably the usual suspects for control. And again, a lot of SOTL doesn’t do that and it should.

Rebecca: I think one thing that comes up a lot for me (and maybe some others who are in disciplines maybe more similar to my own) is that the kind of research that we do is not this kind of research generally, but we’re really interested in what’s happening in our classrooms. So, for faculty who might be in the arts or some other area where we’re doing really different kinds of research, how would you recommend being able to partner or do this kind of work without that background?

Regan: And I think implicit in your question is the “Do I need to have a certain methodological tool bag?” and I remember I was at a conference once and somebody accosted me and said “Hey, is it true that you have to be a social scientist to do this work?” And the answer is no, and I wrote a pretty funky essay called “Get Foxy,” which is how social scientists can benefit from the methodologies of the humanists and vice versa. But, you’re right; you can collaborate if you need to do that kind of work, but there are a lot of questions even within your discipline… and when I think about SOTL I think about answering questions about teaching and learning with the tools of your discipline. Now, I’ll give you an example: a good friend of mine was an art and her project, or something that she wanted to dig into, was to improve student critiques in an art class. Here we have students learning how to do art (and I think it was drawing or jewelry making) and across the course of the semester everybody had to present their work and then critique each other’s work… and those critiques, they just didn’t have the teeth that she wanted them to, so she was giving them skills and how to do it. So here’s a case of how did she know whether or not the critiquing tools were increasing? Well, she came up with a simple rubric and to score them against and look at if the scores changed. Now, you may say, well, we very often in the arts and theater you don’t get skills to do that, which is true, but that’s where I think collaboration comes in and that’s why what’s really neat about scholarship of teaching and learning is very often there are class collaborations. I have a historian on my campus who wanted to change the quality of his essays and he and David Voelker changed how he was teaching and wanted to see it roll out and had students on their essays use teams in a different way. Well, he compared, and John this goes back to your point, he compared essays from before the change with essays from after the change, counted up the number of teams students had and then, Rebecca, to your point went over to my colleague in psychology and said, hey, can you tell me if this is statistically different. So, he didn’t even bother with doing the stats; he just said, “Hey look, I don’t need to do the stats.” But you can, in a click, and literally within minutes my colleague in psychology had done the stats for him. I think that’s the kind of stuff that can happen to truly get at those answers if you go, “You know, I don’t know how to do that.” But, you’d be surprised… the basic skills for SOTL can give you enough to test questions pretty well.

Rebecca: I think John and I have also found in the teaching center that it’s really exciting when faculty from different disciplines start talking about their research when they’re looking at learning because there’s things that we can learn from each other and the more that we’re talking across disciplines can be really valuable as well.

Regan: Right, and I think this is where reading the rich literature that exists in your discipline or even across disciplines on scholarship on teaching and learning really gives you the leg up, because I find now when I do workshops and somebody says, “You know, I’ve got this question; I don’t know how to start.” More often than not it’ll remind me of a study that I can say, hey, here’s what you can do. And it’s just because I read a lot and I’ve got all that in my head and I just matched to that question and it’s pretty easy. I mean, very rarely do we have to invent something from scratch. We go, “Hey, yeah, you know what? Here’s the study that’s pretty close to the question you have, let’s use that methodology.”

Rebecca: So, how do we build a culture of the scholarship of teaching and learning—the departments who might have faculty who are resistant to the idea of their colleagues spending their time doing that? How do we start changing minds and really building a culture that embraces the idea of the scholarship of teaching and learning?

Regan: Well, I think you’ve got to attack it from two different levels. You definitely want a champion in the administration who is educated enough about the scholarship of teaching and learning and how it can be done robustly. If you can convince somebody of it’s worth and then if you go “How do you do that?” …well that’s where you need to make sure you have at your fingertips, as a teaching and learning center, the exemplars of really robust work… and I think if you have that really robust work at your fingertips, that’s definitely a key place to start. One of my favorite examples along those lines of trying to convince (especially administrators) about the worth of scholarship of teaching and learning, I recommend a 2011 publication by Hutchings, Huber, and Ciccone, it’s called A Scholarship of Teaching and Learning Reconsidered and this 2011 publication is a great collection. It does your homework for you. That one book pulls together evidence for why scholarship of teaching and learning helps students, helps faculty, helps institutions. So that’s where the top down—get your administrators to check that book out and go, “Oh yeah, look, there is actually some good research.” Coming at it from the other angle—I know this for a fact—there are people on your campus doing some of that work, but often they may be isolated, they may be a small group. You want to strengthen them so that they can spread that to their circles, and that’s really how it starts. On my campus, when Scott was the Dean at Green Bay, we did a lot to develop scholarship of teaching and learning through the teaching center. There was one year where we had 14 faculty who got together every month and talked about their projects. Now you may say, well, that’s 14 and you had 160 faculty. You know what, you do 10 of working every year and colleagues see the value of the work those 10 or 14 are doing, pretty soon you’re gonna have a culture where people recognize it more and appreciate it more. So I think that’s how it goes… you put your efforts on those people who are already doing it to make them stronger and that’s gonna spill over and pretty soon you’re gonna win over folks.

John: We generally had support from the upper administration and there’s often been a lot of faculty who are new, interested in doing it; it’s usually the promotions and tenure committees that have served as a barrier in some departments, but we’ll work on that and we need to keep working on that.

Regan: Well, just along those lines on our campus we felt so strongly about the scholarship of teaching and learning that the Faculty Senate actually passed a resolution recognizing the importance of scholarship of teaching and learning. Now again, it still gave department chairs some leeway, but at least the faculty voted on it as something that the university values and that goes a really long way to having especially junior faculty say, you know, I can do this.

Rebecca: Certainly makes faculty, especially junior faculty, feel supported when the Senate is saying, “Yes, we believe in this” and it’s not just one person saying we don’t.

Regan: Absolutely. And they’ll be naysayers. We started off this conversation with “There are people out there who think it’s not good enough” and there are people out there but I’ve had conversations with such people on my campus where sharing some information, sharing things about how it’s done goes a long way towards changing minds.

John: In my department, it’s helped that I’ve been the chair of our search committee for a few decades now. We’ve generally hired people who are interested in this, but that’s not the case in all of our departments yet, but we’re hoping that’ll change. For those who have small classes or may not be interested in doing research in their own classes, one other option is meta-analysis. Could you talk a little bit about that?

Regan: So meta-analysis, where one study is taking a look at a lot of different studies, there is the mother of all meta analyses… is one that we should talk about because I think the interested person can run to it. John Hattie, now at the University of Melbourne, did a meta-analysis where actually he did a meta-meta-analysis; took 900 meta analyses and then synthesized the data from those 900 studies that had already synthesized data, and the reason I like talking about that is the sample size when you take all those 900 meta analyses is a quarter of a billion with a “b”; that’s a lot of data points, it’s a lot of students. And what’s neat about meta analyses is that instead of just being one study at one place it’s now multiple studies over multiple contexts, and if you can find an effect over multiple contexts, that’s really saying something because a lot of single studies are so geared into the local context of where that place is that if you run into a meta analysis, so even if anybody listening pulls up an educational journal or an SOTL journal and sees meta analysis in the title, I would spend more time reading that one because it’s gonna be more likely to generalize from that. So, I think it’s statistical and methodological advances now mean that there are more meta analyses around and more meta, meta analyses around as well.

Rebecca: As an advocate for the scholarship of teaching and learning, where do you hope the scholarship of teaching and learning goes in the next five years?

Regan: Honestly, I think it should be a part of every teacher’s repertoire. When I think about a model teacher, and it’s not just when I think about it—I’ve published on evidence-based college and university teaching and when my co-authors and I looked at all the evidence out there and what makes a successful university teacher… one of those components, and we found six… I mean, it wasn’t just student evaluations, no, it was your syllabi, it was your course design, but one big element was doing the scholarship of teaching and learning… and to answer your question, I think if in five years from now we can see it be part of teacher training to look at your class with that intentional systematic lens, I think that’s where the field needs to get to.

John: At the very least it would get people to start considering evidence-based teaching practices instead of just replicating whatever was done to them in graduate school.

Regan: Absolutely. People would be surprised at how much good SOTL there is out there, and I always like sending folks to the Kennesaw State Center for Teaching and Learning where they have a list of journals in SOTL in essentially every field. You will scroll through that list for ages and it is just mind-boggling to realize that, “Wow, SOTL has been going on for a very long time.” And Rebecca, you mentioned art and performance arts and theater and music… not as much, but even there there is a fair amount and I think it’s just a question of getting folks making those resources more available to individuals and that’s why whenever I interact with teaching and learning centers I have a short list of key resources to look at. And again, that’s on my SOTL link. But, even that small list is an eye-opener to most people who never knew this existed, and I think once they realize it’s there they will start seeing it everywhere and once you start doing it it really energizes you. For those of us who’ve been teaching for 20-plus years to look at our classes with that new eye of how can I change something, how can I make it better and then seeing the positive effects of those changes, that’s invigorating.

Rebecca: I’m energized after having this conversation.

Regan: It is good stuff.

Rebecca: Yeah.

Regan: I just got back from a three-day conference and all we did was sit around and talk about cool SOTL. And you’re right …came back and sitting on the plane I was texting people with study ideas to collaborate on. It was that exciting.

Rebecca: The more you talk… collaborate… the more it happens.

Regan: There you go.

Rebecca: So, we always wrap up by asking, what’s next?

Regan: You know, I think I like getting the bang for my buck and you mentioned this in the intro: right now I’m working on the American Psych Association’s Introductory Psychology Initiative and what’s next is basically two years of really focusing on the introductory psychology course. It’s taken by close to a 1.5 million students a year and I’d like to make sure we can make that course the best learning experience for our students as possible, so that’s where my energy is gonna be for the next little bit.

John: That’s a big task and a very useful one.

Rebecca: And definitely worthwhile. Well, thank you so much for spending some time with us this afternoon. it’s been eye-opening and exciting… energizing. I can’t wait to look through some of the resources.

Regan: You know, is there anything else that you’d like, get in touch and I welcome anybody listening to get in touch as well.

John: Thank you, and we’ll share links to the resources you mentioned in the show notes.

Regan: Sounds good.

[MUSIC]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.

[MUSIC]

53. Teaching faculty

Posted in graduate education, higher education, pedagogy, professional development, scholarship of teaching and learning, teaching
October 31, 2018

How do faculty learn to teach? In many graduate programs, the emphasis is on research and publications—yet, many of these graduates end up in teaching positions. In this episode, Kristina Mitchell and Whitney Ross Manzo join us to discuss the structures and incentives that undermine good teaching and explore ways to help grad students and new faculty prepare for their careers in higher education. Kristina Mitchell is a faculty member and Director of the Online Education Program for the Political Science Department at Texas Tech. Whitney Ross Manzo is an assistant professor of Political Science and the Assistant Director of the Meredith Poll at Meredith College in Raleigh, North Carolina.

Show Notes

Transcript

John: How do faculty learn to teach? In many graduate programs, the emphasis is on research and publications—yet, many of these graduates end up in teaching positions. In this episode, we discuss the structures and incentives that undermine good teaching and explore ways to help grad students and new faculty prepare for their careers in higher education.

Thanks for joining us for “Tea for Teaching,” an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

Rebecca: Our guests today are Kristina Mitchell, a faculty member and Director of the Online Education Program for the Political Science Department at Texas Tech, and Whitney Ross Manzo, an assistant professor of Political Science and the Assistant Director of the Meredith Poll at Meredith College in Raleigh, North Carolina. Welcome, Whitney and welcome back, Kristina.

Whitney/Kristina: Hi, thanks for having us.

John: It’s good to have both of you here. Our teas today are…

Whitney: I’m actually drinking water.,

Kristina: I have my usual Diet Coke.

[LAUGHTER]

John: …as on two previous episodes. One of our most popular episodes, by the way, the one on gender bias and course evaluations is in our top three, I think.

Rebecca: Yep.

Kristina: Great!

Rebecca: John, how about you?

John: I am drinking ginger green tea.

Rebecca: And I’m drinking my usual. [LAUGHTER] English afternoon tea once again.

John: I was gonna ask you if it was Dragon Oolong

Rebecca: Yeah, yeah. Sounds like most of us are drinking our usuals, huh? So it’s not uncommon to have conversations about job preparedness and transitioning from student to professional and undergraduate education. What is more uncommon is challenging whether or not PhD or other advanced degree programs prepare students for the work that they will be doing like both of you did in a recent Inside Higher Ed article. What do you think prevents us from having that conversation?

Whitney: I think that there are incentives for professors at R1 universities to recreate themselves. So, I think that it’s an uncomfortable conversation for many R1 professors to even want to start because they probably don’t want to look too closely at this issue because I think to do it correctly might call for a pretty radical change in how we structure a lot of grad programs.

John: Is this because the prestige of the institution or the department is tied to the placements in our one universities and the publication record of the graduates?

Whitney: Yeah, very much so.

Kristina: I definitely think so as well. I think that a lot of times when PhD programs are advertising their programs to potential students, one of the things that students want to see is what kind of placement their graduates are getting. So programs that place their students really well attract the top graduate students and those graduate students, because they are great graduate students that have chosen this program, they get good placements which then continues to attract the best graduate students. So it can be really difficult for sort of mid- to low-level PhD producing institutions to attract good students because of this sort of self-perpetuating cycle. But the incentive is still there to try and compete with the top-tier institution that each faculty member who’s sponsoring a PhD graduate, they want their student to get the highest placement possible which means training them like a researcher.

John: Because we don’t have a similar sort of ranking system in terms of teaching productivity.

Kristina: Not at all.

Whitney: Exactly. We could have a whole conversation about how there should be two kinds of professors at R1 universities: the research professors and the teaching professors. But in the current climate in academia, teaching professors are not considered as prestigious as the research professors for a lot of the reasons that Kristina just outlined. So there’s no reason for an R1 professor to learn how to be a good teacher or to keep up in current pedagogy because what they need to be keeping up on is current research practices.

Rebecca: One of the things that the two of you outline in your article is that there’s a very small subset of people earning PhDs who actually go into R1s and do this academic research. But most of them end up in positions where teaching is a big part of their workload. But as we just mentioned, very few of them have been trained to be teachers. So why do you think there’s such a disconnect other than this prestige piece? Is there anything else to the puzzle?

Whitney: Well, I would argue that many R1 professors, because of the research incentives, haven’t really learned how to be good teachers themselves. So they might feel as though they don’t have the qualifications to teach someone else how to be a good teacher.

John: And they’ve also been hired because of their expertise and their publications, which doesn’t put much weight on the quality of their teaching. So, what can we do about that?

Kristina: That’s a great question. So, right now I am at an R1 institution with a PhD program in Political Science and we’ve had this conversation. It is a difficult conversation to have because a lot of times, I think professors view that if their students get placements at full time at a community college or even at a teaching institution, that that’s not a good placement—that there’s something “less than” or something “failure” about that kind of placement. So I think one of the most important things is just to change the culture about the way we talk about these things and that’s something that can’t change overnight obviously—this is a really slow process. But instead of telling our graduate students “You need to publish so you can get a really good research placement,” asking students to tell us why they’re here, “Why are you in graduate school? What do you want to do afterwards?” and sometimes that can result in really difficult conversations with my undergrads who want to go into graduate school. When they say, “I want to go into graduate school, I want to be you when I grow up, I want to be a professor, I want to do research—this is something I want to do,” I tell them, “then you better make sure you go to a top 20 program.” Because if you aren’t in one of those programs, the likelihood of you getting an R1 research professorship is really low. And so if we have PhD candidates who are saying, “What I really want to do is work at an R1 institution,” we need to be candid with them about what their odds are and how they can go from a mid- or low- tier PhD producing institution up to an R1—it’s gonna be a long process with a lot of publishing and you can kind of publish your way out. But alternatively, it’s also important to value the students that say, “What I want to do is teach” or “What I want to do is go into industry or paid consultant—that’s what I want to do with my PhD.” If we can change the culture enough to not view teaching as a low- end placement instead of to start thinking of it as a legitimate career opportunity, then maybe that can help us think about how we can better prepare PhD students.

Whitney: I think that that’s a really important point to encourage the people who come and get a PhD because they want to be teachers at teaching institutions. Not only because that’s the more likely job that they’ll have, but also because of actually the tweet that started this whole thing, which we referenced in the article from Simon Hix who said that over the course of his career, the thing that has been most meaningful to him have been the interactions he’s had with students and the teaching opportunities that he’s had with them. So I think that if we have this mindset that the only thing that’s worthwhile is being a proph at an R1 and doing high-level research that’s cited all over the place, but that’s not the only thing that is meaningful in academia. There shouldn’t be anything wrong with acknowledging that… yeah, you can be a really awesome teacher and lead students to be the next generation of leaders themselves.

Rebecca: One of the things that you’ve both mentioned is the change that would need to happen takes a lot of time. So, for students who are in the position where they might want to be a teaching faculty member and they want to emphasize teaching but they’re at a university that doesn’t provide those kinds of experiences, what kind of advice or guidance can we give those students to gain the experience that they might need to actually get a job at a teaching institution, because if you don’t have experience then you often can’t get those positions either.

Kristina: Absolutely. So one of the things that I do at Texas Tech with our graduate students… So I do a lot of publication and research on pedagogy, so they’re hearing messages from their graduate faculty—which I don’t teach grad students, I only teach undergrads—but they are hearing the messaging from the graduate faculty that they need to focus on publishing and then they’re also having a realistic expectation of the kind of job they can get. So oftentimes what I do is offer—if they want to co-author a pedagogy piece with me—then that can kind of kill two birds with one stone and I can pull some of those publication expectations while making them more marketable in the teaching faculty job market. So, having a pedagogy piece—a published pedagogy piece—can send a signal to a search committee for teaching intensive position that pedagogy is something that you care about and that you’re applying your research skills that you learned in your PhD program to the way you’re gonna teach.

John: Some of this, I think, carries over a bit to undergraduate institutions where most of the people coming out of grad school tend to emphasize research that often seems to carry through through the promotion and tenure process because even at undergraduate institutions where their primary focus is on teaching, much of the promotion, merit pay, and so forth is tied to publications and it seems like it may be part of a broader cultural issue, not just at the graduate program level. What do you think?

Whitney: Well, so I’m at a teaching institution and I was actually that student that you just referred to, Rebecca; the one who knew they wanted to do teaching right off the bat. My adviser kind of discouraged me from it, but once he could see I was serious he helped me get teaching assignments at my R1 institution so that I could have that on my resume, which I would say is the number one piece of advice I could give anybody who wants to get a teaching job is have a class that you were the primary instructor on. So, at my institution now, I got my job, I’m tenure track and I am still expected to publish, but I do get credit for those pedagogy pieces that Christina was referring to, which don’t always garner the same kind of promotion credit at an R1. So I am expected to publish and be active in my field but what that means is a lot different than what it means at an R1. It doesn’t mean I have to land pieces in the top three political science journals. if I’m getting the name of my institution out in the media in something like this or if I am quoted in an op-ed, then it doesn’t count as much, but it’s kind of an incremental count because one of the things that teaching institutions often deal with is they’re smaller and they have less budgets, so they need the media attention, and that can be even more valuable than if I publish something in JOP.

John: And you mentioned the scholarship of learning and teaching; that’s an area that’s grown quite a bit in, I think, most disciplines. That seems to be perhaps an avenue by which some of this problem could be addressed (as Kristina just said). When I was a grad student, there was very little research being done on teaching and learning and now most academic disciplines have journals and group meetings or sub group meetings where they focus on these things. So, maybe that’s an area where we’re making some progress.

Kristina: I definitely think so. While the scholarship of teaching and learning pieces certainly aren’t as highly valued or are considered as prestigious at this point, I’ve been saying that graduate programs are missing a big opportunity to develop a niche in what kinds of tracks they offer. So most political science graduate programs will offer… you’re an international specialist or you’re an American politics specialist, perhaps you’re a method specialist; graduate programs are missing an opportunity to offer a track where you’re a teaching political science specialist. If we had faculty members who are publishing and experts in the teaching scholarship of political science, that program could market themselves as “we are the program that generates people who are going to teach political science,” and that could be a great way to start getting your graduate program—maybe you’re a mid-level or low-level R1—but if your graduate program gets nationally liberal arts colleges; that’s just as many state tuition dollars for PhD students as a student who’s studying international relations.

John: …and it could give those students a bit of an edge when they go into the job market too.

Rebecca: I also just want to add that these same issues apply to art schools in places where faculty might be getting other kinds of terminal degrees as well, where their focus might not be on traditional research but they’re doing scholarly activity or creative activity, like doing music or art or whatever and they’re focused so much on their studio practice that they don’t focus on teaching either, so most of the conversations focused specifically on PhD programs, but the same issue applies to some of these other contexts as well.

Whitney: Well, and I think another thing that’s important if you’re in a social science, especially, you’ve been heavily trained in methodology and we have some world-class research skills and I think it’s important to apply that to the scholarship of teaching and learning as well. Actually, Christina and I’s whole publishing relationship started because of an instructional designer at Texas Tech who gave Christina evidence that Christina was like, wait a second, I’m not sure I believe this, let me go look it up. And we were disappointed to see the lack of consistent rigor in the scholarship of teaching and learning and so I think because we’ve already been working on these really rigorous methodological skills, it makes sense that we could also apply them to the scholarship of teaching and learning and ensure that we really are achieving the learning outcomes that all of our colleges and accrediting institutions want us to achieve.

Rebecca: One of the things that we haven’t addressed much but I think is worth addressing is the role that colleges who hire PhDs as teaching faculty—what role they play in helping these new faculty members develop teaching skills and what their responsibility is in relationship to the R1 institutions who are producing these potential candidates.

Kristina: I have seen a growth in professionalization courses in PhD programs and most universities and colleges at this point do have something that resembles like a center for teaching and learning or something similar to this that’s trying to systematize the way we teach our teachers. Oftentimes these are geared toward new faculty, maybe not towards graduate students but typically they try to make them available. I think that we could do a better job at requiring them and at encouraging them as valuable for graduate student’s potential careers. I do find that a lot of the professionalism courses and sessions that I observe are more about the professionalization in terms of publishing and going to academic conferences and getting your CV ready to go on the job market and give job talks. So, we’re moving in the right direction in terms of learning to socialize our graduate students into what to expect, but I still think we have some disconnects between the job market as they will experience it. Now, maybe if you’re at one of the top 10 or 20 programs in the country this isn’t gonna matter, but if you’re not, then this could be the difference between you getting a job and having to adjunct seven courses a semester.

John: Now I think some disciplines have made some progress: chemistry and physics, for example, and math have tracks in math ed, or chemistry education, or physics education where people actually focus on research in that, but it hasn’t made it through all the disciplines. I’ve been the chair of our recruitment committee in my department for 30 years or so now roughly, and I have noticed though that more and more students are coming out with some background, even at R1 institutions, and I know when we go in the job market—maybe because of my position in the teaching center here—one of the things we look at is what sort of background they have in evidence-based teaching practices and so forth, and the people who generally come out in the top of our searches are people who have at least considered these issues or are aware of these issues. I’m not sure how widespread that is though in other departments.

Kristina: And to be fair, we are limited; we’re both political scientists, so we’re both limited to what our experience was and the experience of those in similar fields that we know.

Rebecca: So we’ve talked a lot about two different tracks: PhD candidates from an R1 institution who might get those small select positions as a researcher at an R1 institution and then we’ve got the track of people who might become faculty at more of a teaching institution. What about the other PhD candidates and those that might end up in other kinds of roles like consulting or other things that you mentioned previously? What are we doing for them, or what do we need to be doing for them?

Whitney: I think that the research track doesn’t just have to be for people who want to go on to R1 professorships because the research skills that you learn you can use in a lot of places that really need researchers, especially in government. My backup job actually, in case I didn’t get a teaching job, was going to be a statistician just because of all of the stats that I’ve picked up along the way. So, I think that the research track could be just a research track and what you do with it after is up to you but I do think that there is a whole class of people who maybe want a PhD just because they enjoy learning and want the PhD or maybe they just need the credential to move up in their career and they don’t necessarily want to learn how to teach or they don’t necessarily want to learn how to do research at an R1 level and I think those people are definitely falling out of the grad programs and that’s a shame because I think that there are a lot of lower ranked PhD institutions that again, like Kristina was talking about earlier, that could be their marketing: come here and we’re not gonna bombard you with how to publish in APSR and we’re not gonna bombard you with pedagogy, but you can get the basic skills that you need and write a dissertation and get the credential that you’re looking for.

Kristina: I think there’s also some cultural shifts that need to happen here as well because if getting a tenure-track offer at a teaching institution or a full-time offer at a community college is considered a failure then even more so I think often leaving academia completely to go into industry is considered like the ultimate failure, and I don’t know how universal that is across disciplines. I would imagine things that have a little more practical application would have less of this problem than specific to academia disciplines like political science, sociology, psychology. But, thinking about leaving academia completely is sort of the ultimate failure when there’s plenty people that want to do that and are very successful at doing so. We have a department of public administration within political science at Texas Tech and it’s a terminal master’s degree and oftentimes I hear… well,like the culture in the department is sometimes that the students that are seeking this master’s in public administration they don’t care as much about the research methods, they’re not as interested in learning the statistics or, of course, definitely not learning the pedagogy. It’s much more of a professional and vocational degree and at the end of the day our graduates from that program are probably earning a lot more money than our graduates from our PhD in political science programs. So, thinking about how we can shift the way we view our students career goals and try to match what we teach them to that. That’s something that we talk about in undergraduate education all the time: what do our students want to be when they grow up and how can we give them those skills. There’s no reason why we can’t use that same logic to think about our graduate programs.

John: The same is certainly true in economics. A lot of graduate students, sometimes with PhDs, end up working in government research positions as econometricians, working for example for the Department of Labor or the Census Bureau and so forth… and while sometimes it’s seen as being a somewhat lower position, they get paid a lot more, but we call that compensating wage differentials. T hey have to do these jobs that may be a little less pleasant so they have to get paid more to compensate for the fact that they’re not in academia. They disagree on that feeling quite often. [LAUGHTER]

Kristina: Well they don’t get their summers off.

John: They don’t get their summers off. What prompted you to address this topic?

Whitney: I just want to be really clear that Christina and I had an overall pretty positive experience at our grad institution, so this whole conversation didn’t come out of a feeling of anger. The whole idea came to me first when I was looking on Twitter and I saw the Simon Hix tweet about how much he valued teaching and I was texting Christina and I was like, you know, that’s how I feel too. I really value my teaching but I think sometimes that’s not the most valued thing in all of academia and she was like, “yes, at my institution sometimes being at a teaching institution is seen as lesser than” and so it started this whole conversation about how different the cultures are in our work, but how ultimately we’re both satisfied with where we are and that’s where the whole idea for this article came from. Just thinking about the different cultures that there are in academia and how they can vary so much and yet we prepare students generally uniformly across academia.

Kristina: Yeah, that’s a really good way to put it. Whitney and I went to the same graduate program; we were just a couple of years apart. So we received essentially the same training, which had very little focus on teaching or on what you do if you don’t want to be a researcher or to go to an R1 institution. As I’ve spoken to faculty members at our institution since then; of course we warned them that this piece was coming so they wouldn’t think we were trying to trash our department. But they’ve said that they’ve done things since we were there to try and make that better, especially as they’ve seen where their students are ending up. So, while there’s still a big focus on research being an R1 institution, University of Texas of Dallas is never going to not train researchers, but they recognize that a lot of the students that are coming to that program are looking for non-R1 jobs. And our former professors. seeing where we’ve gone—Whitney’s at a teaching institution; I am a non-tenure-track at an R1, and so I think they’ve been able to look at that history and say, how can we better prepare our students for either one of these options.

Rebecca: One of the things that I’d really love to see more programs include is something that I had in my own graduate education which was a training program for teaching—which gave me a leg up in a lot pursuits that I had professionally. So, I went through the equivalent of the professional development for teachers like we do at our teaching learning center here. I learned about ways to evaluate student work, a little bit about assessment, designing syllabi to be inclusive. So, it’d be great to have those kinds of professional development opportunities for a wider variety of potential faculty. We learned about writing syllabi to be more inclusive, we wrote about evaluation systems, thinking about assessment, designing assignments and things. It wasn’t nearly as rigorous as it could have been, but it definitely was more than many other colleagues that I had that went to other institutions and ho w our different experiences when we entered the field.

Whitney: I would have loved something like that whenever I started because I had no idea what SACS even was when I first began my teaching job, and they’re telling me about assessing learning outcomes and I was like, what are you talking about? And I think there is something to be said for throwing me in the deep end and making me learn for myself. And I definitely learned a lot in my first couple of classes, and I apologize to any of those students who are listening. But I think something like that would be excellent, even just like, here, you have to teach this class; write a practice syllabus. And having to think about what kinds of assignments you design is so enormously helpful before you’re actually on the job because, especially if you go to a teaching institution and you’re teaching a 3-3 or a 4-4, you’re not even gonna have time to breathe, nevermind thoughtfully construct a syllabus.

Kristina: I also think that this is a great place for the intersection of research training and teaching training because a lot of the things that they give us in teaching workshops—here’s what works best, here are best practices. Oftentimes I’m left with the question as someone who’s been in teaching for six years and publishing on teaching and learning, a lot of times I’m left with the question: how do you know this is the best practice? Who says? What’s the evidence for it? And there’s not very much yet. The literature is not robust enough at this time to really be able to say what works best. So if we can intersect those research skills that are social science, PhDs, that are even our humanities PhDs and our natural science PhDs, they’re getting some research training and an ability to think critically about what they’re being told. If we can intersect that with looking at what the evidence that does exist on the best practices in teaching and learning then we’re really just creating a positive reinforcement cycle of how these things all work together. None of these exist in a vacuum; teaching doesn’t exist in a vacuum, outside of political science they’re inextricably linked.

John: And even where there’s some areas where there’s a lot of research there’s often not a lot of research in specific disciplines to see whether the results in other fields hold up and there is a little bit of a replication problem in some of the areas. As you said, there’s just not a lot of research on a lot of topics that everyone takes for granted, so it’s a ripe area for research.

Rebecca: I think it’s a ripe area for interdisciplinary research.

John: When I was first teaching I had a fellowship and a faculty member left about two weeks before the semester, so the director of graduate programs came to me and said, hey, would you like some extra money in addition to your fellowship? You’ve got this class that starts in two weeks; you did really well in the graduate field, so here’s your class. And that was the extent of my training in teaching. It was the first time I was ever in front of a class.

Whitney: Well, and that’s actually a really good thing to bring up. If you are a struggling graduate student and you want to work at a teaching institution, not only is adjuncting at a Community College beneficial for your resume, but it can also help feed you for a little while. [LAUGHTER]

Rebecca: That’s very true.

John: We always end by asking, what are you going to do next?

Kristina: I have a couple of pieces right now that are about to be ready to go o ut for review. They’re actually looking at some of these best practices. So we’re looking at—I don’t know if y’all are familiar with Gagne’s Nine Events of Instruction. This is something that is often put out there as the best way to teach and I think it is useful to some extent, but when we examine whether it really made a difference in student performance, we found that students don’t necessarily know what order they want things then, nor does it really seem to affect their performance in the course. So we’re gonna be publishing that. Again, not with the idea that Gagne should be thrown in the trash, but with the idea that a lot of these best practices that we talk about really are just, if it works for you and speaks to you, then you should use it and if it doesn’t then there’s no reason why anyone should force you to use it.

Whitney: For me, I’m actually working on a book right now with the director of the Meredith Poll, David McLennan and a colleague at Coastal Carolina University, Kaitlin Sidorsky and our book is about women in appointed office. I’m at Meredith College which is a women’s college. Besides my passion for teaching I also have a passion for getting women into politics. 65% of women who run for office served in appointed office first and appointed office isn’t as well studied as women who run for office, so we’re writing a book on that.

Rebecca: Sounds like two really exciting things coming out soon.

John: And maybe we’ll get one or both of you back on in the future.

Whitney: Great.

Kristina: That’d be great.

Rebecca: Well thank you both for joining us this afternoon and giving us some good things to be thinking about.

John: It’s an issue that I think affects pretty much all disciplines.

Whitney: Thank you for having us.

John: Thank you.

[Music]

John: If you’ve enjoyed this podcast please subscribe and leave review on iTunes or your favorite podcast service. To continue the conversation join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fischer, Brittany Jones, Gabriella Perez, Joseph Santarelli-Hansen, and Dante Perez.
[Music]

49. Closing the performance gap

Posted in active learning, assessment, collaboration, evidence-based practices, growth mindset, higher education, inclusion, instructional communication, learning, pedagogy, peer instruction, scholarship of teaching and learning, teaching
October 3, 2018

Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, Dr. Angela Bauer, the chair of the Biology Department at High Point University, joins us to discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

Show Notes

  • “Success for all Students: TOSS workshops” – Inside UW-Green Bay News (This includes a short video clip in which Dr. Bauer describes TOSS workshops)
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc.
  • Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative learning techniques: A handbook for college faculty. John Wiley & Sons.
  • Life Sciences Education
  • Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.
  • Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American psychologist, 52(6), 613.
  • The Teaching Lab Podcast – Angela Bauer’s new podcast series. (Coming soon to iTunes and other podcast services)

Transcript

Rebecca: Sometimes, as faculty, we are quick to assume that performance gaps in our courses are due to the level of preparedness of students rather than what we do or do not do in our departments. In this episode, we’ll discuss how community building activities and growth mindset messaging combined with active learning strategies can help close the gap.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Angela Bower, the chair of the biology department at High Point University. Welcome Angela!

Angela: Thank you John! Hello Rebecca

Rebecca: Hi, welcome!

Angela: Thank you

Rebecca: Today’s teas are…

Angela: I have a piping hot mug of Earl Grey

Rebecca: Yes! Someone drinks tea again! [LAUGHTER]

John: I have ginger peach green tea.

Rebecca: And John get ready for this, Dragon Oolong tea

John: That’s actually very good.

Rebecca: We have a joke that I have my standbys and this was not one of them.

John: We’ve invited you here to talk about your work on reducing the performance gap for underrepresented students in STEM disciplines. Before we discuss this could you tell us a little about the performance gap?

Angela: You bet! Well, I can tell you about my experiences at two different institutions, although I can say that my experiences at these two institutions with respect to the performance gap in foundation STEM courses is not unique to these institutions. It’s a natural phenomenon that if we look at foundation courses that are taught with a traditional lecture format typically, which still is often the case with STEM courses. We see a gap in the performance between majority students and then students from underrepresented groups and this was the case when I taught biology courses at the University of Wisconsin in Green Bay. If we went back and looked at the data in a foundation course in Introduction to Human biology course that we had taught for decades, we could see prior to a change in pedagogy in terms of what was happening in that course, we could see for decades prior a gap in the performance between majority and underrepresented students at that institution. Likewise the same is true at High Point University with the way that we used to teach our introductory biology courses. And this performance gap is really troubling because often times what it means is that the students in those courses that are not performing well, we tend to lose them and in the end we end up having a much lower percentage of students who are members of underrepresented groups choosing to major in the sciences and then of course choosing to pursue professions related to the sciences as well.

Rebecca: Where does this performance gap originate?

Angela: Well that’s a really interesting question and there are a lot of misconceptions about where that performance gap originates. I’ll go back to when I first started teaching. What I assumed which was entirely wrong was that the academic performance gap was an indication of a difference in the academic preparedness of students from underrepresented groups. So I like many other faculty in my field assumed that students were coming to our institutions not having a high school education that prepared them to perform well in these introductory science and math courses. I later came to find out that this was not at all the case. We did a study at University of Wisconsin in Green Bay where we controlled for let’s say SAT scores of these students coming in; so if we looked at maybe white students and black students who came in within the same range of SAT scores specifically their math scores which is a strong indicator of how well they will perform in the course and if we looked at how they performed in the course even when we controlled for SAT Math score, we saw a performance gap. So that suggested to us (and this was the big lightbulb moment for me) this doesn’t have anything to do really with academic preparedness at least with respect to our student population. What it has to do is something that we’re doing in the classroom. We really were teaching in a way that favored the academic performance of students of a particular group or did not enhance the academic performance of our underrepresented minority students and so that’s when we decided to make some changes.

Rebecca: How did you start to figure out what to pinpoint?

Angela: Well interesting how we stumbled upon an approach that worked at Green Bay. We had a variety of instructors teaching this introductory course, so we had multiple sections going on at a given time and as you know is often the case to change what’s happening in the classroom to get instructors to adhere to a different pedagogy; who are often swamped and busy and maybe have taught the course for a certain way for a number of years and are not comfortable with moving to for example a more student-centered format with respect to their courses (which by the way is pretty daunting when you have lecture sections of 150 students). So what we initially decided to do just to see if it would have any impact was we started to do things outside of the classrooms in a way that was very student-led and student centered. Students who were enrolled in this introduction to human biology course in Green Bay were provided with the opportunity to attend workshops outside of the classroom that were totally optional and we call them toss workshops TOSS, which was an acronym that stood for targeted opportunities for success in science. And we chose students to serve as mentors for these TOSS workshops who were science majors who had at least received grades of B or better in these introductory courses, but who had really strong social skills. Then I worked with those student mentors to train them in student-centered pedagogies and also a culturally inclusive classroom approaches and then they would offer these tops workshops at a variety of times during the week when we were sure that we would get a number of students who could attend. In other words we tried to make sure that those TOSS workshop opportunities didn’t interfere with their academic schedule and these weren’t required and we went about recruiting students for these workshops in a really interesting way. We conducted this in collaboration with a man named Dr. Sean Robinson who is now on the Whitewater campus in Wisconsin, but he at the time was working within our Multicultural Center and he worked really hard to recruit underrepresented students who were his advisees in the Multicultural Center and spread the word. Likewise he helped us recruit a fabulous mentor for our students Mr. Junie Lee who was a grad of our program at Wisconsin, who was also very good at networking with these students and we made it very widely known and it became a popular fun thing to do. Because the students who led the toss workshops were very socially engaging they planned fun activities to review the week’s content and it certainly became a destination for these students and it created a really strong sense of community. And over the four years that we ran this TOSS programming what we found from the very first semester that we offered these opportunities outside of the classroom was that we closed the gap and that happened every single semester for eight semesters across those four years. Multiple sections even with different instructors in different pedagogies in these courses having that opportunity to participate in TOSS workshops totally changed the culture and we closed the gap. And so what we learned from that was that a sense of community was I think probably key here. We know this because (well we assumed this) of an interesting analysis that Dr. W Furlong did, who had worked with our office of institutional research in Green Bay. We looked at the student mentors that led the TOSS workshops and we looked at the activities that they had planned for their TOSS workshops in a given semester and I also ranked the students then in terms of the level of difficulty of the exercises that they performed. In other words how academically rigorous were those workshops versus how much social stuff is going on and we actually found an inverse correlation with the academic rigor in the workshops and the impact of the workshops themselves. So in other words when those mentors were just more social and creating a strong sense of community and spending more time just engaging students generally not necessarily reviewing academic content, that’s when we saw the most profound effect. So this is what led us then to really now focus our efforts with respect to closing the performance gap on really the more emotional components of learning, the affective domain of learning. And yeah of course I’m not at all saying the cognitive part isn’t incredibly important because absolutely it is, it would be silly to say that it wasn’t but generally what we found in Wisconsin and now also what we’re finding at High Point University is that we can close that gap by paying really close attention to that effective emotional component of learning and taking really intentional steps to just create a very inclusive community for students and to engage them in very intentional ways.

John: Have there been changes in classrooms as well to try to build the same type of inclusive environment within the classroom setting?

Angela: Yes. So once faculty learned that the impact of TOSS workshops in Green Bay, more and more of them started to pay more attention to what inclusive classroom approaches looked like. and then at High Point University now where we have also done some interventions that have been really successful in closing that performance gap. We now are employing a model where all faculty who teach our introductory biology courses are on board with employing these techniques that intentionally address the affective domain of learning.

John: What techniques did you use at High Point to address the affective domain?

Angela: When I came to High Point University five years ago, of course we pulled up some of the institutional data, the historic data regarding performance of students in our introductory biology courses and of course saw that performance gap, a long-standing one. And so we really tried to build a model that built upon the knowledge I gained in Wisconsin about the importance of building community and addressing the affective domain of learning, we then attempted to employ that uniformly at High Point University. This time in the actual courses themselves and not with these optional outside of the class activities, so at High Point now we’ve got all instructors on board employing approaches in the classroom where they then administer to students our weekly messages that address the affective domain of learning and it involves growth mindset strategies, which is based on the work of the Stanford psychologist Dr. Carol Dweck who wrote a book on growth mindset strategies. And what we are doing at High Point University is providing students with weekly growth mindset messages maybe 5 or 10 minutes at the start of the course on Monday or Tuesday. I prepare maybe two or three PowerPoint slides with a growth mindset message that I send to all instructors before the start of the week and then they show that message to students and they reinforce the message throughout the week. And just to review what it means to employ growth mindset strategies; if a person has a growth mindset they believe that with hard work and persistence they can improve at whatever it is that they are doing. If a person has a fixed mindset they believe that essentially we are how we are and maybe you’re born a science person or maybe you’re born not a science person, maybe you’re a math person, maybe you’re not a math person and so students that have a fixed mindset will very much back away from things that challenge them because if they engage in an activity that challenges them and maybe they don’t perform so well with respect to that academic opportunity, they view that as a negative commentary on their intellect and who wants to publicly out themselves as being not smart or not good at something, so they will shy away from that and of course that has a very negative impact on their performance in the classroom, they don’t seek out the help that they need, if they vomit test they don’t go in and figure out what went wrong in that test. By sending them weekly growth mindset messages we’re telling them we believe you hang in there with hard work and with really smart use of resources that are available to you and figuring out what an effective study strategy is for you, you’re going to be successful in this course, we think you can do it now let’s get down to work. And long story short those growth mindset strategies have been very powerful in helping us close the performance gap, also now at High Point University in our introductory biology courses.

John: What would be some examples of those messages?

Angela: I would send out a variety of messages the ending on the week, some of them were science related and some weren’t. Maybe there was an interesting study that I read over the weekend on neuroplasticity and so of course being biology majors they would have an interest in that you can literally change your brain with practice. There was one study I sent a couple of slides showing MRIs of the brain in people who had intentionally performed or intentionally practiced a video game, it was a race car game where they practice going around a particular track for maybe two hours worth of practice and these MRIs showed literally their brain was changing in response to those two hours of practice so that was one example. I chose things outside of the realm of science and maybe one week we had a couple of slides on NBA superstar Michael Jordan and the fact that he didn’t make his high school basketball team, he got cut from his basketball team but again with hard work and persistence and employing the right strategies he was arguably the greatest basketball player of all times. We have a really funny clip from a Nobel laureates Junger report card from back in his grade school days about his teacher saying you’re never going to be any good at science, you just need to give it up and later he went on to win a Nobel Prize for his work in cloning. These are just a few examples of growth mindset messages that we sent to students on a weekly basis.

Rebecca: Have you noticed a difference between the in-class strategies of growth mindset versus community-focused TOSS workshops that you had done at Green Bay?

Angela: Both have been effective in closing the gap. They’re two very different academic settings, so one thing that the TOSS workshops were really great for in Green Bay was really fostering a sense of community among a group of students, many of whom were commuter students. And so they’d go to classes and then they’d leave the campus, they vacate the premises and we would never see them hanging out and studying up by our offices or engaging in social activities that involve their peers in science and with the implementation of these TOSS workshops we saw much more of that, we saw students hanging around after hours talking more with their professors. So I saw a significant change in our sense of community particularly with our underrepresented minority students in Green Bay as a result of those TOSS workshops. It’s I would argue much easier to address the community issues in certain respects at High Point University because we’re a private institution, students are required to live on campus, there are all sorts of social activities to engage them all the time. Now whether or not they’re feeling a strong sense of community or developing a strong sense of trust with the faculty that teach their science courses or feeling a sense of belonging I think what we’ve done with growth mindset strategies is heighten all of that with our High Point University students. So they’re two very different models, I think they are very well suited for those very specific institutional contexts. I have to tell you that when I first got to High Point University I tried the TOSS workshop model and nobody came because it was optional, it was outside of class at a time where quite frankly there were lots of other social events happening on campus, extracurricular activities, students are involved in so many different clubs on campus so that just was a model that wasn’t going to fly here, we needed to do something specifically in the classroom to catch those students when we had our captive audience.

Rebecca: It’s really fascinating to hear those differences, but also recognize that you need to adjust your design based on context.

Angela: I think context is a huge deal.

John: Did you make any other changes at High Point besides adding the growth mindset prompts?

Angela: What we did within the classroom was more than simply take these intentional approaches to address the affective domain of learning through the use of growth mindset strategy. So we also very intentionally changed the way we taught our foundation biology courses by also making sure that we were employing best practices to address the cognitive domain of learning. Prior to when I had come to High Point University lecture was the primary mode of teaching in our introductory biology courses and there was a lot of turnover that happened when I arrived, there were curricular changes, the hiring of many new junior faculty just in response to the growth of our program and as we hired these new faculty we were very intentional in training them in approaches that would involve active learning in the classroom. And we worked very hard with the instructors that teach multiple sections of this introductory biology course to standardize our curriculum in a way to make sure that students were getting exposure to active learning, best practices for addressing the cognitive domain of learning in addition to those that address the affective domain of learning with the growth mindset strategies. And what we found one year when we ran simply the active learning approach where we covered at least 25% of the material in those introductory biology courses with active learning strategies, but didn’t employ the growth mindset approaches. We still, with some of our underrepresented groups, still saw a gap— a performance gap. So again, while some studies have shown that active learning can help to close that gap or narrow that gap, we weren’t very successful in doing that at High Point University. Active learning alone didn’t do it for us and I think that’s an important point to make for whatever reason. It could be our institutional context, it could be a lot of different things, but my point is when we combined active learning with approaches that also address the emotional or affective components of learning, that’s when we hit the sweet spot. We needed to do both of those to close the performance gap.

John: What specific active learning strategies did you introduce?

Angela: Active learning strategies that are chosen by the instructors— they are individual. All of them use student-centered discussions of the primary scientific literature so that’s one of the outcomes of that introductory biology course. For us to introduce our first-year students right away to the primary scientific literature, so that’s one approach I know. Instructors, we primarily leave it up to them to choose what they want to do when we have people join our program or begin teaching our introductory biology courses. We provide them with that Bible of students under learning techniques called Collaborative Learning Techniques written by Elizabeth Barkley and her co-authors. They get that and I know a lot of them use that as their go-to manual if they want to set up an active learning strategy to cover a particular topic on any given day. They’ll employ maybe just simple think-parent shares or something more elaborate than that. That’s really up to the individual instructors to do it but we just ask that they spend about a quarter of their time doing that. We also revamped our laboratory sections associated with that course so that they are also much more student-centered. Now someone argue that laboratories by nature are active learning— I would argue that they’re not in the traditional way that they’re taught. Traditional labs are very cookbook in nature and performed in a way I would argue that don’t really actively engage the students brain. You come in, here’s what you do, settle this up and here’s what you should expect to see— that’s not at all what we’re now doing in these introductory biology courses. We are infusing scientific inquiry into pretty much every aspect of that course. Yes, covering the technical skills that they would need to be proficient cellular and molecular biologists but they’re also expected to come up with hypotheses to test and then use those technical skills as they design experiments, test their hypotheses, analyze their data, etc. So the labs also are very student-centered and, by the way, much more than 25 percent active learning happens in those labs— the majority of what happens is active learning. So when all is said and done, with the implementation of those active learning strategies along with their growth mindset messages, we’ve had a lot of success in enhancing the performance of our underrepresented students in closing the gaps.

Rebecca: One of the things that you mentioned in passing was training your faculty when they come in in active learning strategies and I imagine also these affective strategies as well.

Angela: Yes.

Rebecca: How do you employ that? Is that through a teaching and learning center on campus? Is this through your department? How does that work?

Angela: That is a good question. So when we had the addition of so many new faculty to our program— so one year, we had three new faculty. The next year, we had five new faculty. The year after that, we added two more. That’s huge turnover and again, reflects the growth that was happening in our program at the time. One thing that I did was I established a journal club— typically science departments have journal clubs where people present their data that they’re creating in lab or maybe they talk about a scientific paper. I set up a teaching and learning journal club where every other week, we would get together and talk about our recent publication in the scientific teaching and learning literature. And the new faculty who were coming in were really eager to learn this stuff because they wanted to improve their student outcomes and they wanted to be on board and teach in a way that was equitable within their classroom. So that was very helpful in terms of sharing with new faculty the findings regarding the evidence-based literature about the impact of these teaching practices on their student outcomes. So that was a really effective way to do it. Then that was the first year or two when everyone was on board and then, as happens, once people get their research programs established and then they start getting on committees, it became less and less likely that we would have a good turnout with our teaching and learning journal clubs. So my latest trick now is I just started a podcast that’s really focused on teaching in STEM and I’m calling it The Teaching Lab and my goal is to interview an innovator in STEM education every week or two who can share something about a recent publication in the teaching and learning literature that I can then share with my faculty. So it’s sort of an on-the-go professional development opportunity for them when they’re out what can their dog in the morning, they want to listen to a podcast or driving home at night because it’s becoming harder and harder as they establish themselves and now have their research programs up and running and they’re wanted in a million other areas on campus, it’s less likely that we will get together for this teaching and learning in the sciences journal. We call it our TLS Journal Club. We still have lunch together at least a couple of times a week and just bat ideas around and it’s a really wonderful community of colleagues I have here and I’m so fortunate in really having that journal club earlier on set the tone but now I’ve got to come up with other tricks because of the changes that have happened.

Rebecca: Looks like we’re all looking in the same bag of tricks, right? [LAUGHTER]

Angela: Well, it makes total sense.

John: Yeah, the asynchronous nature of podcast makes it a whole lot easier for people to fit it in their schedules.

Angela: Yes, and it’s easier then to sort of pick and choose. If there’s one podcast that eh, not so much use to me or of interest to me. I guess that’s probably more socially acceptable to skip listening to a podcast then just skip coming to a journal club and feeling the guilt associated with that.

Rebecca: You’re probably right. [LAUGHTER] What resources might you suggest for other faculty who want to work on reducing the performance gap in their classes?

Angela: It’s a good question. Just keeping up with the teaching and learning literature in the sciences. Life Sciences Education is a great resource that publishes a lot of evidence based up with respect to inclusivity, closing the performance gap, etc. I recommend Carol Dweck’s book on growth mindset to everybody and just going back and reading the classic literature by Claude Steele, the Stanford psychologist who did the original work on stereotype threat. Going back to those classic studies and looking at how profound the impact can be of stereotype threat, if you activate the threat of stereotype in a student, maybe it’s pointing out that women typically don’t perform as well as men in a math class. If an instructor just makes a statement like that before handing out an exam, it can have a profound impact on the performance of students. Just going back to read that classic literature about stereotype threat published by Claude Steele and colleagues and really thinking about how things we do in the classroom, totally unintentionally, can just have such a profound impact and how we really need to think about our messaging, and what we tell students, and the importance of building trust, and telling them that we believe in them, and also telling them you gotta work hard too, you know? And here’s how we’re gonna do that, here’s how we’re gonna progress this semester together, and here are the resources we can provide to you. Those are all wonderful resources, I think, for informing our teaching and really keeping at the fork front of our mind how incredibly important it is to address those emotional components of learning.

Rebecca: What are some of the strategies that you’ve used personally in your own classes in response to the work that you’ve done to improve inclusivity?

Angela: So I make sure, always, to be sending growth mindset messages to my students. If they don’t do well on a lab practical or don’t do well on an exam, I make sure to let them know that I believe in them and suggest to them alternative strategies or approaches that can help them to be successful. I work really hard, just generally, to foster a strong sense of community in my classroom, to invite students to come to my office hours even when there’s nothing wrong. Just come and hang out with me and let me know how your life is going. Fostering a sense of community is really important to help students feel like they belong. I try really hard to include in my classroom examples in discussions that are really culturally inclusive and that would reach a variety of learners. Right off the top of my head, those are things that I try to do on a regular basis. I try really hard to bring in speakers from underrepresented groups— scientific speakers from underrepresented groups because often, still, it’s a national phenomenon. We still see underrepresentation of certain ethnic groups within the sciences for sure. It’s less the case at least at undergraduate institutions that women are underrepresented. We’re seeing more and more women now in STEM disciplines, especially teaching at undergraduate institutions, but still we respect to scientists of color— very hard for our underrepresented minority students to find mentors who’ve had the same experiences that they’ve had. So while we work hard at High Point University to recruit faculty from diverse backgrounds and we’ve had some success in that regard, we could do better but we’ve had some success. We also try really hard to bring to campus scientists from underrepresented groups to meet with our students, to serve as role models for our students. That was very much the mode of operation for me in Wisconsin. I got some grant funding to make that happen in my students, who were able to interact with those scientists and be mentored by them, they still speak years later about what a significant impact that was on them in their life. So one example was Tyrone Hayes, who’s a very famous researcher at Berkeley, who is the guy that discovered how the pesticide atrazine is causing malformations and sex reversal in frogs and so forth. He is an African-American scientist who we had come and meet with our students in Wisconsin, talk about his research of course, but also just spend a lot of time during his visit hanging out with students and sharing his experiences and that really had a profound impact on them that they still talked about to this day.

Rebecca: Sometimes it’s things that seem so little to us that are so big to our students.

Angela: Absolutely.

Rebecca: We had a lot of interesting and exciting things going on but we always ask: what’s coming next?

Angela: What’s coming next? [LAUGHTER] I have a lot of data that I’m analyzing right now about the impact or growth mindset strategies on men versus women. I want to see if there’s a difference in responsiveness to the messaging in our male versus female students so there’s that. We’ve also implemented some growth mindset messaging in our introductory chemistry courses and we’re looking to see if they have the same impact in the chemistry courses as well, which is enough for now truthfully.

Rebecca: It seems like plenty.

Angela: Beyond that, you know we now have adopted growth mindset approaches as our campus quality enhancement proposal. So I made a pitch to our faculty when it came time to choose a new QEP topic that growth mindset strategies would really be an effective approach for addressing some of the problems that faculty/staff had identified that they encountered with students that frustrated them that they would like to see some approach adopted at our institution to address. Like students shying away from challenges or not really digging in when they were bumping up against obstacles within the classroom. Growth mindset strategies are incredibly effective for changing the way that students approach problem solving in the way that they feel about failure or challenges. It really helps them to be more resilient. Messaging itself is incredibly effective, we knew that that was the case in the k-12 setting based on Carol Dweck and colleagues work but now we and others are finding that in higher ed classrooms, likewise, they have a profound effect. So the growth mindset strategies and the messaging and the impact on a variety of student learning outcomes are now being employed High Point University campus pretty much universally in a variety of different disciplines and even within student life.

John: Well thank you, you’ve offered some wonderful suggestions that I think would be useful not just in the STEM fields, but in all of our disciplines.

Rebecca: Definitely. Thank you so much for your time and great research that you’re doing. I look forward to hearing what else you find.

Angela: Thank you so much for having me Rebecca and John.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

45. Opening the STEM Pipeline

Posted in active learning, authentic learning, evidence-based practices, higher education, inclusion, learning, pedagogy, project based learning, scholarship of teaching and learning, teaching
September 5, 2018

Preschool through high school experiences have a direct impact on the majors and disciplines that students want to study and engage with in college. Designing these experiences to invite underrepresented groups into the discipline early can help to inspire and motivate a new generation of professionals. In this episode, Dr. Stacy Klein-Gardner joins us to discuss how engineers are attempting to diversify the field.

Show Notes

Related publications:

  • Parry, EA, PS Lottero-Perdue, SS Klein-Gardner.  Engineering Professional Societies and Pre-university Engineering Education.  In M. deVries, L. Gumaelius, and I.-B Skogh (Eds.) Pre-university Engineering Education.  Rotterdam, Netherlands: Sense Publishers. 2016.
  • Reimers, J. E., Farmer, C. L., & Klein-Gardner, S. S. (2015). An introduction to the standards for preparation and professional development for teachers of engineering. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), 5.
  • Klein-Gardner, S. S., Johnston, M. E., & Benson, L. (2012). Impact of RET teacher-developed curriculum units on classroom experiences for teachers and students. Journal of Pre-College Engineering Education Research (J-PEER), 2(2), 4.
  • Klein-Gardner, SS, ME Johnston, L Benson. Impact of the RET Teacher-Developed Curriculum on their teaching strategies and student motivation.  Journal of Pre-College Engineering Education Research. 2(2):21-35. 2012.
  • Faber, C., Hardin, E., Klein-Gardner, S., & Benson, L. (2014). Development of teachers as scientists in research experiences for teachers programs. Journal of Science Teacher Education, 25(7), 785-806.
  • Mckay, M., Klein-Gardner, S. S., Zook, K. A., Yoder, M., Moskal, B. M., Hacker, M., … & Houchens, B. C. (2011). Best Practices in K-12 and University Partnerships Panel Winners ASEE K-12 and Pre-College Engineering Division. In American Society for Engineering Education. American Society for Engineering Education.

Transcript

Rebecca: Preschool through high school experiences have a direct impact on the majors and disciplines that students want to study and engage with in college. Designing these experiences to invite underrepresented groups into the discipline early can help to inspire and motivate a new generation of professionals. In this episode, we explore how engineers are attempting to diversify the field.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Dr. Stacy Klein-Gardner, the founding director of the Center for STEM Education for Girls, and currently an Adoint Associate Professor of Biomedical Engineering at Vanderbilt University, and a Senior Professional Development Provider with Engineering is Elementary, at the Museum of Science in Boston. She recently was appointed as a Fellow of the American Society for Engineering Education.
Welcome, Stacy.

Stacy: Thank you. I’m delighted to be here.

Rebecca: Welcome. Today our teas are…

Stacy: Well, I have to confess that I don’t care for tea. So, I had some lemonade with lunch and I’m good to go now. [LAUGHTER]

John: I’m drinking Tea Forte black currant tea.

Rebecca: …and I’m having Lady Grey today.

John: We’ve invited you to join us because of your very extensive work in improving educational, P to 12 STEM and STEAM education pathways in many ways. First, though, could you talk a little bit about your own pathway to a career in engineering and engineering education?

Stacy: Sure. I’d be happy to. I grew up in the American South…actually went to junior high and high school in Oxford, Mississippi. I wasn’t always satisfied with my educational opportunities there, so I spent every summer possible at the Duke University talent identification program, or Duke TIP. Which is where I made some wonderful lifelong friends that have influenced my personal life and career since then. I did go to Duke University, where I double majored in biomedical and electrical engineering. I spent my summers working at Duke TIP, really falling in love with education and realizing my passion for that. I did a masters and a PhD in biomedical engineering from Drexel and Vanderbilt University, respectively. Then, I always thought I would retire to teach high school one day and realized that was stupid, and if that’s what I really wanted to do, I should go do it. So, in the same Fall, I defended my dissertation, I started teaching high school full time and fell in love with being in the classroom and working with teachers. Since then, I’ve been a high school teacher, full or part-time, for over 20 years now and I’ve been on the Vanderbilt University faculty since 1999…and I’ve done everything from being Associate Dean to research track professor to adjoint professor now…but really have enjoyed creating my own career in engineering education.

Rebecca: You mentioned being the Associate Dean for Outreach at Vanderbilt School of Engineering. Can you describe what your role was like? I think it’s a little unusual, perhaps, to have an outreach dean so I think it’d be interesting to hear about that.

Stacy: Yeah, the title was definitely unusual at the time. You do find more positions now, often maybe at the assistant level. But, I had a really diverse group of things I was in charge of. I worked with our Career Center on setting up appropriate opportunities for the undergraduate and graduate engineers coming out. I managed a big sponsored lecture we had every year. My favorite part was definitely doing K-12 outreach for the School of Engineering and reaching out to local communities and schools and students. Another favorite part, one that maybe surprised me a bit that I ended up really loving, was study abroad for engineers and finding ways to help engineers figure out a way to get abroad. ‘Cause the rumor used to be that engineers couldn’t study abroad, but there’s so many more types of programs that you can go to and so mine was, finding the right kind of programs and aligning those with the degree requirements of Engineers and then helping the engineers know how to plan ahead to actually travel on them.

Rebecca: So, can you talk a little bit more about your work in K-12 and also the study abroad stuff because in fields where we might not usually think about these as being good matches, like engineering, we’re always looking for new strategies to find those relationships and what have you. So, can you talk a little bit about some of the strategies or things that you developed?

Stacy: Sure. In study abroad, a lot of it was doing the logistics, but some of it was also helping engineers realize that in order to come up with good engineering solutions, you have to really understand the client for whom you’re working; the person who’s found the problem that you’re looking to solve. So you need to not just understand the straight up science, technology, engineering, and math, but you also need to understand the culture of the person, perhaps the language…What is it about their environment that makes different design constraints? So, I think, having engineers study abroad, in such an international world that we live in, is crucial now. I’ve really seen it grow in popularity which has been really fun, even though I’m not in charge of it anymore. We have a very high percentage of students at Vanderbilt who now study abroad as engineers. The second half of that question, or maybe I took him in out of order, was around K-12. You know, at the time I was doing a lot of funded work by the National Science Foundation. My favorite project was a Research Experiences for Teachers program (RET). This is a program where you bring, typically, high school faculty (although that’s broadened some since then) onto your university campus, for six weeks during the summer. Then I would place those teachers into different labs that I had picked very carefully and they would have an assigned project that they worked on full-time, for most of those six weeks…and then at the end of that time, I would work with them on designing curriculum that would be both standards-based (so they would be allowed to teach it in their classroom) as well as based in the research of their labs. So that they were bringing in real-world, current research that was going on, and often the people from that lab would come to the high school as well. Then we would publish those units through a wonderful national digital library called TeachEngineering.org. So that was definitely my favorite piece. I did some other work. I designed some high school level medical imaging curriculum units, and getting to where people have a better grasp of “what is ultrasound? or MRI? and how do those things work?” and actually motivate you to want to study high school physics or math or something like that.

Rebecca: That sounds really exciting and a great way to get people involved in fields they might not know that much about.

Stacy: It’s definitely important, especially when you’re thinking about subjects that sometimes get a bad rap for being particularly challenging. It’s good to let people see why it is they’re learning those and to put that, when am I ever going to use that, upfront so they know exactly when and how they’re going to use that.

John: Has there been any follow up in terms of following students to see how many of them did go into careers in STEM fields?

Stacy: It’s a little hard to get some of that data because I often work at the teacher level and it’s a whole other level of IRB [LAUGHTER] to get at student level data.

John: That’s true.

Stacy: You know, I think it’s somewhat depressing in that the numbers for engineering percentage-wise aren’t increasing rapidly at all, even though a lot of people are putting a lot of time and effort into it. So, not always, I mean I definitely have a lot more confident teachers in the Middle Tennessee area who are integrating what is going on in engineering into their classrooms. Of course it helps now that the next generation science standards have engineering embedded into them and just recently in my state the Tennessee state science standards do as well.

John: In 2010 to 11, you established the Metropolitan Nashville Public Schools K to 12 Engineering Pathways. What problems did this address? How has it worked and are you still working with the Nashville public schools?

Stacy: I think one of the biggest problems it was created to address is a misalignment between what different careers and companies are looking for in their high school graduates, as well as probably colleges too, with what the high schools were producing. There’s such an emphasis now on STEM, and problem solving, and computational thinking that really wasn’t being addressed by the schools and so, with Race to the Top money, Metro Nashville Public Schools set out to form this engineering pathway. I was heavily involved with it for that particular year. I did a lot of professional development. I did all of the professional development for the elementary school that was part of this K through 12 pathway, using the Engineering As Elementary curricula and integrating that. Then, at the high school level, I actually co-taught a ninth grade engineering course at this particular school. So, I was helping another teacher who was an engineer by training but didn’t have as much of pedagogy, and that sort of thing. Trying to help her build up her skills and left her rolling. That high school, Stratford High School, is still clicking along and doing really well with STEM education. It’s growing in reputation and now has a middle school that’s been integrated into the campus as well. So, I think it’s it’s been a success and will continue to be. One of the teachers with whom I worked with the most there is now the STEM director for the entire district. It’s been nice to watch her come from being one of my RET teachers to that position at Stratford, now to leading our entire district. My involvement with the district kind of waxes and wanes over the years. You know, I’ll get really involved for a while, and then I’ll be less involved for a while. I’m not working with Metro Nashville Public Schools right now although I’m always available if they call on me for anything in particular. I’ve actually just joined an advisory board for the Williamson County Public Schools which is just south of here. So, I like to keep my finger in the pie in something locally, but then I often try to work more on a national level.

Rebecca: I wanted to follow up a little bit on what you were talking about elementary education and engineering. For many of us, perhaps, when we went to elementary school, engineering wasn’t a part of that curriculum. So, for those of us that aren’t in engineering can you talk a little bit about what that even looks like?

Stacy: I’d be delighted to. If you think about what the characteristics of an engineer really are…it’s around someone who’s creative, and who thinks outside of the box, and brings in different kinds of solutions, and doesn’t have a lot of preconceived notions. If anything, that’s exactly what a preschool to elementary age child does. They haven’t sort of been beaten down by the system to think in a particular way. They still have that inherent creativity. So, the ideal time to introduce the field of engineering is at the preschool through elementary levels…so that they learn what the field is about, can identify what an engineer does, and have positive feelings towards it, and that we’re creating them to be more STEM literate citizens. There are multiple programs out there. The one with which I’m most familiar, and have even liked so much I’ve joined their staff, is with Engineering is Elementary. But, with any of them you find an authentic but sort of compacted version of the engineering design process. I might look at what a college student would use, or even a high school student might use. and we might call out 12 different phases of the engineering design process. But, in elementary school we have five fingers, so we have five steps to the engineering design process, [LAUGHTER], and in preschool we have three steps. So, just kind of compacting it a little bit…always providing an accurate view of the field. Then giving the kids age-appropriate challenges, things that might happen to an elementary aged child, and then asking them to problem-solve.

Rebecca: Can you give an example?

Stacy: Oh sure. There’s one of the EIE units that comes to mind, where the kids are out there playing a sport and their team needs to be cheered on. They find this little turtle nearby, and they win the game, and so they decide that they’re gonna keep the turtle, and they have to bring the turtle back for the next round of the playoffs. Somebody’s got to keep the turtle in a place where the turtle can not die, because that would not be good for school spirit at all. So, the whole question becomes around, what do you need to design in order to have a habitat that this turtle can live in? They draw upon the appropriate science in this particular unit…and a lot of its around membranes and creating a habitat that has enough water but not too much water. So they draw upon things they’re already learning through the science standards for elementary age children, but they’re putting them to use, and they’re working to save the turtle. Of course they do. It’s an exciting unit, it’s based on a story book that sets the stage for it so you get a lot of your reading and ELA minutes and that sort of thing in it, but then really does bring in science and math as they use the engineering design process.

Rebecca: Sounds really fun.

John: It does.

Stacy: It is a lot of fun [LAUGHTER].

Rebecca: I mean I have to admit I asked that question just because I have a toddler and I was just really curious [LAUGHTER].

Stacy: Talking about the new Wee Engineer, WEE, it’s very cute its for preschool kids.

Rebecca: Yes, yes. Yeah, I want to hear about it. [LAUGHTER]

Stacy: Oh, you really do want to hear about it?

Rebecca: No, I really do.

John: She does [LAUGHTER].

Stacy: The new Wee Engineer units that are coming out are meant for the preschool setting where the teacher introduces the problem…and it’s actually not a teacher, it’s a puppet…and so the puppet comes and introduces and says something like “I want to throw a party for my friends, and I want to make this noise maker really loud, and what do you think of my noisemaker?” …and of course it makes no noise. The puppet then says, “Can you help me?“ …and so the students go through an explore stage, where they explore the materials that are available. A lot of the work at this age focuses on helping students think about how a material is made and how that affects its function. So, they explore different materials and then they get to create their own noisemaker in small groups… and they test it… and then they do it again…and they improve (which is a big part of the engineering design process), until they all have really loud noise makers which they then share with each other and they of course give back to the puppet so the puppet can help throw a good party.

Rebecca: I like that it’s given back to the puppet so that the teachers don’t go crazy. [LAUGHTER]

Stacy: Yes, that would be a critical part of not driving the poor preschool teacher insane.

Rebecca: That sounds like a lot of fun. Maybe I need to go back and teach preschool engineering instead of web design. [LAUGHTER]

John: But, so many students get turned off early on and reaching them early can be really effective in stimulating later interest.

Stacy: They do.

Rebecca: Can you talk a little bit about what you’ve done with STEM for girls and other underrepresented groups and how to get them interested and excited about STEM?

Stacy: Absolutely. My study of the literature shows that a lot of the things that motivate girls also motivates different underrepresented groups, particularly underrepresented minorities/ethnicities, and is often just generally in line with what is good pedagogy, if people actually stopped and thought about that in engineering. I’ll focus on the girls just because that’s been my wheelhouse for the last seven or eight years now. But, a lot of the research shows that girls are interested in helping people or the environment, or something like that. If we can help frame STEM as being something in which you can help people, we will inherently pull a lot more girls into that field. So, that’s kind of one basic way. Often, if I talk to a girl, she’ll say she wants to be a doctor or nurse or something like that in the medical field, because it’s so painfully obvious of how you help people. I try to turn her into thinking about engineering and what engineering really is. Majors like biomedical engineering and environmental engineering are often popular with women, because again, it’s obvious how you’re helping people or things. But if engineers are good, and there’s actually a whole book called Changing the Conversation published by the National Academy of Engineering, on how do we praise engineering appropriately, because it is all about solving human needs and want. If we can present the field more accurately and more fully that will help. I think also when I look at a lot of these things, I like to be very explicit about things like stereotype threat, or implicit bias, or imposter syndrome, and I try to be very overt about teaching students what these things are so that they can recognize it in themselves, know what it is. There’s something about identifying it. I even still have imposter syndrome at times, where I feel like somebody’s gonna figure me out…that I’m not actually that good at engineering education, despite having just been named a fellow of a prestigious Society, I feel like still somebody might figure that out. But I know what it is, I can call it out and say you’re just having a case of imposter syndrome and, somehow, it’s easier to move aside and move along if you know that it’s a real thing and you’re not the only person who has some of these…I call them issues, I’m not sure that’s the right word.

Rebecca: I agree with you. The ability to name it out and file it away allows you to move forward. When I finally learned what some of those things were as a designer, I too, was able to overcome some of those hurdles.

Stacy: I guess the other thing I’ll add, is Carol Dweck’s work with growth mindset, has really put a name to something…about having the ability to think of your brain as a muscle that you can flex and you can grow and it can get stronger. I think letting students know that that’s a thing. Or, at the school where I worked most recently, you were not allowed to say “I’m not good at whatever it was,” you were only allowed to say “I’m not good at _____ yet.” …and I really appreciated that word “yet” there, and the implication that you can and will be good with it, but it’s going to take some hard work, and things don’t always come easily…whether you’re gifted or not doesn’t really matter, you still have to work to accomplish anything good.

John: Besides stereotype threat, implicit bias, and imposter syndrome, what are some of the things that are being done in classes now that deter women and minorities from entering engineering and other STEM fields?

Stacy: The first thing that pops in my mind there is thinking about the examples that are used in a classroom. If there are examples that are supposed to illustrate some concept, yet they are completely unfamiliar to you because the situation in which you’re growing up provides you no context for experiencing that or understanding that, you’re immediately set at a disadvantage in the classroom, and that’s not going to encourage anyone to want to continue in that field. I think there’s also some cases of just downright bias. I had a professor in college that didn’t really seem to think women should be engineers, and well I do know that that is improved, that’s not gone. There are cases of bias that are still out there. I also think a lot about parents and the role of parents, and what they believe their children, their daughters especially, can do…and what’s appropriate for them. Because there are some cultures that have a lot of bias kind of built into them and so it’s about changing the way parents think. Because if a student…if her parents don’t think she should study engineering or science or something, she’s probably not gonna go study that in college. So, we need our parents to understand what these fields are about…educate them…and then get them as a part of our moving more and more diverse people into these STEM fields.

Rebecca: One of the things that I think a lot about is the relationship between design and engineering. As a visual designer, I know that I end up with a lot of students who seem to have a fear of math, or a belief that they just can’t do math, which the process of engineering in the process of visual design is, I don’t know, almost exactly the same. So it’s always interesting to me that they err on the side of the arts thinking that they’re somehow avoiding math, but then of course they discover that there is math there too. Are there things that we can do to help overcome this…I don’t know…. it’s like almost like a preemptive strike, that like “Oh, there’s math. so I obviously can’t participate in this.”

Stacy: I hope so. I feel like we’re making some strides in that area, because you’re right, it is often math that is the big hang-up on why people don’t stay in STEM. Some of that is from having one of your parents, especially the mom, saying “Oh, Honey, I wasn’t good at math, you don’t have to be either.” …which, of course, we would never in a million years say about reading or a lot of other areas. So, I’m not sure why we say that about math sometimes. I think we’ve got to figure out how to let the math come naturally; that, if it’s a part of some problem that you are actually interested in solving, you have empathy for your client, and you’re invested in it, the teachers picked a good problem…“Oh gosh, look we’re gonna have to do this math here” and suddenly it makes sense why you’re doing the math…and you have a reason to want to do it. I think those are critical things that we need to have in our math sequences from elementary on up, so that students don’t develop this hatred or fear of it that is somehow irrational. I also think that while there is math in engineering, not every engineer does mathematics all day long. So, there is some conceptual understandings you have to obtain in order to become an engineer, but it doesn’t mean you sit around and solve differential equations all day long, necessarily. Some can, but many don’t.

Rebecca: I think that’s an important thing. I think there’s a lot of fields where we just assume that people just do math all day and it’s just a misunderstanding or misconception about the field. I think it’s also, sometimes, we present some things in such an abstract way that it doesn’t seem relevant. So, I always like to share with my students the experience that I had around geometry. When I was learning geometry in high school, I could do geometry, I could answer the questions correctly, but I never really understood what the point was and like “I’m never gonna use this.” Then I started doing more programming stuff and made visual interfaces and then all of a sudden was like: “I understand why this is relevant” [LAUGHTER] and I had that breakthrough moment where I was using all kinds of different geometry equations and things to create visual interfaces, essentially.” So, I share that with students and that sometimes helps a little bit. I could put in the math, and then all of a sudden I saw a visual, and then it just clicked and made sense.

Stacy: …made sense…it had purpose to it.

Rebecca: Yeah.

Stacy: I think a lot of math traditionally has been taught like as this separate silo, never to be used…and sometimes I think it’s because the math teachers themselves don’t know when it’s used. They don’t know the science or whatever the other field is…or psychology…and there plenty of places with statistics that use math. But, I feel like we have to lead with those things. So, when I was a high school math teacher and I wanted to teach sinusoids, I would lead with “What’s the temperature gonna be on your graduation day?” …and so we would have to develop this whole model to predict what the temperature was going to be on their high school graduation day – which was not just in a few months, most of them are juniors taking the class. So, we would have to develop this whole mathematical model which involves sinusoids and all the different parameters of one, and then on the test I would give for that unit they had to answer that question for me. It was always fun on their actual graduation day to see how close we had come.
[LAUGHTER]

Rebecca: That sounds like a lot of fun and a great way to make things seem relevant.

Stacy: mm-hmm.

John: So, one thing that would help is if math instruction and science instruction was made more meaningful by using more meaningful examples, so that the math is motivated…so people can get past the fear, because they see that there’s a purpose to it it’s difficult to do that. But, you’re doing some of that with the Nashville schools. I hope we see that more nationwide.
Could you talk a little bit about your work with Engineering is Elementary in designing curriculum?

Stacy: Sure. I have been affiliated with the Engineering is Elementary program for about a decade now. I have followed their work…their research as it was designed and presented at the American Society for Engineering Education. I was super impressed along the way that they were actually doing real education research and they weren’t just developing some curriculum and going “Oh, look how many people use our curriculum.” They were actually looking to see if learning objectives were being met and things like that. So, I’ve had a lot of respect for them. About ten years ago, I affiliated myself with them and became one of their endorsed network providers, such that I could provide their workshops whenever I wanted to around the country, I always really loved and admired how they set up their professional development…how its implemented…that the PD itself is based in research. So, when I was looking for a new educational intellectual pursuit to take on in my career, I approached them and asked if I could come work for them, and to my delight they said “Yes.” So, I started working there part time in January and have enjoyed that. There are separate teams within EIE and I serve on the professional development team right now. So, I’m enjoying working with our extended network of partners, so the people I used to be one of, I now work with and help to take the best of what we’re doing in house in Boston and get that material and the best practices for engineering education out to all these people across the country who can help spread the word and get more kids into engineering. So, I’m really enjoying that piece. I’m gonna be developing some of my first, I’ll call it online PD, but it’ll have some hands-on components to it also, for the adult learners. I think that’s a fun new pursuit for me. In house at EIE, they’ve just created (as I was mentioning earlier) this new Wee Engineer program for preschool and pre-K and there’s also a new EIE for Kindergarten, which I’m thrilled to see, because those age groups desperately need some authentic well-designed, well-researched curriculum. That’s kind of been my role right now. We’ve got some middle school projects as well, but I’m really enjoying starting young and going all the way up through eighth grade and looking at how do you do that best.

Rebecca: As a college faculty member, I’m interested in how you got involved in more of the P-12 things. How might you encourage other faculty, no matter what their discipline is, to get involved at those earlier levels?

Stacy: Great question. I think it’s kind of fun to see, once it clicks to faculty members that what goes on in P-12 definitely affects what can go on at the university level. Some of it can even be slightly self-serving in that they want more students or more diverse students to enter into the university process. So, I think that’s part of it. I think that it’s fun to help a university faculty member see how they can take their passion and enthusiasm that they have for whatever their research field is and take that down to younger kids and distill it to the basics, but increase people’s understanding of their field overall. So, I like that piece of it. There’s the other motivating factor, if you’re gonna apply for like a National Science Foundation grant you have to have an education and outreach component. So, that there’s that external motivator as well to think about how could I be involved in this process. I also worry a lot about we have these new next generation science standards. which I think are quite good but they have a lot of engineering in them. So, who exactly do we think is going to help the K-12 teachers know how to do that, and do it authentically, if we ourselves are not out there helping them and teaching them.

Rebecca: What would you encourage a faculty to do as their first step to get involved in P-12
STACY. I’m trying to think of one single first step…probably, reach out to your kids’ local school and listen…ask the teachers and the administrators…but especially the teachers…ask them what they need. Because the teacher will know. He or she may not know how to go about getting it, but they will know what they need. Don’t go in and be like “I know everything” when in fact you really don’t know everything about what it’s like to be a K-12 teacher…but go in and ask how to be helpful. Listen to what they say and honor the fact that they have to be standards-based.

Rebecca: Sounds like really good advice, but at times, it’s just that little encouragement of what that step could be is helpful. So, thanks for that nugget.

Stacy: Yeah. I mean, go ask. People really want you to.

John: …and in many disciplines, I think, there are organizations that work with elementary and secondary schools. In economics, there’s the centers for economic education spread throughout the country, who do work with middle schools and high schools in providing some educational resources. I don’t know how common that is in other disciplines. Is there anything like that in art?

Rebecca: There is something more general for art, but not for design. Design stuff is kind of under the umbrella of art which, depending on what your position is, you might not think that they’re entirely related. They are, but they aren’t, you know. It’s kind of complicated. {LAUGHTER]

John: You’re working on a new advanced course in engineering for high school students. Could you talk a little bit about that?

Stacy: Yeah, this is a project that has been near and dear to my heart, and some of my wonderful colleagues, particularly from the University of Maryland (Dr. Leigh Abts, in particular). We have been trying to get an AP engineering course started for 14 years now, and when we started, really, the colleges weren’t ready for it. The schools of engineering were not interested in it. They weren’t interested in accepting credit for it. They didn’t really see the value of it. Thankfully, that has changed over time, which I’m really excited about. The College Board got interested in AP engineering and created a framework for the course. They had to put the brakes on that for a little while while they focused on the redesign of the SAT, but now they’re back to being interested in it. So, my team approached the National Science Foundation who said “Yes, we’re on board with this. This needs to happen.” So, we’re modeling our upcoming work off the very successful work of creating the new AP computer science principles course which has been a highly successful course for AP…and it’s also really successful in that a lot of women and underrepresented students are taking the course and taking the test, so that’s super exciting. So, we’re trying to draw some best practices from that and, knock on wood, I hope that the NSF will approve our final proposal to begin the work later this fall. The basic idea around it is we’re going to finish the framework…make sure it’s right…and we’re going to be developing a sample curricula and sample professional development for that. As you may know, with AP courses there isn’t any one set curricula that you’re expected to follow. You can do it however you see fit. You just have to make sure it fits the framework. So, we’ll be developing some sample ones and then we’ll be partnering with high schools. We’re hoping for about 70 high schools all over the country and a lot of diverse settings to train their teachers and have them work with our students and do engineering design at that level. We’re looking at having ultimately an assessment that is a bit like AP art studio actually in that we’re hoping for a portfolio process where you would submit engineering design work that you’ve done over the course of the semester or the year, and then that work would later be evaluated for possible engineering credit.

Rebecca: That sounds like an exciting advancement.

Stacy: I hope so. we’ve had over 110 Deans so far say that they’ll be interested in giving some sort of credit for it. I think it’ll be interesting to see how the universities handle it. I think some might give credit for their actual Intro to Engineering course while others might give it as more general science and engineering credit. I think that the universities now see it as a great way to get more STEM literacy in our population and I think they’ve started to see it as a great way to get more diverse students into their programs, because they will have done engineering at a younger age and done it in the more friendly confines of their local high school.

Rebecca: …and perhaps introduced populations who aren’t really familiar with the field at all to what the field is rather than expecting college students to just magically know what all of our disciplines are.

Stacy: Right, that’s true of a lot of disciplines, so it’s not just engineering.

Rebecca: Yeah, exactly.

Stacy: There aren’t tons of schools that have economics in them but you’re probably not gonna major in it if you don’t know what it is.

John: Actually, I think most schools do now have economics. It’s part of the New York State curriculum and I think most states do have at least one semester economics course. But, it took many many years before that was widespread.

Stacy: We’re trying to catch up with you, John.

John: It’s not always taught by people who know much about economics, unfortunately…. [LAUGHTER] …which is one of the reasons why the Centers do so much work.

Stacy: I would mention that if someone is interested in engineering education, I know that there are now actual programs in engineering education. You can get graduate degrees in it, and I would also steer them towards the American Society for Engineering Education. It’s a wonderful Society. It is the place to go to for what the best pedagogy is in engineering education, and to find your people there. They have lots of divisions, some are specific to your field of engineering and then we have a wonderful pre-college division there as well. So, it can be a great resource if anybody who’s listening wants to jump in jump in and join it.

Rebecca: So, we generally wrap up our podcast by asking what’s next?

Stacy: What’s next? Well, I feel like I’ve hit some of the “what’s next” because I’m in this great transition period in my career, which I’m excited about. So, I’m hoping that what’s next is that personally I’m able to view engineering education from preschool all the way through 12th grade and then into college and look for “How does that pathway work?” Are there things that are missing? Are there things we should be doing differently? So, I’m excited about taking that long view across engineering education and I’m always looking for new collaborators and people who are as excited about the field as I am.

Rebecca: Thank you so much for joining us. I hope that you’ve motivated a lot of others to think about their own disciplines plans from the elementary level all the way up through the university level. Sometimes, that longitudinal perspective can really help us have better perspective on what we’re teaching in higher ed.

Stacy: Definitely. Just to think about like “What matters? “What’s actually important?” It’s easy to get caught up in the minutiae of these little things you have to be sure you’ve mentioned to your students. Not really. Focus in on the big thing.

John: …and if you really want to do something about the gender imbalance in STEM fields you do have to reach out earlier because by the time you get to high school, people have already been sorted out. So, it’s really important to do that sort of work early.

Stacy: Very true. Most of the girls are getting sorted out late elementary to middle school, at the latest. So, you’re absolutely right there.

John: Well, thank you.

Stacy: Well, thanks for having me. This was fun. I appreciate you reaching out to me John, I was flattered.

John: Thank you, Stacy. We very much appreciate you joining us today.
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer.

42. Flipping the classroom

Posted in active learning, classroom management, evidence-based practices, higher education, online learning, pedagogy, scholarship of teaching and learning, teaching
August 15, 2018

Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode, Dr. Dominick Casadonte, a Chemistry Professor at Texas Tech University, joins us to discuss research and best practices related to flipped classrooms.

Show Notes

  • Camtasia
  • Mediasite
  • Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement. International Society for Technology in Education.
  • Sams, A., & Bergmann, J. (2013). Flip your students’ learning. Educational leadership, 70(6), 16-20.
  • Bowen, J. A. (2011). Rethinking technology outside the classroom. Journal of Music History Pedagogy, 2(1), 43-59.
  • Bowen, J. A. (2014). The Teaching Naked Cycle: Technology Is a Tool, but Psychology Is the New Pedagogy. Liberal Education, 100(2), n2.
  • Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve student learning (ed.). Jossey-Bass
  • Belford, R. E., Stoltzfus, M., & Houseknecht, J. B. (2015). ConfChem Conference on Flipped Classroom: Spring 2014 ConfChem Virtual Poster Session. Journal of Chemical Education, 92(9), 1582-1583.
  • Stoltzfus, J. R., & Libarkin, J. (2016). Does the room matter? Active learning in traditional and enhanced lecture spaces. CBE—Life Sciences Education, 15(4), ar68.
  • Stoltzfus, Matthew (2016). Engaging Students in the Flipped Classroom (video)
  • Coats, H. J. (2016). A study on the effect of lecture length in the flipped classroom (Doctoral dissertation).
  • Casadonte, D. J. (2016). “The Effectiveness of Course Flipping in General Chemistry – Does It Work?” ACS Symposium Series, “The Flipped Classroom”, December 2016 (Book Chapter) The Flipped Classroom Volume 2: Results from Practice, Chapter 2, pp 19–37, Chapter DOI: 10.1021/bk-2016-1228.ch002, ACS Symposium Series, Vol. 1228, ISBN13:9780841231627eISBN:9780841231610,
  • POGIL.org

Transcript

Rebecca: Flipping the classroom is one way to dedicate class time to active learning. In theory it sounds great, but how do you flip a classroom without flopping? In this episode we discuss research and best practices.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is dr. Dominick Casadonte, a chemistry professor at Texas Tech University. Dr. Casadonte is recognized as a global leader in flipped learning by the Flipped Learning Global Initiative.
Welcome.

Dominick: Thank you very much.

Rebecca: Today our teas are:

Dominick: Well, my usual afternoon beverage is an iced green tea with three pumps of raspberry, but since I’m getting over a cold, today I’m drinking a hot green tea with vanilla, lavender, and honey.

Rebecca: That sounds really nice. Finally, a tea drinker.

[LAUGHTER]

John: I’m drinking Irish Breakfast tea.

Rebecca: …and I’m drinking chai today.

John: You’re recognized as a an expert on flipping the classroom, and you’ve been doing it for a while. Could you tell us a little bit about what a flipped classroom is?

Dominick: Sure. The usual paradigm, if you think about it with regard to teaching, is lecture, homework, lecture, homework, lecture, homework, tests, lecture, homework, lecture, homework, and so on… and the problem with that format is that it’s not conducive to deep learning… with the exception of maybe very bright students or very bright asynchronous learners. Trying to keep up with a lecturer is often difficult. A student, for example, may not have gotten down everything that they wanted to write down… they may not have gotten it down correctly… or it might be fragmented… or perhaps they just didn’t understand well enough to write it down in a meaningful way, or a way that’s meaningful for them. Then they try to go home and they try to do homework, but since they didn’t really understand the lecture in the first place, the homework is again difficult and they might only be able to work part of it. So then, the very next day, there’s a new lecture and a new topic. The normal sort of didactic approach to education often leads to what I’ll call fragmented learning, in the sense that there isn’t enough time in class to practice many of the active learning strategies that work so well because faculty are so concerned with getting the lecture done. In the flipped model, on the other hand, we flip the homework-lecture-homework paradigm so that the students’ homework is to watch a lecture online. Now it can either be PowerPoint or a video and different people use different techniques. It can be done on Camtasia, it can be done through Mediasite, it can be done through a variety of different platforms. It could be done on YouTube for example. So, they’ll watch usually a video or a lecture online before they come to class, and then they may or may not do online homework prior to coming to class as a part of the pre-class experience. Then, once in class, time can be spent, for example, checking the students knowledge… clearing up muddy points or any misconceptions they might have… working advanced problems. But now, in this particular case, class becomes more of a discussion and you can use a lot of the active learning strategies that really engage the students in learning the material. By flipping the classroom, we freed up time to really engage and interact with our students and it helps with their learning in significant ways which I can talk about later.

John: So, in a traditional class, you tell students what the content is in class, and they come in with very different backgrounds and some of them are able to pick it up, others get lost along the way, and then they’re sent out to do homework… where students seem to have the most difficulty on homework or tests… and we’re pretty much leaving them alone to do that. But, here in the classroom, you’re able to work with them and help them through some of the issues.

Dominick: Right… and so, for example, in the classroom with homework, now they have a mentor who can help them work through the more difficult problems to help guide them along. There are models of the flipped classroom where there are peer mentors in the class who can help and walk around and mill around as well. So, there are opportunities for group work. There are opportunities for group discussion. You can engage the topics in a much deeper way than you can by just simply lecturing at students. So, it’s a very very non-passive way of teaching.

Rebecca: What prompted you to get into a flipped classroom model?

Dominick: Desperation, I would say. [LAUGHTER] I first started teaching an online class in the fall of 2007 as part of a multidisciplinary science master’s program of which I was a part here at Texas Tech… and getting everyone on the same platform at the same time, my thinking was that lecturing to them live online was a waste of time, basically. The question that motivated me was the same one that motivated John Bergmann and Aaron Sams around the same time at the high school level up in Colorado. Namely, what could I do to use my classroom time more effectively? What could I do to use my online time more effectively? …and I thought, “Well, let’s take the lecture out of the classroom.” So, I pre-recorded my lectures. I had the students watch the lectures before we got together in the online environment, and then we spent the majority of our time having discussion and working problems…. and these were teachers at the time and they were very very enthusiastic about the ability to actually discuss the material that they were trying to learn… and so then I thought “Well, hmm, if that works really well on an online format why don’t I try it face to face?” …and so then, starting in 2008, I adopted this to my general chemistry class. The following summer, a high school teacher that I had as part of a workshop said “Oh, I see you’re flipping your class” and I said “Flipping, what’s that?” ‘Cause at the time, there were a lot of different terms for flipping. It was called: time-shifted instruction, reverse instruction, blended instruction, all sorts of things. Flipping is a term that’s really sort of stuck.

So, I started out of a sense again of “How can I engage my students more effectively in the classroom?” and once I realized that it worked swimmingly in the online environment and then I said “Well, okay, will this work in the face-to-face?” …and it worked even better there. By that point, there are pockets of people around the country who are doing this, and they used all sorts of interesting terms. Jose Bowen at SMU, who’s an art professor, used the term “naked teaching” because it can be unsettling when you walk into a classroom and not have the comfort of your lecture notes to be able to project or read to the students. So, very often I will walk into the classroom and say “What are we going to talk about today? or “What would you like to talk about today?” You can’t do that unless you feel comfortable about the subject material and you have some expertise, but it’s a great way, in the long run, to really, really impact students.

Rebecca: You mentioned both doing this in an online environment and also in the classroom. Is there a difference between your experience in both?

Dominick: Yes, it’s so much easier face-to-face. To be able to walk around to gauge what’s going on in the classroom, their level of understanding, and how learning is happening. The power of direct peer-to-peer contact should never be underestimated, I think. Now, in the online format, my focus was more on the development of a learning community working on a particular topic, rather than in peer-to-peer mentoring and things like that. So, they’re very different approaches, but you do develop a sense of community in both, I think.

John: Now, with your online classes are they synchronous or asynchronous?

Dominick: My online classes are synchronous, in the sense that everybody meets in one place at one time. We have various software programs that allow us all to be in the same place. Yeah, so they’re all together at one point, and that’s kind of interesting because I have people from all over the country who are taking these classes. They’ve never actually physically met, at least for the first class that I teach. It’s interesting trying to, in the discussions, get a sense of the personalities of the people who were providing discussion.

John: Are they participating in video formats or is it just audio or text?

Dominick: I’ve done both. I’ve done video and audio and depending on the bandwidth and the number of students in the class both can work well.

John: How large are your classes?

Dominick: Well, my online class had 24 students in it. So, a fairly large, I think, from an online perspective class. It wasn’t one of these very very large classes like you might see at MIT, for example, but 24 is a good class in terms of bandwidth… trying to get everybody in the same room at the same time… and my face-to-face classes… they’ve ranged anywhere from a low of 25 to a high of 150.

Rebecca: You mentioned two different techniques that you use both in online versus in person. So, online you mentioned community formation and then in person you mentioned peer to peer. Can you expand upon each of those?

Dominick: Yeah, my first adage for teaching is know your audience, and in the face-to-face environment I was working largely with in-service teachers who were trying to develop a more significant content knowledge of chemistry, and so there the idea was the development of a supportive community where these teachers could bring their ideas to the table in terms of not only the content that I was teaching them, but also how they could then apply that content in their own classroom settings… and in that regard they were able to help teach each other techniques that they could use in the classrooms based on what they were learning in terms of the content. There really was a community focus, sharing knowledge as opposed to just gaining knowledge. In the face-to-face classroom, it’s more a sense of the students trying to understand the content at a deep level. There I found that it is true that when you teach something you really hopefully really understand it and so peer-to-peer mentoring is much more effective there. So, we tend to work in groups… we tend to work with dyads… two people working next to each other… and then sometimes I will just have people go to the board… but once again it was in that context of community because I think it’s very important, if you’re going to do flipping, and do it well, that it’s an active, encouraging, engaging classroom experience. If my students go to the board, whether they get the problem right or wrong, the class gets into a habit very very quickly of applauding the student for their attempt… whether or not it’s right or wrong… and then we debrief. We talk about what works… what doesn’t work… So, it really is both peer to peer and community building there as well. But, the emphasis is more on the individual in the face-to-face classroom, I think.

Rebecca: It sounds to me a little bit like the choices that you’re making online and in-person aren’t necessarily because of the medium, but rather who’s in those particular classes. Am I hearing that correctly?

Dominick: That’s right, and one of the nice things about flipping is that it is such a rich environment in which to work. As I mentioned earlier, the pre-class videos can be video lectures, they can be audio lectures (if that’s appropriate), they can be PowerPoint presentations, they can be any number of things. In the in-class experience, it’s an active learning environment. so you tailor it to the people in the classroom. For example, if you’re trying to teach in a flipped environment of the class of 24, there you can do all sorts of things that promote individual learning in ways that it’s a little bit more difficult to do in a class of 300, for example. But, I have a good friend, Matt Stoltzfus at Ohio State, for example, who routinely flips like general chemistry class of 600 students, and he’s able to give them as close to a personal learning experience as one can, I think. The point that I’m trying to make is that the flipped environment is a very rich one and it allows you to tailor the learning experience to your class specifically.

John: Do you create your own videos or do you use ones created by other people?

Dominick: I actually create my own videos. I have my own recording studio in my office… I have a Mediasite setup…. I have a video camera… I have a document camera… and I have a wonderful microphone…. and so every couple of years or so I re-record all of my videos. Now, I do use a mixture of other formats. So, for example I do have post-video homework that the students have to do online before they show up to class… and that’s done using an online learning platform that we have here at Texas Tech through a national distributor. The advanced problems that we work in class come both from the textbook that we use and also from problems that I develop. So, it’s sort of a hybrid. But, the pre-lecture videos I actually produce…… and one of the things that studies have shown is that students develop a certain sense of identity with regard to the person teaching the class. Some people, when they’re starting to flip, might want to just use Khan Academy videos, for example, or things like that. But the studies show that the class wants to see folks on the videos (or your voiceovers if you’re using PowerPoint with voiceover)… they want to see the professor who’s teaching the class, because that’s their professor… and so they develop ownership, if you will. Plus if you’re using other media, for example a Khan Academy, they may not be teaching it exactly the same way that you want to teach it, and so then you have to either reteach or undo. There’s a quality control issue there. So, it’s just easier, if you’re gonna do flipping, to make your own videos… and there are so many different ways of doing that now that there’s really no excuse.

Rebecca: That had been my experience as well when I’ve done videos in my classes. The students really liked the quirkiness or knowing that it’s the same person that they had in their classroom and if you try to slip in something else occasionally they really didn’t like it.

Dominick: …and a lot of people have a perfectionist tendency and really want their videos to be really super perfect. Well, once again, studies have shown that that’s not really what students want. They want to see the foibles. They want to see you as you are. If they know that you’re going to say “um” or “uh” in the classroom then if you don’t say “um” or “uh” in your videos, then they’re gonna say “Is that a robot teaching the class that looks like my professor?” So, it’s okay to be human when you’re doing the pre-lecture videos. But, I think one of the things that often hangs up people when they’re starting to do flipping is this notion that it has to be perfect, and it really doesn’t.

Rebecca: I think they really appreciate when you make mistakes and things, too. I know that my students did when I do like a coding mistake, I’m like “Whoops, I made a mistake..” and go back and fix it and explain what I did and why it was wrong. We all make those kinds of slips and errors and things and we would do it live. So, it’s kind of nice to do it in videos, too.

John: It makes you seem more human by doing that.

Dominick: One of the nice things about them seeing you make mistakes is that it gives them permission to make mistakes. They don’t have to be perfect. I’ve had experience with students in classrooms where they come in and they’re intimidated… they’re shy… they’re just afraid that they can’t master the materials. So, seeing somebody make a mistake who is an expert gives them permission to make mistakes… and one of the things that that really does is it empowers them to learn, because at the end of the day when somebody’s trying to learn they’re going to make mistakes and I give my class permission to make mistakes. In fact, I tell them you have permission to make mistakes while you’re learning. After you’ve learned something, that’s a little different. If you’re an engineer, for example, and you’re building a bridge, I don’t want you making mistakes. But while you’re learning… absolutely, make mistakes. Part of education is this movement from novice to expert and in that process one makes mistakes. So, I told my class “You have the right to make mistakes” and I use an example. So, let’s say you’re a five-year-old and you’re trying to learn how to ice-skate you fall down, what does a five-year-old do? They laugh, they brush themselves off, they might giggle a little bit, they get up and then they just skate some more. Now, imagine you’re an eighteen year old and you’re learning to ice skate and you fall down. Most of the time, people stand up and say “Did anybody see me?” and they worry about what people are gonna think, instead of just getting up, laughing, and moving on. So, what often happens is the 18-year old never learns to ice skate, whereas the five year old, who’s willing to make mistakes, learns. So, I tell them it’s okay to make mistakes while they’re trying to learn… and also it’s about empowering students to be able to have confidence in themselves… and we talk about this a lot. We did a study of what motivates students in the flipped environment, and part of it is the confidence… the autonomy… that they develop in the flipped environment. When they really think they’ve got and they really understand, there’s a certain level of “Geez, I understand this. I can understand the next thing…” and so on and so forth. That sense of confidence really improves their educational experience.

John: How long are your videos? Do you tend to have very long ones or do you chunk them up into smaller chunks? and what would you recommend in terms of video length?

Dominick: We did a study a few years ago on the optimum length for flip videos, and it came about because our book dealers: McGraw-Hill, Prentice-Hall, Cengage… those are the top three that we have here at Texas Tech… were telling us that they were creating videos and they’re creating these five to seven-minute videos… and I said why are you making five to seven-minute videos?” and they said “Well, everybody knows that the Millennial generation has a five- to seven-minute attention span and so they want five to seven minute videos.” I said: “But, what is best for learning and improving learning outcomes? I don’t care what their preference is. What’s best for them to learn?” He said “Well, we don’t know.” I said “Well, do you have any data or evidence that shows the five- to seven-minute videos are really great for learning?” and they said “No, but we would be interested if someone would do a study and tell us.” So, I had a graduate student who embarked on a study of lecture video length. We set up essentially short video people… those are people to watch five to seven-minute videos… and we did this with master videos, so each video was probably 40 minutes to an hour long. We put stop signs in the video and when they hit a stop sign they would stop. They would work some online homework and then they could pick up again… or they could go off and do something else. but the short videos were five to seven minutes and then we had a long view group, as it were. We allowed them to decide which videos they wanted to do and not surprisingly, 62% decided to do short videos. So, I have no problem with the book dealers’ notion that Millennials prefer shorter videos. But, then we let the semester go on and we didn’t force them to stay into one group or the other. We let them move and we watched their video habits. We weren’t video stalking them, but we could watch their lecture habits using the Mediasite analytics down to the millisecond.

What we found was that 60% of that short video group switched to the long video length, which is very surprising to us, and we did also a variety of assessments. We looked at online homework grades. We looked at quiz grades. We looked at exam grades. We looked at final exam grades. We looked at the American Chemical Society standardized tests that we give as a pre-post and what we found was that there was a subgroup of the long-view group that watched the video as a long video, but then they stopped at specific points to either have a snack with a friend, go to the bathroom, whatever they needed to do… and we called them the long pause people. It turned out that in every assessment that required global understanding… so final exams, ACS exam individual exams… the long-pause viewers actually scored one standard deviation higher than the short viewers. In fact, the short viewers had the worst learning outcomes of all three groups. Then we gave them Likert scale questionnaires, and we also gave them open-ended questions, and we said “Why did you make the switch? What do you find?” and they said “Well, it just got too fragmented to look at these short videos and we couldn’t take what we saw in video A and connect it to video B and so on and so forth, especially if several hours had gone by, because we then had to go back and watch video A again to remember what we forgot. But, if we did the long videos we were able to just put it all together.” Also in that we found that the best optimum video time. So, what constitutes a short video versus a long video for Millennials is 20 minutes or less is short, 30 minutes or more is long. So, 20 to 30 minutes is the sweet spot for video length. That’s what the study showed us and we’ve just submitted that for publication.

Rebecca: Have you adjusted how you’re teaching based on that information?

Dominick: Yes, my videos are roughly in the range of 30 to 40 minutes, but I tell my students “Take the time you need. Take breaks. It’s going to help your learning outcomes.” I also share with them the results of our studies, because I think if you’re going to, in my case if you’re gonna be a scientist, you should be data-driven. So we want our students to know that we’re not just telling them this because it’s something anecdotal, but rather it comes from data that we’ve collected.

John: Have you thought about controlling for self selection and randomly putting students in groups? Because one concern I’d have with that is that it could be the case that those students who select the short videos might have done less well no matter which group they were in or vice versa… although you do have switchers in there.

Dominick: Yeah, I’ve actually been very lucky in my studies that I teach our honors general chemistry sections… and so I look at the SAT scores… I look at their previous class performance… and it’s a very very, as much one can have, a homogeneous group. So there’s really not much of a selection bias, I think, as far as the study goes.

Rebecca: You’ve also done some research on the flipped classroom approach in general, not just the video length. Can you share some of your findings?

Dominick: Sure. We found out a lot about the flipped environment over the past 10 years or so. As I mentioned a minute ago, I’ve been very lucky in that I tend to teach very bright students. I did a five-year longitudinal study on the effect of flipping which has been published in American Chemical Society monograph and we found that the average exam grade increased by 9.2 percent over that five-year period, and that the largest gains in learning came during summative assessments (for example during final exams and externally developed independently normed exams like our ACS exam). We also have done work on what motivates students to do well in the flipped classroom and we just recently presented at the biennial conference on chemistry education regarding the effectiveness of peer mentoring during the flipped classroom. The results there were very astounding to me. It shouldn’t have been because peer-led team learning has been around for more than 15 years, but nonetheless, in trying to tweak the classroom, the addition of the peer mentor took what was already much better than had been before and improved it dramatically. I’ll give you an example. On the American Chemical Society end-of-term exam, without the peer mentor present… So, I give my students incentives if they score above the 95th percentile on that exam, and it’s a challenging exam, they get an automatic “A” in my class. They don’t have to take the final because this exam is normed against thousands of students around the country in a variety of different university settings. Pre-flipping, I had zero to one student scoring above the 95th percentile; post-flipping the average was roughly nine. So, it increased tremendously. With the presence of the peer mentor, the number went from 9 to 34 and so almost a six-fold increase in the number of students scoring above the 95th percentile. The only difference being, since I used to run the discussion sections that I subsequently allowed the peer mentor to do, that was really the only difference in the environment as far as I could see in terms of controlling for the different factors, except for the obvious one of different students. But she did this in the fall of 2016 and the fall of 2017 and there were 34 in one in the fall of 2017, 33 in the fall of 2016. The percentages of the class actually increased. This is kind of reproducible, if you will. So, we’ve looked at a variety of different things, and with regard to motivation we looked at that from self-determinacy theory and found essentially three things that really sort of motivate people: one is autonomy, second is pace, and the third is responsibility.

In the flipped environment, this sense of autonomy… the sense that “Oh yeah, I am really learning something” is very important to the students. The pace, the fact that they could watch the videos in their pajamas, for example, was very important to the students as a motivating factor… and responsibility, the fact that they had to take responsibility for their own education as opposed to being sort of spoon fed in a lecture format was something that motivated them as well. That student just graduated in December, so we’re now preparing that for publication. So, we looked at a variety of different aspects of flipping.

Rebecca: Can you clarify about your peer mentor model? The students are watching videos outside of class, and then they’re coming to a class with you, and then also recitation session with another student?

Dominick: They watch a pre-lecture video. They do online homework. We have a class discussion… work problems… and then there’s a separate recitation section… and historically I have done all of that. I had a very bright student who really wanted some teaching experience and she said “Would you let me run your recitation?” and I, like many faculty perhaps, don’t really want to give up control of my classroom environment. So, it took a lot of cajoling on her part to get me to do that…. and I said “Well, let’s look at it and see what happens.” She took over the recitations and… talk about a motivated young lady… she would provide review sheets for them… she would do all sorts of things that I would do, but the way she would do it, I think, spoke to the students so much more effectively than I was able to do. I think that’s one of the real reasons why their scores went up.

Rebecca: She knew how to meet them where they were at in a way that, as we become more of an expert in our field, we lose touch with that.

Dominick: Yeah, that’s exactly right. So, one of the things that I’ll be interested in following is that there was a recent study that Prentice-Hall published about the Millennials versus the Gen Z students… and one of the things they noticed is that, while there was about a sixty or so percent “like” amongst the Millennials for using computers and computerization and video and things like that, that number was almost half for the gen Z population. It’s almost as though it’s such a normal part of their life, the gen Z population, that there’s nothing special… there’s nothing unique about it… there’s no value-added to doing videos… it’s the sort of normal expectation. So, I’ll be very curious to see how this sort of flipped environment works over the next 10 to 15 years when the expectation is that they’ll be seeing videos at some point either prior or post classroom experience.

Rebecca: …definitely an interesting question.

John: You’ve organized several symposium on flipped classrooms. What are some of the biggest takeaways from those symposia when you bring people from many different disciplines together? Are the results pretty similar across disciplines or do they vary substantially?

Dominick: Well, I think the first thing that struck me with regard to these symposia is how diverse the flipped environment really can be. Since active learning occurs during the classroom time, there’s almost as many different active learning strategies as there are teachers, and so no two flipped classrooms are the same… and that’s the first thing that I learned. The other thing that I learned is that you have to be committed to flip. You can’t do it half-baked. If you try to do flipping, there was one example of a professor who said “Well, all this is is what we’ve always done. You tell the students to read the material before they come to class and then we have a discussion.” So, he said “Okay, I’m going to ‘flip my classroom’ and just have them read the textbook before they come to class and then we’ll have a discussion.” He found very quickly that the students weren’t reading the textbook, so there was very little discussion going on, which frustrated the professor and frustrated the classroom and set up more of an adversarial relationship.. and it was the worst teaching experience he ever had and he said “I’m never going to flip my classroom again.” So, one of the takeaway messages, and this was reported at one of the symposia, is that if you’re really going to flip your classroom, you’re in for a dime, you’re in for a dollar. You do it as well as you can and be very concerned about what you’re putting in and what you’re expecting to get out or it can be a very very bad experience. Now, I will say this, that almost everybody when they get up on board the flipped bandwagon, especially if you’re using technology prior to classroom, it’s hard at first. That’s the other real take-home message. It takes a lot of time to flip your class the first time. But, once you do it, it actually is much more enjoyable. It’s actually easier, I think, than the regular didactic approach. Those are some of the take-home messages.

Rebecca: Can you talk a little bit more about some of the other challenges faculty might face if they’re doing it for the first time?

Dominick: Sure, yeah. The first and biggest challenge that people have is time. Don’t decide you’re gonna flip your classroom two weeks before the start of a semester. It can be the most horrible experience you’ve ever had as a teacher, because students have an expectation that each lecture is going to be there for them. As we all have various things come up or university requirements… meetings… things like that… and you may find yourself at 3 o’clock in the morning trying to record a video for dispersal at 8 o’clock in the morning. So, don’t wait till the last minute. That’s the biggest sort of thing. That’s one of the reasons why people are sometimes a little bit risk-averse with regard to flipping their classroom.

Other things that have come up: saying “Oh, I can just use Khan Academy videos or videos that are on the web” …and they haven’t previewed them, for example, and then they find that the students have a very different concept of the material than the professor has and then you end up spending a lot of time: “Well, which one do we believe? Do we believe the video that you showed us? or do we believe what you’re telling us?” and so it can create an environment which doesn’t propagate trust in the learning experience. Other issues are: online homework versus homework that you put together for the students… what are you gonna do during class time? how are you gonna fill that time. If you’re not familiar with active learning strategies, that can be very daunting. For example, you walk into a classroom and the students have already watched a video… they’ve done some homework… So, what’s your value added? If you’re not used to active learning strategies, that can be very difficult the first time that you’re doing it…. and then do I need to change my assessment strategy based on the fact that now I’m using a different kind of pedagogy? So, there are a lot of different moving parts and I think putting all those moving parts together can somewhat be inhibitory for people who are trying to flip for the first time.

John: What types of active learning strategies do you use? You’ve mentioned some group work on problems, but could you give a few examples of types of activities you use during the class sections?

Dominick: Sure, there’s group work first of all. I send students to the board. I’ll pass out file cards, for example, and ask the students to put down one thing that was unclear in the video or something that they really would like to learn about in addition to the video. So, it’s not just muddy points, but it’s also how do we expand and extend. Because in my case it’s an honors class and I want to give them a little bit more than the normal amount of material and experience, and so those are some of the things that we do. We have to make molecules and structures and do all sorts of things, and so I use human atoms. I have volunteers come up to the front of the class and they have to then make molecules. They have to develop particular structures. They have to show how they bond, how they vibrate, how they move. What they do. So, they have to actually sort of insert themselves into the molecular dynamic, if you will. They’re trying to understand that, as an atom won’t understand its environment. The first couple of times, students aren’t used to that level of kinesthetic learning, but once they get it, then I usually have a fair number of students who are willing to volunteer and come up. Because, while they’re doing that, we’re also discussing, and I’m asking the class: “Okay, so why is it that this particular atom won’t bond here? What’s wrong with this?” and so it’s a real discussion… but now using human beings as the models rather than just making stick figures or things like that.

So, I try to move students through a variety of different learning environments that engage them not only visually, auditorily, but also tactilely. and kinesthetically, and that’s somewhat unusual, I think, in chemistry classes. Because, once again, most people don’t think of chemistry as a visceral activity in many cases.

Rebecca: Actually, it sounds like a lot of fun.

Dominick: Well, and it is, and I’ll tell you I walk into my class the very first day and they’re all very respectful, because Texas students typically are very respectful and they’re honors students. So, they really want to make a good impression. I usually start my class laughing, telling them this is the last time this class will be quiet. A noisy class is a learning class, a quiet class can be a sleeping class, I don’t know. So, yeah, my class is always very, very engaging, I hope.

Rebecca: Have you ever had the moment when you’ve asked your students “What do you want to talk about today?” and then not have anything they want to talk about?

Dominick: Yes, I think every faculty member has that “Oh, my gosh” moment. I always come prepared with questions, so if they don’t have anything to talk about, then I’ll ask them “Well, what did you think about this particular part of the video?” or “Did you really understand this?” or “Let’s take this concept and move it farther” because one of the things I never do in my classroom is just do a rehash of my video. I figure they’ve watched the video. So, I might say “Well, okay, you saw this but how could we apply this in this other setting?” …and then if they really didn’t understand it, then I’ll be able to tell in a heartbeat whether or not there are real questions because they really don’t have a good sense of understanding… and then I can go back and say “Okay, at what point is this breaking down for you? How did this not work?” Silence kind of tells me that there is usually a breakdown somewhere… and so I try to address what that breakdown is and try to correct it. So, that’s what I meant earlier when I say one of the things I do during my class is try to clear up misconceptions… try to address muddy points… and just make sure that they really understand the lecture part as well. That can take anywhere from five minutes to 45 minutes. My classes are an hour and a half long. Depending on how difficult a lecture was… on what their level of understanding is… So, I want them to have deep understanding of the content and so if they’re coming in silent then I worry that that depth is not there.

John: When they see the videos, you have them take tests. Are the tests the same for each student? or do you vary the questions? Is there some randomization there?

Dominick: Well, no. So, we give departmental exams, so the exams are the same for every student, basically. I know many universities have a format and they can vary the numbers that are put in, for example, but no, my students really… they’re fairly separated when it comes to the exam, so I don’t really feel the need to give each one a separate numerical set of calculations.

John: …and it sounds like many of your students are honor students where that might be less of an issue.

Dominick: Right, and we have an honors code through the Honors College, and I tell them the first lecture if you’re cheating you’re out of here. If you’re cheating you’re out of the University… and I’ve had the unfortunate occasion to have students suspended from universities… not at Texas Tech, but at other places I’ve been… and so they understand that I’m very serious about that.

John: On those tests, do they have multiple attempts or just a single take on the test?

Dominick: Well, my tests are usually set up so that they’re half multiple choice, half free response. Because at the end of the day, I tell my students “Qe don’t live in a multiple-choice world… A. agree, B disagree… you know, we don’t… and so I need to know how you’re thinking. While multiple choice exams are expedient in terms of grading, they don’t let me know what you know. So, I give them both because some students like multiple choice exams. They think they’re really good at them. Some students really want the free response. So, it’s a mixture of both. I try to give them a rich assessment environment as much as possible.

Rebecca: So, to follow up on what John was asking, are your homework assignments kind of a multiple attempt to help learning? or is it a one attempt kind of thing?

Dominick: It’s a one attempt. Yeah. Now, what I do allow them to do after they’ve provided their answer, is that I’ll allow the question to be open so they can go back and review it, especially if the answer is wrong…. and I allow them to do that to help them review for the exams as well.

John: So, they can go back and retake it, but only the first attempt counts towards their grade?

Dominick: Right. So, they have to think about what they’re putting in there before they put it in. Because in some cases, I’ve heard stories of students who put a wrong answer in purposefully and then the online learning environment gives them hints or tells them how to work a problem just like that… and then they go back and they have another problem with just different numbers but they’ve already been coached essentially in terms of how to answer the problem. Once again, perhaps it’s because I think you have to do things right eventually at the end of the day, I really want them to get it right… and these are… once again… they’re relatively straightforward pre-class questions. Their designed as just-in-time or warm-up questions. They’re not multifaceted. The questions we’re getting in the class are really challenging problems. They’re challenging problems using an honors book. So, hopefully they differentiate between those.

John: Are they graded on the problems they do in class as well?

Dominick: They’re not. Because part of that environment, once again, is to have groups… have community… have a mentoring process… and so, the ultimate goal in that whole process is the solution of the problem. So, I’ve already tested them in the online learning environment. I’ll test them on the exams. I’ll test them on quizzes. But, in the class I want the process of how to solve the problem come forth and not the grade be the most important thing.

Rebecca: So, we always end or wrap up our podcast with the question: what are you gonna work on next?

Dominick: Well, I have a pretty active and diverse chemical education research group and with regard to flipping, specifically, I mentioned that I had a recent PhD who looked at motivation in the classroom and what we found there were there were basically three reasons that motivates students to want to do well: autonomy, pace, and responsibility. Through the flipped environment they learn how to develop confidence in their ability to learn, and secondly they liked that the class is largely self-paced and they get to watch the videos in their pajamas if they want to, for example… and finally they really appreciate the fact that they are responsible for their own learning. So, we’re going to be looking at the role of metacognitive intervention as an autonomy motivator in the flipped classroom. That is to say, if we help them think about how they’re thinking during the early parts of the flipped classroom, do they proceed to confidence in their ability to learn that much faster… and we’ll also be looking at how the flipped classroom, especially with community building activities and community building learning strategies can improve the learning outcomes among historically underrepresented communities in the sciences… though especially communities where the notion of family and community is so important in their lives… that are not necessarily in the classroom. So, those are the two areas that we’re going to be moving into with regard to flipping… and I have a number of other projects that are not related to flipping as well. So, it’s a very diverse group of questions that we’re trying to answer, but once again I think that the flipped environment is a very value-added environment for both the students and the faculty… and so I think it’s a mature pedagogy in the sense that we talk about process oriented guided inquiry learning (POGIL) being mature and peer led team learning activities (PLTL) as mature pedagogy. Service learning is another mature pedagogy that has matured over the last 20 years or so… and I think it’s now safe to say that flipping is a mature pedagogy. In fact, at the biennial conference on chemistry education, there was a wonderful paper doing a meta-analysis on flipping and the presenter showed that in terms of looking.. I think he looked at 18 or 19 different papers on the flipped environment… and he found that, in general, there’s about a 30% improvement in student learning outcomes and it’s even better in organic chemistry than general chemistry (which was surprising to me). But, nonetheless it really does improve learning for students and that, in the final analysis, is what we’re trying to do.

Rebecca: It sounds like some really interesting projects. We’ll be looking forward to finding out what you find out.

Dominick: Thank you.

John: Well, thank you. this has been fascinating.

Rebecca: Yeah, thanks so much for spending some time with us today.

Dominick: Thanks, I’m gonna have another sip of tea. Thank you.

Rebecca: Thank you
[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Music by Michael Gary Brewer.

41. Instructional Communication

Posted in classroom management, higher education, inclusion, instructional communication, learning, pedagogy, professional development, scholarship of teaching and learning, teaching
August 8, 2018

There is often a misperception that being a well-liked, kind and caring faculty member comes at the cost of rigor or high expectations. In this episode, Dr. Jennifer Knapp, an expert in the field of instructional communication, joins us to discuss strategies we can employ to make the classroom a positive and productive learning environment.

Show Notes

  • National Communication Association instructional resources
  • Mottet, T.P., Richmond, V.P., & McCroskey, J.C. (2006). Handbook of instructional communication: Rhetorical and relational perspectives. London: Routledge.
  • Chesebro, J.L., & McCroskey, J.C. (2002). Communication for teachers. Boston: Allyn and Bacon.
  • The journal Communication Education also contains many useful articles.

Transcript

Rebecca: There is often a misperception that being a well-liked, kind and caring faculty member comes at the cost of rigor or high expectations. In this episode, we turn to an expert in the field of instructional communication to provide us with strategies we can employ to make the classroom a positive and productive learning environment.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Dr. Jen Knapp, an associate professor of communication studies and an associate dean in the School of Communication, Media and Arts at SUNY Oswego. Welcome, Jen.

Jen: Thanks, John. Thanks, Rebecca.

Rebecca: Thanks for coming. Today, our teas are:

Jen: Black raspberry green tea.

John: Tea Forte black currant tea.

Rebecca: I’m having Prince of Wales tea.

John: We’ve invited you to join us today to discuss your primary research area, instructional communication. What does research in instructional communication tell us about creating a productive classroom environment?

Jen: So, I’ll start by telling you exactly what instructional communication is… and what we do. Essentially we’re talking about communication between instructors and students that enhances learning or perhaps in some way affects the learning process negatively. We’re more interested in how messages are delivered than the actual content of the course. So, we’re talking purely about communication behaviors by instructors and students and how that affects what goes on in the classroom, which should be learning.

Rebecca: Is your area of research focus only on in-classroom communication or does it expand beyond the classroom?

Jen: One of the things I research is out-of-class communication and I think maybe at some point we will talk a little bit about that, but primarily I focus on what is going on in the classroom – specifically what instructors are doing in terms of communication and how that affects students.

John: What can instructors do to create a better classroom environment?

Jen: There are a lot of communication variables related to instructional comm. The primary instructional comm bread-and-butter concept is this idea of immediacy – and immediacy has to do with increasing physical or psychological closeness between instructors and students… and the bottom line is, if you, as an instructor, engaged in these verbally and non-verbally immediate behaviors, there’s going to be more positive outcomes in the classroom for your students… specifically learning… but ultimately, what I think is really interesting, is that even on a nonverbal level, you can influence what’s going on with your students and how they are perceiving your messages… but also how they’re wrestling with the content. So, it comes in two flavors: verbal and nonverbal immediacy. We were talking about nonverbal communication… we’re talking about everything but the words. People will commonly refer to it as body language, but it’s also your tone of voice and how you use space and touch and things of that nature. E ven something as simple as eye contact can make a difference in terms of what’s going on between instructors and students in the classroom… engaging in vocal variety… but also using humor… calling students by name… all of these things can help increase the connection between students and instructors. Most people believe that the instructor-student relationship is an interpersonal relationship, or a type of an interpersonal relationship, which means you’re connected to each other in some sort of meaningful way. All the things that you value in terms of how you communicate with your friends and your family… a lot of that plays into what goes on in the classroom as well. People want you to make eye contact. People want to be around people who are funny. So, there’s a lot of research that suggests instructors that use humor in the classroom tend to get more positive evaluations, but also there’s more learning that occurs in the classroom if an instructor is using humor effectively.

Rebecca: Does that shift with culture?

Jen: Yes. All communication occurs within a context. Culture is our biggest context. Immediacy, in particular, is very culturally based. It is something that you need to be careful of. Most of the research that I do and that I’m familiar with has been conducted here in the United States with traditional college-age populations, but certainly if you were to travel abroad and perhaps you were to teach a semester away then these rules may not apply.

John: …and it might not also apply if we have foreign students here who have not adjusted to U.S. classroom climates.

Jen: Of course. Yes.

Rebecca: So, what are your biggest secret secrets? [LAUGHTER]

John: …related to teaching.

Rebecca: …related to teaching.

Jen: Oh… no one warned me that I had to divulge my… my biggest secrets today.

Let me go back to immediacy for a little bit and talk a little bit more about that and why that essentially is a positive thing. I don’t think I listed the outcomes. You’re perceived as more approachable… you are perceived as more student-centered… more responsive… you’re friendly… you’re open… and you are essentially inviting communication. So, if you engage in these types of behaviors you are going to invite communication. If you are an introvert, I don’t recommend that you try to be overly immediate because students are going to pick up on that and then they’re going to think: “Oh, well this person is friendly. This person is a good listener, so I want to spend time with them. I’m gonna visit with them. I want to get to know them.” So, you are inviting communication when you engage in these behaviors. But something you should also keep in mind, in terms of immediacy, and this is probably more of a personal choice for me… and other people may not agree… is that it decreases the status differential between you and your students. You are trying to give the perception (hopefully it’s not just a perception and it’s reality) that you care for your students… you are engaged… you are enthusiastic… they see that you’re passionate about your content… you’re moving around the room… you kind of work the room when you engage in these physical behaviors… and so it decreases the status differential between you and them. For me, I like that in my classroom. I don’t want to give the air of being the professor who has all the knowledge and the expertise and I’m looking down on everyone and being condescending. For me, I like to have… not an equal partnership… but I want my students to feel like they are a partner in what is going on in the classroom and anyone can share an idea. I can share an idea. It’s open. It’s friendly… and that’s important when you’re teaching something like interpersonal communication. You’re talking about relationships. Sometimes that class turns into a self-help class and everyone’s talking about their problems with their partner or their family. Everyone’s telling personal stories. You can’t not tell personal stories when you’re in that class. You don’t want anyone to feel like you’re being judged or that you are judging other people. So, I like to have low status differential… low power distance between me and my students… and I can get to that point by engaging in these types of behaviors. I don’t know if that’s a secret, necessarily.

Rebecca: …maybe a secret if you don’t know about it.

Jen: …it could be…

Rebecca: …not a secret anymore,

Jen: …it could be… but I think a misconception… and if you think of it in terms of power differential or having low power distance between you and your students… and some instructors might be uncomfortable with that setup…

Rebecca: Is there a difference in gender, related to this low power difference perception?

Jen: I don’t know if there’s a difference in perception but female instructors and feminine communicators… so those are two different things… are more likely to engage in immediate behaviors than more masculine communicators.

John: You talked a little bit about how instructors can create more of a sense of immediacy by walking around the classroom, by maintaining more eye contact, and by using humor. What else can faculty do to help create the sense of immediacy?

Jen: So, remember that it’s psychological closeness or also physical closeness… if you ever had a student approach you after class and they want to talk to you, and the desk is between you and that student… or the teacher station… or something like that. Something you can do in order to create that perception of closeness is to come out from behind objects. You don’t want to stand in front of the classroom. You don’t want to stand behind the little desk. If you’re in Lanigan 101 and you’ve got that teacher station, but you also have a couple of tables in the front… the student approaches, you don’t stand behind the table. You can move out from behind the table… trying to make eye contact with people in the room… smiling goes a long way in terms of just coming across as approachable and friendly… and the idea is, if people find you to be approachable and friendly, they’re going to engage in something like out-of-class communication. You’re not going to go to your instructor’s office hours if you feel like they’re an evil troll, but you will go to their office hours if it feels like “You know what? I got this thing that’s going on in my life. I need some extra time on an assignment. I feel like if I were to go see Rebecca, she seems like the type of person who would understand or who would at least listen to me” and you can do all of that just by modifying your behavior in the classroom.

Rebecca: What happens when that openness gets to a point where those conversations move beyond class-related conversations like you just mentioned?

Jen: Yeah.

Rebecca: So, that particular example is “There’s something in my life but it’s related to the class.” What happens when it goes past that?

Jen: Sure. That is definitely a risk. If you are engaging in this behavior and you are giving the impression that you are approachable and friendly and someone that listens, as I mentioned earlier, that invites communication. So, you will have students show up at your door for reasons completely unrelated to the class… and maybe it is to seek help or advice about the relationships because they’re in your interpersonal communication class… or it just might be they think you’re a friendly person to talk to. That has happened to me and I’ve sat through very awkward conversations or heard things from students that I felt like I had no business hearing. But, you know what? Maybe if you can be a force of good… or if they are disclosing something to you… if it’s something like a sexual assault or something like that, then obviously it’s much better… you don’t ever want to hear that type of message… but it’s better for them to feel as if that’s someone they can talk to you and they can confide in and then you could help them get connected to resources, or something like that. But, then there are also, on a much less serious note, students who are just looking for a friend and they’re hangers on… and they don’t understand leave-taking cues. So, you might be packing up your things to teach your next class and trying to give the signal that it’s time to go, and they might not realize that. Sometimes you have to have very direct conversations at that point: “I have to go. I’m sorry, I can’t talk to you about this any longer.”

Rebecca: You had mentioned a physical closeness, but you also said that there was verbal immediacy as well?

Jen: Right… psychological closeness… the verbal messages would be: using students’ names, using humor, telling personal stories, engaging in self disclosure. Those would be all examples of verbal immediacy… and then the nonverbal immediacy would be: moving around the room, using vocal variety, decreasing space between you and the students, using eye contact. That would all be examples of nonverbal immediacy… and ultimately this leads to affective learning… and my goal as an instructor is always to create more communication nerds. So, I did not start as a communication major, but once I fell into it, I absolutely fell in love with it and thought I cannot live my life without this… and everything I was learning in the classroom I could immediately apply outside of the classroom. Every day in the classroom that is my goal with my students: to get them to know something… be able to do something… to better their lives… better their relationships… find an internship… whatever it might be… and I just love helping to produce comm nerds… people who are quoting comm theories to me… who are analyzing their conversations or the relationships and then telling me about it… or having them explain how they taught their father about cognitive dissonance theory and then how they used it in a work situation or something like that. That’s something that I love… and ultimately affective learning, I feel, is really one of the best outcomes of immediacy and something that’s important to instructional communication: getting students to learn because they like what they’re doing… they see the value in it… they develop a positive attitude to what’s going on in the classroom and the content that you’re teaching them… and also a lot of these behaviors… instructor behaviors… Frankly, if you like your instructor, there’s a good chance you’re going to work harder for that instructor and that you’re going to do well in the class. You might get to a point where you don’t want to disappoint your instructor… but I’d actually like to ask you a question: if you could talk about some of your favorite professors and the types of behaviors that they engaged in that you really liked?

Rebecca: That’s a good question. I need a minute to think. It’s funny, but the first thing I can come up with are all the behaviors I don’t like… [LAUGHTER]

John: Yeah… a strong emotional reaction, either way…

Jen: Sure. Absolutely.

John: I think, thinking back to my college career, which was a while ago… sometime last century… many of the professors that had the most impact on me did exhibit these behaviors. They interacted with you outside of class a bit and they demonstrated some sort of passion for the subject.

Jen: …and I think students want you to care about them… for sure. I start all my classes by asking them how they’re doing? What’s going on? So, many are in clubs and organizations, so I say “What are you promoting right now? What is your organization doing? What’s important to you?” and then finally “Does anyone have any good news?” I just like to hear good news and students appreciate that… and they sometimes, maybe once a month, remember to ask me how I’m doing, which is a win I think… to get that at least once a month? [LAUGHTER]

Rebecca: If you model it and eventually eventually it’s reflected back, right? [LAUGHTER]

Jen: Yeah, Eventually. I guess that’s the theory behind it.

Rebecca: The faculty that I remember the most, or that I had really good experiences with, are the ones that I had, probably, interactions with outside of class. Those are the faculty that I felt like I could go talk to. Who maybe pushed me harder because they got to know me a little bit, to know how to push me in a way that was positive rather than pushing in a way that would have a negative impact on me. They always got more out of me. So, I think everything you’re saying was completely true for me.

Jen: Yeah. That out-of-class communication piece is really important, and before we were studying it in communication and calling it out-of-class communication, people in education were calling them out-of-class experiences. There’s a whole program of research in education devoted to this… and they studied more the outcome of those events. In comm, we study what leads to out-of-class communication more than anything else. In education, they were saying “But here’s the good news… here’s all the good stuff that happens if students are communicating with you outside of the classroom.” So, whether it’s during office hours or whether they run into you at Price Chopper, the first time you see an instructor outside of the classroom can be a bit daunting or jolting. Students think that we just get put away in a closet overnight and brought back out the next day to teach. The first time they see you it might be a little bit weird, but ultimately if they see you, they see you as human and you stop and you say “Hi Rebecca. Hi John. What’s going on? I know you’ve been playing your bass lately. What are you working on? What are you excited about?” In those little things, like you mentioned, Rebecca, they add up and they definitely make students feel better about themselves. It really helps with their development of sense of self and can also help with motivation in the classroom.

John: How would this work in a larger class setting? Can these behaviors scale very nicely? Certainly walking around can, but what else can you do if you have a class of 400 students or so?

Jen: Sure. All of this can certainly be scaled up. Now I don’t recommend if these types of behaviors or being immediate does not come natural to you, that you launch right into trying to do all these things, because students will sniff out that…

Rebecca: inauthentic…

Jen: Yeah …lack of authenticity. They will definitely sense that. The same with verbal immediacy; using humor is an example of verbal immediacy, but if you’re not funny do not try to be funny. It will not go well. But, certainly you can scale this to larger classes. Whether you’re teaching Micro at 400 or I used to teach Comm 100 to over 200 students and I want to say (I’m sure it’s not true…)… I want to be able to say that my teaching style was not that different, whether I was in front of 20 students for a capstone or 200 students for a large introductory course, because ultimately I’m still teaching the way that I think students should be taught. I’m still engaging in these behaviors. I’m still aware of other instructional variables like clarity… like credibility. All of those things are still important. It doesn’t matter necessarily the size of your audience. We typically say “the bigger the audience, the more formal your communication needs to be.” But, I think there are exceptions to that as long as you are still being authentic in some sort of way. Any of our instructional variables that you might learn about can certainly be applied in a large lecture room. There’s no set of categories that “here’s what you do in a large lecture versus here’s what you do in a smaller studio level class.”

John: I know when I teach the large class I generally get in somewhere between 30 and 50 flights of steps every class and usually two or three miles of walking, because it’s a big ways around.

Jen: Oh my gosh. Yeah, Lanigan 101 is a big room. It’s a hard room to work too, because there’s a whole sea of people in the middle that you can’t get to. That’s where eye contact really makes a difference. You just try to make eye contact with them because you can’t physically get that close to them, but you still want them to feel as if you are speaking directly to them, and you’re not trying to be everything to 200 people in the room.

Rebecca: Other than immediacy, are there other theories or principles that we should be aware of as instructors?

Jen: There are a lot of instructional variables, and I think I’ll share some resources that maybe your listeners would be interested in taking a look at later on. Something else that is important to me is credibility. Credibility is essentially believability, and if you are a professor you should be in the business of being believable. It’s important to remember that communication is about messages, but at the end of the day meaning is in the mind of the receiver, and so you can do your absolute best to craft what you think is the perfect message. However, whoever is getting or receiving that message in decoding that message… it’s going through their personal filter. It might be a very benign message, but maybe they’re having a bad day… maybe they’re really hungry, so they’re not quite paying attention. You don’t have complete control over how people decode your messages. You have to remember that meaning is in the mind of the receiver. What you might find credible is going to be different than what John feels as credible. Credibility is a perception. Whether or not I am truly credible doesn’t matter. As long as you think I’m credible, I win. I might be a complete moron, but if you think I’m credible then it doesn’t matter because then everything I say is going through that credibility filter.

We usually talk about credibility as the three C’s: competence, character, and caring. …and for some people different elements are more important. Some people (who perhaps are more logically based) competence or that perception of expertise or knowledge rules the day, always. For some people, they just want to feel like you have some level of goodwill, and you have their best intentions in mind, and that’s the caring aspect of it… and for some people it’s character or it’s honesty and trust that you are being honest and your being truthful with them, and nothing else matters other than that character piece or that trust piece. For different people, different things are important, or they’re gonna pay attention to different aspects of the message based on what they value more… whether it’s the competence the, character or the caring. So, credibility is an instructional variable and it’s not just instructional it goes across different communication contexts. But, that’s something that I think would be interesting for people to know about and to learn about power… how you influence what’s going on in the classroom… also something that can be studied across communication contexts. But how ultimately are you influencing your students? Are you getting them to do what you want them to do because you are rewarding them? …’cause you’re punishing them? or are they doing it because they feel like it’s the right thing to do and they are internalizing your message and they believe in the value of the work? …and there’s some other types of power as well… and then just plain clarity. Clarity is another instructional variable that’s important, in terms of how you structure your messages for your students in the classroom.

John: The next thing we should probably talk about is: what might go wrong or what should faculty avoid doing that might create a negative environment?

Jen: There’s a program of research in the 90s that investigated teacher misbehaviors. So, I thought it’d be fun to ask you what some of those categories are. I bet you can come up with a lot of teacher misbehaviors. So, what are things that instructors do that students don’t like? Just rattle them off at the top of your head.

Rebecca: I’m thinking. I’m a thinker.

Jen: Don’t overthink it.

Rebecca: I know, but I have to still think. They don’t like it when when you’re condescending or like a know-it-all.

Jen: Sure.

John: …especially if you’re not only condescending but wrong. So, that competence is kind of important as a factor there.

Jen: Yeah. I do want to add a fun fact… yet, also our cross to bear as people who study communication. I love producing communication nerds. I love people who are analyzing their conversations. They are putting into practice positive conflict management strategies. However, you can often get accused of applying your communication knowledge in a less than savory way. So, some people get really upset because they feel like you’re Jedi mind tricking them with your communication skills. …something to keep in mind… that as comm majors, we often get yelled at for actually using what we’re learning in the classroom… because people don’t wanna fight fair. They want to get below the belt and say mean things when you’re like “Let’s be constructive. We don’t want to be verbally aggressive. Let’s try to just be argumentative… we’ll stick to the arguments.” That doesn’t go over very well when you’re having a fight with your girlfriend. [LAUGHTER]

Rebecca: I think being late…

Jen: Yup, that’s a big one.

John: …or not being prepared at the start of class is another thing.

Rebecca: I hate when the technology doesn’t work or there’s serious user error.

Jen: For sure. Anything else on your mind?

Rebecca: They don’t like it when you don’t know their name or… that extends to… it’s not just name but gender pronoun… pronunciation. There’s a whole slew of things that probably snowball onto that.

Jen: Absolutely. You got some good ones. I thought I would touch on a couple others that maybe you hadn’t been thinking about. You did mention being condescending… but sometimes being sarcastic and using put-downs is a problem for students, naturally. Unreasonable or arbitrary rules… If you think about your syllabus and what’s in there. Your syllabus sends a message on day one. You want to think about ultimately what you’re sharing with students based on your syllabus. Inaccessibility… Students want to be able to see you out of the classroom. They want to visit you during office hours. Being late… definitely. But one I think that’s interesting, that we probably don’t often think about, is information underload. Students want to be challenged. Most students want to be challenged, and this ties into something that we’ve been talking about previously. There’s this misconception that if you have a classroom that seems to be open and friendly and you are approachable as an instructor, that that means you are the easy instructor… and I have a major problem with that. I think it’s absolutely possible for you to do all of those things to be liked as an instructor, but to also have high standards… and frankly, if you set a bar for your students and they exceed it then you should continue to raise that bar. …and ultimately having or doing tasks that the students don’t feel like are getting them to the end goal of the course is actually considered by them a misbehavior. That’s something that you would want to avoid.

Rebecca: It was a good one that it’s most definitely overlooked… and you definitely hear those conversations: “Oh, take this class because so-and-so is easy. All we do is talk.”

Jen: Yeah, there’s certainly that misconception too… in comm studies, in particular, like “What do you do in that major? …and I come from what we call “communication and social interaction” or “communication,” “communication studies.” We’ve had different names over the years. We thought CSI would be super cool and hip and turns out people are like “I don’t get it. I don’t know that is.” [LAUGHTER] We’re changing it back to “communication,” but if I tell someone “Oh, I’m a journalism professor or public relations professor or a broadcasting professor” like everyone has an idea of what that means… and if I’m the communication professor they’re like “So, you just talk all the time?” I’m like like “No, there’s actually more to it than that.”

John: Well, you do talk all the time, but it’s about something. [LAUGHTER]

Jen: We’re communicating about communication. So, it’s all very meta. Yes. [LAUGHTER] It’s a good time.

Rebecca: It’s very deep.

Jen: Yeah, it is. Of course it is, all the time.

John: Where can faculty go to find more information about instructional communication?

Jen: Penfield [Library at SUNY-Oswego] does own the handbook of instructional communication. We asked them (we being the Comm Studies department) a few years ago to purchase that so people can check that out of the library. The National Communication Association has some great links in terms of instructional communication and what to do in the classroom and how to enact certain behaviors. That is a great resource. There’s another book that I like a lot called Communication for Teachers which summarizes a lot of instructional communication literature and also talks about how to apply that to a classroom… whether it’s K through 12 or in a college classroom.

John: We’ll share links to some of these materials in the show notes.

Rebecca: So, we mentioned earlier on about talking about communication that happens outside of the classroom and we’ve hinted at a couple things here and there, but could you talk a little bit more about those out-of-class experiences and that impact on learning?

Jen: It impacts student motivation, positively. So, they have those moments…and it can just be passing in the hallway or walking through the breezeway in Marano and it’s just a simple “Hello” to a student. That’s something that they can take with them, put it in a little pocket and store that. “Oh, Professor Kane remembers my name” or whatever it might be that makes a difference. But, ultimately it gives a student an opportunity to connect with you on a different level… in a different sort of time-space continuum, if you will. Everything is crazy before class… after class… lots of people want a piece of you… If they take the time to come visit you during office hours and that’s more that’s one-on-one time that they get to spend with you to develop those relationships and certainly that can help them. Students who engage in more out of class communication tend to do better in their classes than students who do not engage in out of class communication. But, it also has… besides classroom outcomes… has better outcomes for them personally. Networking, which you were alluding to earlier… as you met with your professors, you got to know them… they got to know you… now, when they get a call that someone needs an intern or needs someone who can do graphic design work, well you and I were just talking an hour ago in my office and I know that you have this skill set, so now I’m gonna pass this opportunity on to you… because I know that you’re interested and I know that you can do the work. So, that’s a tremendous outcome for students if they take the time to get to know their professors and their professors know them, when those opportunities come past, they can give those to the students that they’ve met and they’ve spent time with… and it just gives students another way to practice their interpersonal communication skills.

John: We always end with the question: What are you going to do next?

Jen: Something that is important to me, as someone who studies communication, 1. is to always correct people who say “Communications” instead of “Communication.” No “s” just “Communication” but also to show people the value of what we study, in what we know as communication scholars. One of the committees I sit on is the Title IX committee, and I’m also a Title IX investigator. One day, Lisa Evaneski was describing some of the cases that she was seeing as Title IX investigator and she said “These aren’t necessarily Title IX cases. We’re not talking about instances of interpersonal violence or sexual assault or anything like that. They’re just, I don’t know, messy breakups…” and I’m like “Ah, we can help with that.” So, in communication, and those of us that study interpersonal communication, we talked a lot about how to treat people positively… how to breakup constructively… how to just be a good human during those difficult times… and so there’s been a group of us that are working in comm studies to create a workshop that Lisa can potentially direct people to that maybe need a little bit of coaching about how to treat people or how to be in a relationship or how to break up… but also we would open it to the campus in general. So, anyone who’s going through a nasty breakup or thinking about “maybe it’s time for me to dump this person and move on. How can I do that in a healthy positive productive way?” …how to use social media or not use social media during during those those times… So, we’re working on building a workshop on messy breakups… which will maybe eventually have a different title, but so far we’re just stuck on messy breakups.

Rebecca: I think it works.

Jen: Yeah, and our goal would also then be to turn that into some type of research as well. Something that we could could share with our discipline, in terms of how we are applying and using our knowledge as communication scholars to help solve a problem on campus… something of that nature… A dream that I’ve always had, and that I know John knows about, is to develop some sort of instructor boot camp. It would go nicely with your badging program if we could have something where people would learn ultimately how to teach… or how to best employ some of these instructional communication variables, in order to get the best out of their students. We can also talk about how to build a syllabus… how to write a syllabus… how to structure assignments… how to ensure that your messages are clear to your students… those types of things. So, one real thing that I’m working on and one thing that I would like to at some point…
JOHN… an aspirational goal…

Jen: Yeah… actually launch…

John: oI think we’d like to see something along those lines to here.

Jen: …and I do think it’s important to say I’m not the only person that knows about this stuff and that studies it so I’ve got colleagues in Comm Studies Katherine Thweatt and Mary Toale, all three of us graduated from the same doctoral program in instructional communication, so there are a handful of us that are interested in this and that are dedicated to it, along with some other great interpersonal scholars in Comm Studies.

Rebecca: I think that what’s really exciting about your workshop idea… that hopefully is not just an idea real soon… is that students will see a discipline in action… and the more ways that we can do those sorts of things on campus, the more real it is for students about how these things that seem like they’re not applicable or they’re not applied somehow…

Jen: Right.

Rebecca: …in action. Some fields are maybe more obvious than others and so the more we can be visible as scholars in the community and sharing that knowledge with the community, I think, is always really nice.

Jen: Yeah, instructional communication is a great example of an applied field.

John: Very good. Well, thank you.

Jen: My pleasure. Thank you both very much.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

38. Reflective practice

Posted in higher education, professional development, scholarship of teaching and learning, teaching, teaching centers
July 18, 2018

Now that we have been on summer vacation for a while, we thought it would be useful to take a break from our usual interview format to reflect on the previous semester and our plans for the fall. We also provide some recommendations on summer reading related to professional development.

Show Notes

  • Sue, D. W. (2016). Race talk and the conspiracy of silence: Understanding and facilitating difficult dialogues on race. John Wiley & Sons.
  • Guffey, E. (2017). Designing Disability: Symbols, Space, and Society. Bloomsbury Publishing.
  • Evans, N. J., Broido, E. M., Brown, K. R., & Wilke, A. K. (2017). Disability in higher education: A social justice approach. John Wiley & Sons.
  • Hogan, Lara (2016). Demystifying Public Speaking. A Book Apart (https://abookapart.com/products/demystifying-public-speaking)
  • Hoffman, Kevin H. (2018). Meeting Design: For Managers, Makers and Everyone.  Rosenfeld Media.
  • Openpedagogy.org
  • Schwartz, D. L., Tsang, J. M., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. WW Norton & Company.
  • Cavanagh, S. R. (2016). The Spark of Learning: Energizing the College Classroom with the Science of Emotion. West Virginia University Press.
  • McGuire, S. Y. (2015). Teach students how to learn: strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Stylus Publishing, LLC.
  • Parkes, J., & Zimmaro, D. (2016). Learning and assessing with multiple-choice questions in college classrooms. Routledge.
  • Lewis, M. (2016). The undoing project: A friendship that changed our minds. WW Norton & Company.The Undoing Project – Michael Lewis
  • Tea for Teaching podcast: 15. Civic Engagement – a discussion with Allison Rank about the Vote Oswego project.
  • DeRosa, Robin (2017). “OER Bigger than Affordability” Inside Higher Ed. November 1.
  • Tea for Teaching podcast: 30. Adaptive Learning
  • Videoscribe
  • Flipgrid
  • Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
  • Learning How to Learn MOOC
  • Oakley, B. A. (2014). A mind for numbers: How to excel at math and science (even if you flunked algebra). TarcherPerigree.
  • Oakley, B. (2017). Mindshift: Break through obstacles to learning and discover your hidden potential. Penguin.
  • Oakley, B. (2018). Learning How to Learn: How to Succeed in School without Spending all your Time Studying; a Guide for Kids and Teens. Penguin.
  • Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.
  • Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc..
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Teaching in Higher Ed – Bonni Stachoviak
  • Teach Better – Doug McKee and Edward O’Neill
  • Email addresses: john.kane@oswego.edu and rebecca.mushtare@oswego.edu

John: Now that we have been on summer vacation for a while, we thought it would be useful to take a break from our usual interview format to reflect on the previous semester and our plans for the fall.

[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Today our teas are:

Rebecca: …a mix of seven different kinds of tea, and it’s not really describable at this point.

John: After I’ve had many different types of tea today, I have Twinings’ Wild Berries herbal tea.

Rebecca: Finally dropping the caffeine after a long day?

John: …after many teas earlier in the day, yes.

Rebecca: So, I start my reflective practice while grading during finals week and for me it’s a really effective and productive procrastination technique. As I’m reading assignments or looking at projects and making notes about things that clearly did not work or “Wow, I really should cover these skills better” or “This really worked…” and I have a running dialogue with myself while I’m grading them and I use that for planning for the fall. What are your practices like, John?

John: I’d like to do that a bit during grading week but during grading week I’m generally busy working on the workshop schedule for our workshops here…

Rebecca: What?

John: … and also working on plans for various presentations at the SUNY Conference on Instructional Technology and so forth… and then getting ready for my trip down to North Carolina for the summer. So, I try to do it as I’m going during the semester so that I keep in my blackboard folder for each course a hidden folder where I list any problems… and I’ll do that for the course overall, as well as within individual modules. That way, when I go to refresh the course in the future I’ll have a list of things in general I want to do differently as well as specific recommendations in specific components of the course.

Rebecca: Have you ever accidentally made one of those hidden files not hidden?

John: I have not, no. [LAUGHTER]. I’m much more likely to leave something hidden that the students have as an assignment, but they’re usually pretty good at reminding me of that as we go through.

Rebecca: I think my greatest fear of having notes like that would be that I would make them really public and then probably have some sort of snarky comment in my hidden files. [LAUGHTER]

John: So, we thought maybe we talked a little bit about our lists of plans and then make some general recommendations of things that we found useful. So, Rebecca would you like to go first?

John: Sure, I think both of us have a fairly aggressive reading reading dream list. I don’t know how much either of us will get through that list, but my list includes Race Talk and [the] Conspiracy of Silence by Derald Wing Sue… which jDerald Wing Sue’s coming to our campus in the fall to give a talk based on this book… and we’re gonna have a reading group again. So, I want to make sure I’m on top of that.

John: That’s also on my list. I started reading it earlier, but I got buried in the semester, so it’s on the top of my summer reading list.

Rebecca: Yeah, I read the first chapter but then that’s as far as I got. I’m also planning to read… I started reading but I didn’t have time to finish a book called Designing Disability: Symbols, Space, and Society by Elizabeth Guthrie. It’s a really interesting book about the history of the wheelchair symbol. So, it’s related to design, obviously, which is my area of teaching… but also my interest in accessibility, which I’ve been working on a lot on campus. Related to that, I also am planning to read Disability in Higher Education: a Social Justice Approach by Nancy Evans. I started reading that during this semester and read a few chapters here and there but didn’t get all the way through. It’s a pretty hefty read. So, I’m hoping to get through a lot of that this summer… and then I have two other books that are not so much teaching related but come out of the design field. One of them is Demystifying Public Speaking, by Laura Hogen, which is from a series called A Book Apart… it’s made for designers, so I’m hoping to read that book and pull out some nuggets that might be helpful for students who get a little nervous about public speaking… or see whether or not it’s a good recommendation for our advanced students in our program… and then the other one that comes from a designer is Meeting Design for Managers, Makers, and Everyone by Kevin H. Hoffman. I’ve seen Kevin speak and have had some conversations with him in the past about designing meetings, so that meetings are actually productive and useful rather than unproductive and something that could maybe have happened in an email. So, I’m looking forward to reading a fuller version of his process. What are you hoping to read, John?

John: Well, several these I’ve already started again but haven’t gotten too far but they’re enough so that they’re on my Kindle or I have the books very handy… and I plan to read them as soon as I can. One is The ABCs of How We Learn by Daniel Schwartz. I actually made it, I believe, through letter L before I had to put it down to get caught up on some other things.

Rebecca: Yeah, I remember getting some updates in the various letters and it did kind of fizzle out.

John: So, I will finish that fairly soon, I believe. The Spark of Learning is a book I’ve heard wonderful things about from Sarah Rose Cavanagh. I’m hoping to read that this summer. It’s also on my Kindle app. The Teach Students How to Learn book by Saundra Yancy McGuire and Thomas Angelo is a really good book that talks about ways of improving student metacognition. Again, I’ve read a little bit of that just to see that it is something I really want to continue with. Another thing I’d like to look at, since I teach large classes where I use a lot of multiple-choice questions, is a book that I heard about on a couple of other podcasts on teaching and learning… in particular, the Teaching in Higher Ed podcast, which is Learning and Assessing with Multiple-Choice Questions in College Classrooms by Jay Parkes and Dawn Zimmaro. That’s something I haven’t started yet, but I do have a copy of that and I’m looking forward to reading it. Another book that somewhat on the border between teaching and learning and my work in economics is The Undoing Project by Michael Lewis. It’s a book on the early development of behavioral economics by Kahneman and Tversky, and the reason why it’s on the border of economics and teaching is that behavioral economics explains why people don’t always behave as rational agents… and certainly that’s important in trying to understand how people work from an economics perspective… but when we’re dealing with students and faculty we observe that people don’t always behave, perhaps,in an optimal fashion. We don’t see people engaging in activities that are in their long-run self-interest, and they often will prefer short-run benefits over long term benefits, even though they know they’d be better off doing their work a bit earlier and so forth. So, it overlaps between those two interests. I’m looking forward to that I guess that’s it for my books.
So, what are your plans for redeveloping or redesigning some of your courses?

Rebecca: Well, I have a new class that I’ll be offering in the fall that’s related to some other special topics I’ve taught before on experience design… and in that class we’re gonna do two community projects: one is called “recollections storytelling through mementos“ which is the design of an interactive exhibit that will travel to multiple adult care facilities in central New York. It’s the second exhibition in a series. The last one we did was a couple of years ago… and so the design and development of that will happen partially through the summer and then in my class in the fall… and then the exhibit will go up and travel next year in 2019… and then the other project that we’re gonna work on is our very famous [LAUGHTER] regular guest Allison Rank, who’s talked about her project Vote Oswego. My students will be working on that project as well, doing some design work with her class. We scheduled our two classes so that they would be at the same time slot, so that they could collaborate a little bit easier this time. so I’m looking forward to working with Allison a little bit this summer to make some specific plans for that for the Fall. So, I’m doing that and then revisiting my web design courses like I do every year: a) the content generally changes because standards and things and web change but I’m also… I had my little list, as I was grading, of things that I want to make sure that I’m doing and some of that means integrating more reflective practice opportunities I think it’s really important and I always plan on doing that and then somehow it gets cut. So, I decided I really need to just actively decide to cut something else out, so that there is actually that room and that’s not what gets cut in the future.

I’m also working on some new accessibility modules and I’m also really thinking of… I’ve been doing a lot of quizzes based on our reading groups and things that we’ve been talking about for retrieval practice… but I’m really thinking about switching to trying some in-class polls even though my class is relatively small and mixing in some practical exercises and I was doing both of those kinds of things in the quizzes and I think spreading those out a little bit will actually help with engagement, and also make it so it doesn’t take up as much class time.

John: In terms of the use of polling in small classes… for the last five or six years now I’ve been using polling in classes that I teach at Duke where generally there are between sixteen and twenty students, and it works just as well in small classes as it does in large ones. In some ways it works a little bit better.

Rebecca: Yeah, I can imagine that and I know that you’ve talked about that in the past, so you’re wearing on me. [LAUGHTER]

John: It’s a good practice.

Rebecca: Yeah, how about you?

John: Well, I’ve got a number of things planned. One is, I’ve been wanting to adopt an OER for a long time, but I’ve been somewhat tied to the adaptive learning tools and so forth provided by publishers, as well as the array of materials they provide… but, I want to explore some OER options for my large introductory class.

Rebecca: For those that aren’t familiar, what’s an OER.

John: Open educational resources… basically things that are released under Creative Commons licenses… and there’s two major advantages of that: one is that it would be free for students… students would also have access from the first day of class, and we’ll be talking about that more in future episodes… and another thing I’d like to do more is explore some alternatives to publisher provided adaptive learning tools so that it might be possible to find some ways of integrating OER with it, or to investigate ways in which OER materials can be used with adaptive learning systems that can work in classes where you want to have enough variety in the question so students can’t just look them up on the internet…

Rebecca: …and if you’re a little more interested in OER and the kind of big impact that that can have on students, you may want to check out Robin Derosa’s article in Higher Ed “OER Bigger than Affordability.” …and then we also have a previous episode that’s about adaptive learning that people might want to check out if they’re curious about that.

John: I believe was episode 30. Another thing I’d like to do, along the same lines, is I had written an econometrics text that I’ve been using in class for a while. I’d like to rewrite that as an OER text, and one of the things I need to do is update some of the old videos I’ve created. Last winter, when I was at the OLC conference in Orlando (at Disney World) I saw a presentation on Videoscribe and I had seen some videos created by that and it just looked really really cool and so I purchased a subscription to that and now I actually have to actually learn how to use it… and it does involve a bit of work… and there’s a bit of start-up costs in that, but it’s a very powerful tool and it looks like a really good way of presenting technical material.

I’d also like to explore a little bit of Flipgrid just because i’ve used voicethread now and I keep hearing really good things about Flipgrid, so I’d like to look at that and compare the benefits of the two systems.

Rebecca: What’s a Flipgrid?

John: Flipgrid is very much like Voicethread except the videos are provided in a grid. In many ways, it’s very similar to Voicethread except your class shows up as an array on the screen. You can click on any of the boxes for the students and hear or see their responses.

Rebecca: So, it sounds like the interfaces may be the big benefit there.

John: I believe so. I need to explore it more. It’s something I’ve been hearing a lot about from a lot of people who do some really good work, so I’d like to see how it compares.

Rebecca: You know all your talk of OERs and open education resources reminded me that one of the key things I have on my to-do list is to explore all the available resources that are available on openpedagogy.org. After hearing Robin DeRosa talk about it at CIT, the conference that John and I were at in late May, I got really excited about some of her teaching techniques and I just really want to see what else is out there and what’s available. So, who knows, it might really overhaul something.

John: I was at the same talk and we were both so impressed by it we went down and we talked to Robin at the end and we’ve invited her to come back to Oswego in the fall to give a presentation here, and there’s a good chance that she will appear as a guest on a future episode of the podcast. So, there’s also some things we’d like to recommend to others: books and tools that we found really useful. So, would you like to start?

Rebecca: Alright, so most of our recommendations are publications that have highly influenced our show. So one of those is Minds Online by Michelle Miller, a great cognitive psychologist. The book is about being online, but all the things she talks about works in in-person classes too, so I highly recommend that book.

John: Michelle Miller, after I had read her book, so impressed me that I invited her to come up to campus to give a workshop here… and people were so impressed by that that we created our reading group series here. Our first one was Michelle Miller’s Minds Online and participants were so enthused about that they insisted that we bring her back again at the end of the reading group and she was a wonderful speaker as well as a very good author.

Rebecca: Yeah, and a great facilitator too. We also want to recommend Barbara Oakley’s Learning How to Learn MOOC. It’s a great way to learn the basic cognitive science behind the evidence-based practices. So, if you’re not familiar, that’s a great way to follow along and get involved and her videos are fantastic.

John: It’s also the most popular MOOC in the world…

Rebecca: …and it’s the biggest one too, right?

John: and it’s the biggest one and she’s got hundreds of thousands of students taking it. It’s a four-week experience and I encourage all my students to take it.

Rebecca: …and if you’ve never done a MOOC, what a great experience to take one of the best MOOCs in the world.

John: It also provides very good examples of effective practice for online teaching that are very scalable. So, there’s a lot of good reasons to do it.

Rebecca: She also has some other great books including: A Mind for Numbers, Mind Shift, and Learning How to Learn: How to Succeed in School without Spending all your Time Studying; a Guide for Kids and Teens. That last one is a new one that’s directed specifically at middle school and high school students.

John: Another book, I think, that we’d both strongly recommend is Make it Stick. We used that as our second reading group here at Oswego a couple years ago, and Peter Brown came up and presented on that. but it’s by Brown, Roediger, and McDaniel. Peter Brown is a novelist and Roediger and McDaniel have done a tremendous amount of work in studying how people learn.

Rebecca: We can’t go without mentioning Carol Dweck’s Mindset book as well. We often see who we might traditionally think of as being quote unquote good students, “A” students maybe who hit something in college where they realize that they have to struggle a little bit and they don’t know what to do, because everything’s always come easily to them… but they struggle because they don’t have a growth mindset. So, this is a great way to learn more about the differences between fixed and growth mindsets and maybe put some strategies in place to help all of our students move more towards a growth mindset in the courses we teach.

John: The next thing we recommend is Jim Lang’s Small Teaching. it covers much of the same material as Minds Online and Make it Stick but it does it in a somewhat different way. It focuses on small techniques that you can change in your classroom that pay off very substantially. So, for people who don’t want to substantially revise their courses, it’s a very effective way of making small modifications… activities that take five to ten minutes in a class… that have a very large impact without requiring a dramatic overhaul or restructuring of your course.

Rebecca: Yeah, and the faculty here have responded very well to this book and have made a lot of small changes to their classes in the last year and had big success.

John: Another thing we’d like to mention are some podcasts that we listen to that have some really good coverage of topics related to higher education. The first one is Teaching in Higher Ed by Bonnie Stachoviak. The other one we want to recommend is Teach Better by Doug McKee and Edward O’Neill and you might remember Doug McKee from a previous episode.

Rebecca: So, we usually conclude by asking what’s next, but if you really want to know you could just listen to this episode again. We made a lot of references during this episode to a lot of great material and I can’t imagine that you wrote it all down, especially if you’re driving in your car, right? So, remember to check the show notes will have specific links and details so that you can find all these resources so that you can also enjoy some of these during your summer.

John: If any of you have any recommendations for topics for the show, please write to either of us. Our email addresses will be in the show notes.

Rebecca: We also wanted to take a couple minutes and just reflect on the podcast itself. We really appreciate the community of listeners that we’ve gained. We never expected this to even go on this long. It was a little experiment that we had that we wanted to try out in the fall and now we’re on Episode… oh, I don’t know what episode we’ll be on.

John: We’ve been really impressed by how many listeners we’ve reached across the U.S. and throughout the world. We were expecting we’d mostly get people listening from our institution and perhaps some of our colleagues in other places. So, we very much appreciate all the support you shown.

Rebecca: …and please let us know if there’s other things that we can cover that you’re really interested in or really need some professional development in.

John: We hope you’re enjoying your summer vacation. Enjoy the rest of your summer!

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

37. Evidence is Trending

Posted in active learning, assessment, attention, augmented reality, evidence-based practices, higher education, learning, mental models, metacognition, online learning, pedagogy, scholarship of teaching and learning, teaching
July 11, 2018

Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode, Michelle Miller joins us to discuss recent research that will likely shape the future of higher education.

Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.

Show Notes

Rebecca: Faculty are increasingly looking to research on teaching and learning to make informed decisions about their practice as a teacher and the policies their institutions put into place. In today’s episode we talk to a cognitive psychologist about recent research that will likely shape the future of higher education.
[Music]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[Music]

John: Our guest today is Michelle Miller. Michelle is Director of the First-Year Learning Initiative, Professor of Psychological Sciences, and President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology. Her research interests include memory, attention, and student success in the early college career. She co-created the First-Year Learning Initiative at Northern Arizona University and is active in course redesign, serving as a redesign scholar for the National Center for Academic Transformation. She is the author of Minds Online: Teaching Effectively with Technology and has written about evidence-based pedagogy in scholarly as well as general interest publications.
Welcome, Michelle!

Michelle: Hi, I’m so glad to be here.

Rebecca: Thanks for joining us.
Today’s teas are:

Michelle: I’m drinking a fresh peppermint infused tea, and it’s my favorite afternoon pick-me-up.

Rebecca: …and it looks like it’s in a really wonderfully designed teapot.

Michelle: Well, thank you… and this is a thrift store find… one of my favorite things to do. Yeah, so I’m enjoying it.

John: I have Twinings Blackcurrant Breeze.

Rebecca: …and I’m drinking chai today.

Michelle: Pretty rough.

John: We invited you here to talk a little bit about things that you’ve been observing in terms of what’s catching on in higher education in terms of new and interesting innovations in teaching.

Michelle: Right, that’s one of things that I really had the luxury of being able to step back and look at over this last semester and over this last spring when I was on sabbatical… One of the really neat things about my book Minds Online, especially now that it’s been out for a few years, is that it does open up all these opportunities to speak with really engaged faculty and others, such as: instructional designers, librarians, academic leadership, educational technology coordinators… all these individuals around the country who are really, really involved in these issues. It’s a great opportunity to see how these trends, how these ideas, how these innovations are rolling out, and these can be some things that have been around for quite some time and just continue to rock along and even pickup steam, and some newer things that are on the horizon.

John: You’ve been doing quite a bit of traveling. You just got back from China recently, I believe.

Michelle: I sure did. It was a short visit and I do hope to go back, both to keep getting involved in educational innovations there and, hopefully, as a tourist as well. So, I was not there for very long but I had the opportunity to speak at Tsinghua University in Beijing, which is a really dynamic institution that’s been around for about a hundred years. For a while in its history it specialized in things like engineering education polytechnic, but now it’s really a selective comprehensive university with very vibrant graduate and undergraduate programs that are really very relatable for those of us in the United States working in similar contexts. My invitation was to be one of the featured speakers at the Future Education, Future Learning Conference, which was a very interdisciplinary gathering of doctoral students, faculty, even others from the community, who were all interested in the intersection of things like technology, online learning, MOOCs even, and educational research (including research into the brain and cognitive psychology), and bringing all of those together… and it was a multilingual conference. I do not speak Chinese but much of the conference was in both English and Chinese and so I was also able to really absorb a lot of these new ideas. So yes, that was a real highlight of my sabbatical semester and one that I’m going to be thinking about for quite some time.

I should say that part of what tied in there as well is that Minds Online, I’ve just learned, is going to be translated into Chinese and that’s going to come out in May 2019. So, I also got to meet with some of the people who were involved in the translation… start to put together some promotional materials such as videos and things like that.

Rebecca: Cool.

John: Excellent.

Rebecca: So, you’ve had a good opportunity, as you’ve been traveling, to almost do a scavenger hunt of what faculty are doing with evidence-based practices related to your book. Can you share some of what you’ve found or heard?

Michelle: This theme of evidence-based practice, and really tying into the findings that have been coming out of cognitive psychology for quite some time, that really is one of the exciting trends and things that I was really excited to see and hear for so many different quarters I visited in different institutions… and so I would say definitely, this is a trend that is continuing and is increasing. There really does continue to be a lot of wonderful interest and wonderful activity around these real cognitively informed approaches to teaching, and what I think we could call scientifically based and evidence-based strategies. One form this has taken is Josh Eyler’s new book, called How Humans Learn: The Science and Stories behind Effective College Teaching. This is a brand new book by a faculty development professional, and a person coming out of the humanities, actually, who’s weaving together even from his humanities background everything from evolutionary biology to classical research in early childhood education to the latest brain-based research. He’s weaving this together into this new book for faculty. So, that’s one of the things that I’ve noticed and then there’s the issue which i think is another great illustration of best-known practice which is the testing effect and retrieval practice.

John: One of the nice things is how so many branches of research are converging… testing in the classroom, brain-based research, and so forth, are all finding those same basic effects. It’s nice to see such robust results, which we don’t always see in all research in all disciplines.

Rebecca: …and just breaking down the silos in general. The things are all related and finding out what those relationships are… exploring those relationships… is really important and it’s nice to see that it’s starting to open up.

John: We should also note that when you visited here, we had a reading group and we had faculty working on trying to apply some of these concepts, and they’re still doing that… and they still keep making references back to your visit. So, it’s had quite a big impact on our campus.

Michelle: This wasn’t true, I don’t think, when I first entered the teaching profession… and even to the extent when I first started getting interested in applied work in course redesign and in faculty professional development. you would get kind of this pushback or just strange looks when you said “Oh, how about we bring in something from cognitive psychology” and now that is just highly normalized and something that people are really speaking across the curriculum… and taking it and running with it in a lasting ongoing way, not just as a “Oh, well that was an interesting idea. I’m going to keep doing what I’m doing” but really people making some deep changes as you mentioned. This theme of breaking down silos… I mean I think if there’s kind of one umbrella trend that all of these things fits under it’s that breakdown of boundaries. So, that’s one that I keep coming back to, I know, in my work.

So, the idea of retrieval practice, drilling down on that one key finding which goes back a very long ways in cognitive psychology. I think of that as such a good example of what we’re talking about here… about how this very detailed effect in cognition and yet it does have these applications across disciplinary silos. Now when I go to conferences and I say “Okay, raise your hand. How many people have ever heard of retrieval practice? How many people have ever heard of the testing effect? How many people have heard of the book Make it Stick (which really places this phenomena at its center)?” and I’m seeing more hands raising.

With retrieval practice, by the way, we’re talking about that principle that taking a test on something, that retrieving something from memory actively, has this huge impact on future memorability of that information. As its proponents like to say, tests are not neutral from a memory or from a learning standpoint… and while some of the research has focused on very kind of stripped-down laboratory style tasks like memorizing words pairs, there are also some other research projects showing that it does flow out to more realistic learning situations.

So, more people simply know about this, and that’s really the first hurdle, oftentimes, with getting this involved disciplinary sometimes jargon riddled research out there to practitioners and getting it into their hands. So, people heard of it and they’re starting to build this into their teaching. As I’ve traveled around I love to hear some of the specific examples and to see it as well crop up in scholarship of teaching and learning.

Just recently, for example, I ran across and really got into the work of Bruce Kirchhoff who is at University of North Carolina – Greensboro and his area is botany and plant identification. He has actually put together some different really technology-based apps and tools that students and teachers can use in something like a botany course to rehearse and review plant identification. He says in one of his articles, for example, that there just isn’t time in class to really adequately master plant identification. It’s just too complex of a perceptual and cognitive and memory test to do that. So, he really built in from the get-go very specific principles drawn from cognitive psychology… so, the testing effect is in there… there’s different varieties of quizzing and it all is about just getting students to retrieve and identify example after example. It brings in also principles such as interleaving, which we could return to in a little bit, but has to do with the sequencing of different examples… their spacing… So, that’s even planned out exactly how and when students encounter different things that they’re studying. It’s really wonderful. So, for example he and his colleagues put out a scholarship of teaching and learning article talking about how this approach was used effectively in veterinary medicine students who have to learn to identify poisonous plants that they’ll see around their practice. This is something that can be time-consuming and very tough, but they have some good data showing that this technology enhanced cognitively based approach really does work. That’s one example. Coincidentally, I’ve seen some other work in the literature, also on plant identification, where the instructors tagged plants in an arboretum… they went around and tagged them with QR codes… that students can walk up to a plant in the real environment with an iPad… hold the iPad over it… and it would immediately start producing quiz questions that were are specific to exactly the plants they were looking at.
So, those are some of the exciting things that people are taking and running with now that this principle is out there.

Rebecca: What I really love about the two stories that you just shared was the faculty are really designing their curriculum and designing the learning experiences with the students in mind… and what students need and when they need it. So, not only is it employing these cognitive science principles, but it’s actually applying design principles as well. It’s really designing for a user experience and thinking about the idea that if I need to identify a plant, being able to identify it in this situation in which I would need to identify it in makes it much more dynamic I think for a student… but also really meets them where they’re at and where they need it.

John: …and there’s so many apps out there now that will do the plant identification just from imagery without the QR code, that I can see it taking it one step further where they can do it in the wild without having that… so they can build it in for plants that are in the region without needing to encode that specifically for the application.

Michelle: I think you’re absolutely right once we put the technology in the hands of faculties who, as I said, they’re the one to know: “Where are my students at? Where are the weak points? Where are the gaps that they really need to bridge?” and that’s where their creativity is giving rise to all these new applications… and sometimes these can be low-tech as well… or also things that we can put in a face-to-face environment… and I’d like to to share just some experiences that I’ve had with this over the last few semesters.

In addition to trying to teach online with a lot of technology, I also have in my teaching rotation a small required course in research methods in psychology which can be a real stumbling block… the big challenge course… it’s kind of a gateway course to continued progress in our major. So, in this research methods course, some of the things that I’ve done around assessment and testing to really try again to stretch that retrieval practice idea… to make assessments really a more dynamic part of the course and more central part of the course… to move away from that idea that tests are just this kind of every now and again this panic mode opportunity for me to kind of measure in sorts of students and judge them… to make good on that idea that tests are part of learning. So, here’s some of the things that I try to do. For one thing, I took time out of the class almost every single class meeting as part of the routine to have students first of all generate quiz questions out of their textbook. So, we do have a certain amount of foundational material in that course as well as a project and a whole lot of other stuff is going on. So they need to get that foundational stuff.

Every Tuesday they would come in and they knew their routine: you get index cards and you crack your textbook and you generate for me three quiz questions. Everybody does it. I’m not policing whether you read the chapter or not. It’s active… they’re generating it… and also that makes it something like frequent quizzing. That’s a great practical advantage for me since I’m not writing everything. They would turn those in and I would select some of my favorites I would turn those into a traditional looking paper quiz and hand that out on Thursday. I said “Hey, take this like a realistic quiz.” I had explained to them that quizzes can really boost their learning, so that was the justification for spending time on it and then I said: “You know what? I’m not going to grade it either. You take it home because this is a learning experience for you. It’s a learning activity.” so we did that every single week as those students got into that routine.

The second thing that I did to really re-envision how assessment testing and quizzing worked in this particular course, was something inspired by different kinds of group testing and exam wrapper activities I’ve seen, particularly coming out of the STEM field, where there’s been a lot of innovation in this area. What I would do is… we had these high stakes exams at a few points during the semester. But, the class day after the exam, we didn’t do the traditional “Let’s go over the exam.” [LAUGHTER] That’s kind of deadly dull, and it just tends to generate a lot of pushback from students… and as we know from the research, simply reviewing… passing your eyes over the information… is not going to do much to advance your learning. So, what I would do is… I would photocopy all those exams, so it has a secure copy. They were not graded. I would not look at this before we did this… and I would pass everybody’s exams back to them along with a blank copy of that same exam. I assigned them to small groups and I said “Okay, here’s your job. Go back over this exam, fill it out as perfectly as you can as a group, and to make it interesting I said I will grade that exam as well, the one you do with your group, and anything you get over 90% gets added to everybody’s grade. This time it was open book, it was open Google, it was everything except you can’t ask me questions. So, you have each other and that’s where these great conversations started to happen. The things that we always want students to say. So, I would eavesdrop and hear students say “Oh, well you know what, I think on this question she was really talking about validity because reliability is this other thing…” and they’d have a deep conversation about it. I’m still kind of going back through the numbers to see what are the impacts of learning? Are there any trends that I can identify? But, I will say this: in the semesters that I did this, I didn’t have a single question ever come back to me along the lines of “Well, this question was unclear. I didn’t understand it. I think I was graded unfairly.” it really did shut all that down and again extended the learning that I feel students got out of that. Now it meant a big sacrifice of class time, but I feel strongly enough about these principles that I’m always going to do this in one form or another anytime I can can in face-to-face classes.

Rebecca: This sounds really familiar, John.

John: I’ve just done the same, or something remarkably similar, this semester, in my econometrics class which is very similar to the psych research methods class. I actually picked it up following a discussion with Doug McKee. He actually was doing it this semester too. He had a podcast episode on it. It sounded so exciting, I did something… a little bit different. I actually graded it but I didn’t give it back to them because I wanted to see what they had the most trouble with, and then I was going to have them only answer the ones in a group that they struggled with… and it turned out that that was pretty much all them anyway. So, it’s very similar to what you did except I gave them a weighted average of their original grade and the group grade and all except one person improved and the one person’s score went down by two points because the group grade was just slightly lower… but he did extremely well and he wasn’t that confident. The benefits to them of that peer explanation and explaining was just tremendous and it was so much more fun for them and for me and, as you said, it just completely wiped out all those things like “Well, that was tricky” because when they hear their peers explaining it to them the students were much more likely to respond by saying “Oh yeah, I remember that now” and it was a wonderful experience and I’m gonna do that everywhere I can.

In fact. I was talking about it with my TA just this morning here at Duke and we’re planning to do something like that in our classes here at TIP this summer, which i think is somewhat familiar to you from earlier in your academic career.

Michelle: That is right we do have this connection. I was among, not the very first year, but I believe the second cohort of Talent Identification Program students who came in, I guess you would call it now, middle school (back then, it was called junior high) and what a life-transforming experience. We’ve had even more opportunities to talk about the development of all these educational ideas through that experience.

John: That two-stage exam is wonderful and it’s so much more positive… because it didn’t really take, in my class, much more time, because I would have spent most of that class period going over the exam and problems they had. But the students who did well would have been bored and not paying much attention to it; the students who did poorly would just be depressed and upset that they did so poorly… and here, they were actively processing the information and it was so positive.

Michelle: That’s a big shift. We really have to step back and acknowledge that, I think. that is a huge shift in how we look at assessment, and how we think about the use of class time… and it’s not just “Oh my gosh, I have to use every minute to put such content in front of the students.” Just the fact that more of us are making that leap, I think, really is evidence this progress is happening… and we see also a lot of raised consciousness around issues such as learning styles. That’s another one that, when I go out and speak to faculty audiences, 10 years ago you would get these shocked looks or even very indignant commentary when you say “Ok, this idea of learning styles, in the sense that say there are visual learners, auditory learners, what I call sensory learning styles (VAK is another name it sometimes goes by). The idea that that just holds no water from a cognitive point of view…” People were not good with that, and now when I mentioned that at a conference, I get the knowing nods and even a few groans… people like “Oh, yeah. we get that. Now, K-12, which I want to acknowledge it’s not my area, but I’m constantly reminded by people across the spectrum that it’s a very different story in K-12. So, setting that aside… but this is what I’m seeing… that faculty are realizing… they’re saying “Oh, this is what the evidence says…” and maybe they even take the time to look at some of the really great thinkers and writers who put together the facts on this. They say “You know what? I’m not going to take my limited time and resources and spend that on this matching to styles when the styles can’t even be accurately diagnosed and are of no use in a learning situation. So, that’s another area of real progress.

Rebecca: What I am hearing is not just progress here in terms of cognitive science, but a real shift towards really thinking about how students learn and designing for that rather than something that would sound more like a penalty for grade like “Oh, did you achieve? Yes or no…” but, rather here’s an opportunity if you didn’t achieve to now actually learn it… and recognize that you haven’t learned it, even though it might seem really familiar.

John: Going back to that point about learning styles. It is spreading in colleges. I wish it was true at all the departments at our institution, but it’s getting there gradually… and whenever people bring it up, we generally remind them that there’s a whole body of research on this and I’ll send them references but what’s really troubling is in my classes the last couple years now, I’ve been using this metacognitive cafe discussion forum to focus on student learning… and one of the week’s discussions is on learning styles and generally about 95 percent of the students who are freshmen or sophomores (typically) come in with a strong belief in learning styles… where they’ve been tested multiple times in elementary or middle school… they’ve been told what their learning styles are… they’ve been told they can only learn that way… It discourages them from trying to learn in other ways and it does a lot of damage… and I hope we eventually reach out further so that it just goes away throughout the educational system.

Rebecca: You’ve worked in your classes, Michelle, haven’t you to help students understand the science of learning and use that to help students understand the methods and things that you’re doing>

Michelle: Yes, I have. I’ve done this in a couple of different ways. Now, partly, I get a little bit of a free pass in some of my teaching because I’m teaching the introduction to psychology or I’m teaching research methods where I just happen to sneak in as the research example will be some work on say attention or distraction or the testing effect. So, I get to do it in those ways covertly. I’ve also had the chance, although it’s not on my current teaching rotation… I’ve had the chance to also take it on as in freestanding courses. As many institutions are doing these days… it’s another trend… and what Northern Arizona University, where I work, has different kinds of freshmen or first-year student offering for courses they can take, not in a specific disciplinary area, but that really crossed some different areas of the student success or even wellbeing. So, I taught a class for awhile called Maximizing Brain Power that was about a lot of these different topics. Not just the kind of very generic study skills tip… “get a good night’s sleep…” that kind of thing… but really some again more evidence-based things that we can tell students and you can really kind of market it… and I think that we do sometimes have to play marketers to say “Hey, I’m going to give you some inside information here. This is sort of gonna be your secret weapon. So, let me tell you what the research has found.”

So, those are some of the things that I share with students… as well as when the right moment arises, say after an exam or before their first round of small stakes assessments, where they’re taking a lot of quizzes… to really explain the difference between this and high stakes or standardized tests they may have taken in the past. So, I do it on a continuing basis. I try to weave it into the disciplinary aspect and I do it in these free-standing ways as well… and I think here’s another area where I’m seeing this take hold in some different places… which is to have these free-standing resources that also just live outside of a traditional class that people can even incorporate into their courses… if say cognitive psychology or learning science isn’t their area… that they can bring in, because faculty really do care about these things. We just don’t always have the means to bring them in in as many ways as we would like.

John: …and your Attention Matters project was an example of that wasn’t it? Could you tell us a little bit about that?

Michelle: Oh, I’d love to… and you know this connects to what it seems to be kind of an evergreen topic in the teaching and learning community these days, which is the role of distracted students… and I know this past year there just have been these one op-ed versus another. There’s been some really good blog posts by some people I really like to follow in the teaching and learning community such as Kevin Gannon talking about “Okay, do you have laptops in the classroom? and what happens when you do?” and so I don’t think that this is just a fad that’s going away. This is something that the people do continue to care about, and this is where the attention matters project comes in.

This was something that we conceptualized and put together a couple years ago at Northern Arizona University with myself, and primarily I collaborated with a wonderful instructional designer who also teaches a great deal… John Doherty. So, how this came about is I was seeing all the information on distraction… I’m really getting into this as a cognitive psychologist and going “Wow, students need to know that if they’re texting five friends and watching a video in their class. It’s not going to happen for them.” I was really concerned about “What can I actually do to change students minds?” So, my way of doing this was to go around giving guests presentations in every classes where people would let me burn an hour of their class time… and not a very scalable model… and John Doherty respectfully sat through one my presentations on this and then he approached me and said “Look, you know, we could make a module and put this online… and it could be an open access within the institution module, so that anybody at my school can just click in and they’re signed up. We could put this together. We could use some really great instructional design principles and we could just see what happens… and I bet more people would take that if it were done in that format. We did this with no resources. We just were passionate about the project and that’s what we did. We had no grant backing or anything. We got behind it. So, what this is is about a one- to two-hour module that, it’s a lot like a MOOC in that it there’s not a whole lot of interaction or feedback, but there are discussion forums and it’s very self-paced in that way… so one- to two-hour mini MOOCs that really puts at the forefront demonstrations and activities… so we don’t try to convince students about problems with distraction and multitasking… we don’t try to address that just by laying a bunch of research articles on them… I think that’s great if this were a psychology course, but it’s not. So, we come at it by linking them out to videos, for example, that we were able to choose, that we feel really demonstrate in some memorable ways what gets by us when we aren’t paying attention… and we also give students some research-based tips on how to set a behavioral plan and stick to it… because just like with so many areas of life, just knowing that something is bad for you is not enough to really change your behavior and get you not to do that thing. so we have students talking about their own plans and what they do when, say, they’re having a boring moment in class, or they’re really really tempted to go online while they’re doing homework at home. What kinds of resolutions can they set or what kind of conditions can make that that will help them accomplish that. Things like the software blockers… you set a timer on your computer and it can lock you out of problematic sites… or we learned about a great app called Pocket Points where you actually earn spendable coupon points for keeping your phone off during certain hours. This is students talking to students about things that really concern them and really concern us all because I think a lot of us struggle with that.

So, we try to do that… and the bigger frame for this as well is this is, I feel, a life skill for the 21st century… thinking about how technology is going to be an asset to you and not detract from what you accomplish in your life. What a great time to be reflecting on that, when you’re in this early college career. so that’s what we try to do with the project…and we’ve had over a thousand students come through. They oftentimes earn extra credit. Our faculty are great about offering small amounts of extra credit for completing this and we’re just starting to roll out some research showing some of the impacts… and showing it in a bigger way just how you can go about setting up something like this.

Rebecca: I like that the focus seems to be on helping students with a life skill rather than using technology is just a blame or an excuse. We’re in control of our own behaviors and taking ownership over our behaviors is important rather than just kind of object blaming.

Michelle: So, looking at future trends, I would like to see more faculty looking at it in the way that you just described, Rebecca, as this is a life skill and it’s something that we collaborate on with our students… not lay down the law… because, after all, students are in online environments where we’re not there policing that and they do need to go out into work environments and further study and things like that. So, that’s what I feel is the best value. For faculty who are looking at this, if they don’t want to do… or don’t have the means to do something really formal like our Attention Matters approach, just thinking about it ahead of time… I think nobody can afford to ignore this issue anymore and whether you go the route of “No tech in my classroom” or “We’re going to use the technology in my classroom“ or something in between… just reading over, in a very mindful way, not just the opinion pieces, but hopefully also a bit of the research, I think, can help faculty as they go in to deal with this… and really to look at it in another way, just to be honest, we also have to consider how much of this is driven by our egos as teachers and how much of it is driven by a real concern for student learning and those student life skills. I think that’s where we can really take this on effectively and make some progress when we are de-emphasizing that ego aspect and making sure that it really is about the students.

John: We should note there’s a really nice chapter in this book called Minds Online: Teaching Effectively with Technology that deals with these types of issues. It was one of the chapters that got our faculty particularly interested in these issues… on to what extent technology should be used in the classroom… and to what extent it serves as a distraction.

Michelle: I think that really speaks to another thing which I think is an enduring trend… which is the emphasis on really supporting the whole student in success and what we’ve come to call academic persistence… kind of a big umbrella term that has to do with, not just succeeding in a given class, but also being retained… coming back after the first year. As many leaders in higher education point out, this is as a financial issue. As someone pointed out, it does cost a lot less to hang on to the students you have instead of recruiting more students to replace ones who are lost. This is, of course, yet another really big shift in mindset of our own, because after all we did used to measure our success by “Hey, I flunked this many students out of this course” or” Look at how many people have to switch into different majors…our major is so challenging…”

So, we really have turned that thinking around and this does include faculty now. I think that we did used to see those silos. We had that very narrow view of “I’m here to convey content. I’m here to be an expert in this discipline, and that’s what I’m gonna do…” and sure, we want to think about things like do students have learning skills? Do they have metacognition? Are they happy and socially connected at the school? Are they likely to be retained so that we can have this robust university environment?

We had people for that, right? It used to be somebody else’s job… student services or upper administration. They were the ones who heard about that and now I think on both sides we really are changing our vision. More and more forward-thinking faculty are saying “You know what? Besides being a disciplinary expert, I want to become at least conversant with learning science. I want to become at least conversant with the science of academic persistence…” There is a robust early literature on this and that’s something that we’ve been working on at NAU over this past year as well… kind of an exciting newer project that I like very much. We’ve started to engage faculty in a new faculty development program called Persistence Scholars and this is there to really speak to people’s academic and evidence-based side, as well as get them to engage in some perspective-taking around things like the challenges that students face and what it is like to be a student at our institution. We do some really selected readings in the area we look at things like mindset… belongingness… these are really hot areas in that science of persistence… in that emerging field. But, we have to look at it in a really integrated way.

It’s easy for people to say just go to a workshop on mindset and that’s a nice concept, but we wanted to think about it in this bigger picture… really know what are some of the strengths of that and why? Where do these concepts come from? What’s the evidence? That’s something that I think is another real trend and I think as well we will see more academic leaders and people in staff and support roles all over universities needing to know more about learning science. There are still some misconceptions that persist, as we’ve talked about. We’re making progress in getting rid of some of these myths around learning, but I will say… I’m not gonna name any names… but, every now and again I will hear from somebody who says “Oh well, we need to match student learning styles” or “Digital natives think differently, don’t you know?” and I have to wonder whether that’s a great thing. I mean, these are oftentimes individuals that have the power to set the agenda for learning all over a campus. Faculty need to be in the retention arena and I think that leaders need to be in the learning science arena. The boundaries is breaking down and it’s about time.

Rebecca: One of the things that I thought was really exciting with the reading groups that we’ve been having on our campus… that we started with your book, but then we’ve read Make it Stick and Small Teaching since… is that a lot of administrators in a lot of different kinds of roles engaged with us in those reading groups, it wasn’t just faculty. There was a mix of faculty, staff, and some administrators, and I think that that was really exciting. For people who don’t have the luxury of being in your persistence scholar program, what would you recommend they read to get started to learn more about the science of persistence?

Michelle: I really, even after working with this for quite some time, I loved the core text that we have in that program, which is Completing College by Vincent Tinto. It’s just got a great combination of passionate and very direct writing style. So, there’s no ambiguity, there’s not a whole lot of “on the one hand this and on the other hand that.” It’s got an absolutely stellar research base, which faculty of course appreciate… and it has a great deal of concrete examples. So, in that book they talk about “okay, what does it mean to give really good support to first semester college students? What does that look like?” and they’ll go out and they’ll cite very specific “Here’s a school and here’s what they’re doing… here’s what their program looks like… here’s another example that looks very different but gets at the same thing.” So, that’s one of the things that really speak to our faculty… that they really appreciated and enjoyed.

I think that as well we tested good feedback about work that’s come out of the David Yeager and his research group on belongingness and lay theories, and lay theories is maybe a counterintuitive term for kind of a body of ideas about what students believe about academic success and why some people are successful and others are not and how those beliefs can be changed sometimes through relatively simple interventions and when it happens we see great effects such as the narrowing of achievement gaps among students who have more privilege or less privileged backgrounds… and that’s something that, philosophically, many faculty really really care about but they’ve never had the chance to really learn “Okay, how can I actually address something like that with what I’m doing in my classroom, and how can I really know that the things that I’m choosing do have that great evidence base…”

John: …and I think that whole issue is more important now and is very much a social justice issue because, with the rate of increase we’ve seen in college cost inflation, people who start college and don’t finish it are saddled with an awfully high burden of debt. The rate of return to a college degree is the highest that we’ve ever seen and college graduates end up not only getting paid a lot more but they end up with more comfortable jobs and so forth… and if we really want to move people out of poverty and try to reduce income inequality, getting more people into higher education and successfully completing higher education is a really important issue. I’m glad to see that your institution is doing this so heavily and I know a lot of SUNY schools have been hiring Student Success specialists. At our institution they’ve been very actively involved in the reading group, so that message is spreading and I think some of them started with your book and then moved to each of the others. So, they are working with students in trying to help the students who are struggling the most with evidence-based practices …and I think that’s becoming more and more common and it’s a wonderful thing.

Rebecca: So, I really liked Michelle that you were talking about faculty getting involved in retention and this idea of helping students develop persistence skills, and also administrators learning more about evidence-based practices. There’s these grassroots movements happening in both of these areas. Can you talk about some of the other grassroots movements that are working toward, or efforts that faculty are making to engage students and capture their attention and their excitement for education?

Michelle: Right, and here I think a neat thing to think about too is just it’s the big ambitious projects… the big textbook replacement projects or the artificial intelligence informed adaptive learning systems… those are the things that get a lot of the press and end up in The Chronicle of Higher Education that we read about… But, outside of that, there is this very vibrant community and grassroots led scene of developing different technologies and approaches. So, it really goes back for a while. I mean, the MERLOT database that I do talk about in Minds Online has been trove for years of well hidden gems that take on one thing in a discipline and come at it from a way that’s not just great from a subject-matter perspective but brings up the new creative approaches. In the MERLOT database, for example, there’s a great tutorial on statistical significance and the interrelationship between statistical significance and issues like simple sizes. You know, that’s a tough one for students, but it has a little animation involving a horse and a rider that really turns it into something that’s very visual… that’s very tangible… and it really actually tying into analogies, which is a well-known cognitive process that can support the advancement of learning something new. There is something on fluid pressures in the body that was treated for nursing students by nurses, and it’s got an analogy of a soaker hose that this is really fun and is actually interactive. So, those are the kinds of things. The PhET project, P-h-E-T which comes out of University of Colorado, that has been around for a while… again, faculty-led and a way to have these very useful interactive simulations for concepts in physics and chemistry. So, that’s one. CogLab, that’s an auxiliary product that I’ve used for some time in like hundred psychology courses that simulates very famous experimental paradigms which are notoriously difficult to describe on stage for cognitive psychology students. That started out many years ago as a project that very much has this flavor of “We have this need in our classroom. We need something interactive. There’s nothing out there. Let’s see what we can build.” It has since then picked up and turned into a commercial product, but that’s the type of thing that I’m seeing out there.

Another thing that you’ll definitely hear about if you’re circulating and hearing about the latest project is virtual reality for education. So, with this it seems like, unlike just a few years ago, almost everywhere you visit you’re going to hear that “Oh, we’ve just set up a facility. We’re trying out some new things.” This is something that I also heard about when I was talking to people when I was over in China. So, this is an international phenomenon. It’s going to pick up steam and definitely go some places.

What also strikes me about that is just how many different projects there are. Just when you’re worried that you’re going to be scooped because somebody else is going to get there first with their virtual reality project you realize you’re doing very very different things. So, I’ve seen, for example, it used in a medical application to increase empathy among medical students… and I took a six or seven minute demonstration that just was really heart-rending, simulating the patient experience with a particular set of sensory disorders… and at Northern Arizona University we have a lab that is just going full-steam in coming up with educational applications such as interactive organic chemistry tutorial that is is just fascinating. We actually completed a pilot project and are planning to gear up a much larger study next semester looking at the impacts of this. So, this is really taking off for sure.

But, I think there are some caveats here. We still really need some basic research on this… not just what should we be setting up and what the impacts are but how does this even work? In particular, what I would like to research in the future, or at least see some research on, is what kinds of students… what sort of student profile… really gets the most out of virtual reality for education. Because amidst all the very breathless press that’s going on about this now and all the excitement, we do have to remember this is a very, very labor intensive type of resource to set up. You’re not just going to go home and throw something together for the next week. It takes a team to build these things and to complete them as well. If you have, say, a 300 student chemistry course (which is not atypical at all… these large courses), you’re not going to just have all of them spend hours and hours and hours doing this even with a fairly large facility. It’s a very hands-on thing to guide them through this process, to provide the tech support, and everything else.

So, I think really knowing how we can best target our efforts in this area, so that we can build the absolute best, with the resources we have, and maybe even target and ask the students who are most likely to benefit. I think those are some of the things that we just need to know about this. So, it’s exciting for somebody like me who’s in the research area. I see this as a wonderful open opportunity… but those are some of the real crossroads we’re at with virtual reality right now.

Rebecca: I can imagine there’s a big weighing that would have to happen in terms of expense and time and resources needed to startup versus what that might be saving in the long run. I can imagine if it’s a safety thing that you want to do a virtual reality experience, like saving people’s lives and making sure that they’re not going to be in danger as they practice particular skills, could be a really good investment in these… spending the resources to make that investment… or if it’s a lot of travel that would just be way too expensive to bring a bunch of students to a particular location… but you could virtually… it seems like it would be worth the start-up costs and those are just two ideas off the top of my head where it would make sense to bend all of that resource and time.

John: …and equipment will get cheaper. Right now, it’s really expensive for computers that have sufficient speed and graphics processing capability and the headsets are expensive, but they will come down in price, but as you said, it’s still one person typically and one device… so it doesn’t scale quite as well as a lot of other tools or at least not at this stage.

Rebecca: From what I remember, Michelle, you wrote a blog post about [a] virtual reality experience that you had. Can you share that experience, and maybe what stuck with you from that experience?

Michelle: Right, so I had the opportunity, just as I was getting to collaborate with our incredible team at the immersive virtual reality lab at NAU… one of the things I was treated to was about an hour and a half in the virtual reality setup that they have to explore some of the things that they had… Giovanni Castillo, by the way, is creative director of the lab and he’s the one who was so patient with me through all this. We tried a couple of different things and of course there’s such a huge variety of different things that you can do.
There’s a few things out there like driving simulators that are kind of educational… they’re kind of an entertainment… but he was just trying to give me, first of all, just a view of those… and I had to reject a few of them… I will say, initially, because I am one of the individuals who tends to be prone to motion sickness. So, that limits what I can personally do in VR and that is yet another thing that we’re gonna have to figure out. At least informally, what we hear is that women in particular tend to experience more of this. So, I needed, first of all, to go to a very low motion VR. I wasn’t gonna be whizzing through these environments. That was not going to happen for me. So, we did something that probably sounds incredibly simplistic, but it just touched me to my core… which is getting to play with Google Earth. You can spin the globe and either just pick a place at random or what Giovanni told me is… “You know, I’ve observed that when people do this, when we have an opportunity to interact with Google Earth, they all either go to where they grew up or they’ll go to someplace that they have visited recently or they plan to visit. So, I went to a place that is very special to me and maybe it doesn’t fit into either one of those categories neatly, but it’s my daughter’s University… her school… and I should say that this is also a different thing for me because my daughter goes to school in Frankfort, Germany… an institute that is connected to a Museum. So, I had only been to part of the physical facility… the museum itself… and it was a long time ago… and part of it was closer to the holiday. So, this is my opportunity to go there and explore what it looks like all over… and so, that was an emotional experience for me. It was a sensory experience… it was a social one… because we were talking the whole time… and he’s asking me questions and what kinds of exhibits do they have here… and what’s this part of it. So, that was wonderful. it really did give me a feel for alright, what is it actually like to be in this sort of environment?

I’m not a gamer. I don’t have that same background that many of our students have. So, it got me up to speed on that… and it did show me how just exploring something that is relatively simple can really acquire a whole new dimension in this kind of immersive environment. Now the postscript that I talked about in that blog post was what happened when I actually visited there earlier in the year. So, I had this very strange experience that human beings have never had before… which is from this… I don’t know whether to call it deja vu or what… of going to the settings and walking around the same environment and seeing the same lighting and all that sort of stuff that was there in that virtual reality environment… but this time, of course, with real human beings in it and the changes… the little subtle changes that take place over time, and so forth.

So, how does it translate into learning? What’s it going to do for our students? I just think that time is going to tell. It won’t take too long, but I think that these are things we need to know. But, sometimes just getting in and being able to explore something like this can really put you back in touch with the things you love about educational technology.

Rebecca: I think one of the things that I’m hearing in your voice is the excitement of experimenting and trying something… and that’s, I think, encouragement for faculty in general… is to just put yourself out there and try something out even if you don’t have something specific in mind with what you might do with it. Experiencing it might give you some insight later on. it might take some time to have an idea of what you might do with it, but having that experience, you understand it better… it could be really useful.

John: …and that’s something that could be experienced on a fairly low budget with just your smartphone and a pair of Google cardboard or something similar. Basically, it’s a seven to twelve dollar addition to your phone and you can have that experience… because there’s a lot of 3D videos and 3D images out there on Google Earth as well as on YouTube. So, you can experience other parts of the world and cultures before visiting… and I could see that being useful in quite a few disciplines.

Rebecca: So, we always wrap up with asking what are you going to do next?

Michelle: I continue to be really excited about getting the word out about cognitive principles and how we can flow those in to teaching face-to-face with technology… everything else in between. So, that’s what I continue to be excited about… leveraging cognitive principles with technology and with just rethinking our teaching techniques. I’m going to be speaking at the Magna Teaching with Technology Conference in October, and so I’m continuing to develop some of these themes… and I’m very excited to be able to do that. I’m right now also… we’re in the early stages of another really exciting project that has to do with what we will call neuromyth… So, that may be a term that you’ve turn across in some of your reading. It’s something that we touched on a few times, I think, in our conversation today… the misconceptions that people have about teaching and learning and how those can potentially impact the choices we make in our teaching. So, I’ve had the opportunity to collaborate with this amazing international group of researchers who’s headed up by Dr. Kristen Betts of Drexel University… and I won’t say too much more about it other than we have a very robust crop of survey responses that have come in from, not just instructors, but also instructional designers and administrators from around the world. So, we’re going to be breaking those survey results down and coming up with some results to roll out probably early in the academic year and we’ll be speaking about that at the Accelerate conference, most likely in November. That’s put out by the Online Learning Consortium. So, we’re right in the midst of that project and it’s going to be so interesting to see what has the progress been? What neuromyths are still out there and how can they be addressed by different professional development experiences. We’re continuing to work on the Persistence Scholars Program on academic persistence. So, we’ll be recruiting another cohort of willing faculty to take that on in the fall at Northern Arizona University. I am going to be continuing to collaborate and really work with and hear from John and his research group with respect to the metacognitive material that they’re flowing into foundational coursework and ways to get students up to speed with a lot of critical metacognitive knowledge. So, we’re going to work on that too… and I like to keep up my blog and work on shall we say longer writing project but we’ll have to stay tuned for that.

Rebecca: Sounds like you need to plan some sleep in there too.

[LAUGHTER]

John: Well, it’s wonderful talking to you, and you’ve given us a lot of great things to reflect on and to share with people.

Rebecca: Yeah. Thank you for being so generous with your time.

John: Thank you.

Michelle: Oh, thank you. Thanks so much. It’s a pleasure, an absolute pleasure. Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts, and other materials on teaforteaching.com. Theme music by Michael Gary Brewer. Editing assistance from Nicky Radford.

31. Writing Better Writing Assignments

Posted in higher education, pedagogy, professional development, scholarship of teaching and learning, teaching, writing
May 30, 2018

Complaints about student writing are embedded in faculty conversations across disciplines. What if the issues with student writing, though, are not their fault, but ours instead? In this episode, Allison Rank and  Heather Pool join us to share suggestions about writing better writing prompts that provide student with explicit expectations.

Allison Rank is Assistant Professor of Political Science at the State University of New York at Oswego and Dr. Heather Pool is an Assistant Professor of Political Science at Denison University.

Show Notes

  • Rank, A., & Pool, H. (2014). Writing Better Writing Assignments. PS: Political Science & Politics, 47(3), 675-681. https://doi.org/10.1017/S1049096514000821
    Hypothesis
  • Bean, J. C. (2001). Engaging ideas. The Professor’s Guide to Integrating Writing, Critical.
  • Rockmore, E. B. (2015). How Texas teaches history. The New York Times, A31.
  • Braver, Lee (2014). How I Mark Up Philosophy Texts. APA Newsletters, Fall, 14,1 Special section. p. 13

Transcript

Rebecca: Complaints about student-writing are embedded in faculty conversations across disciplines. What if the issues with student-writing though are not their fault but ours instead? In this episode, we’ll talk about writing better prompts to make explicit what the expectations are and how to get there.

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an Economist.

John: And Rebecca Mushtare, a Graphic Designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

John: Today, our guest are Dr. Allison Rank, Assistant Professor of Political Science at the State University of New York at Oswego and Dr. Heather Pool, Assistant Professor of Political Science at Denison University. Allison and Heather are co-authors for an article titled Writing Better Writing Assignments published in Political Science and Politics. Welcome, Allison and Heather.

Heather: Thank you.

Allison: Thanks.

Rebecca: So, welcome back to Allison, I think, right?

Allison: Yes.

John: Yes, welcome back, Allison.

Rebecca: So, today, our teas are?

John: Tea Forte, black currant black tea.

Allison: Water again.

Rebecca: It was coffee last time.

Allison: Okay.

Heather: I’m also water because I forgot that this was tea-oriented.

Rebecca: Yeah. Yeah. We have to send out those reminders ahead of time, I guess. Mine is Harney & Sons Paris tea.

John: What prompted your interest in writing about writing assignments?

Heather: I’ll start with that. I was director of a writing center at the University of Washington for a social science writing for a couple of years and then, Allison filled my seat after me. It was after we had seen numerous prompts that our students were coming in and asking for help with, and Allison, after she had completed her time at the writing center, came to me and was like, “I think we can do this. We can do some people feedback about how to do a better job at writing these.” We saw a lot of prompts that could have been more clear, let’s just say that.

Rebecca: Were there prompts that you didn’t understand?

Allison: I think usually we could figure out how to interpret them, but it was very easy to see why students couldn’t figure out how to interpret them.

Heather: Yeah. Right. And so, oftentimes, what happens is prompts are basically dissertations, right? Where you could literally write hundreds of pages on them or they’re so narrow that if you answer all of the questions, then, there’s no space for analysis or creativity or anything like that.

Allison: To add some details, so Heather had that job for two years and then, I had the job for two years. We’ve had four years between us of seeing these various prompts come in across the sub fields of political science and we’re actually seeing a lot of very similar problems and prompts on very different topics, which I think, for us, was part of being able to think about it’s the structure of how we write the prompts and how professors think about prompts is actually a place for an intervention and then, starting to teach our own classes sort of getting the sense that sometimes what comes back from students is on them, but also, we need to be a little bit more responsible around what it is we ask students to do because sometimes, some components of their poor writing may actually be more our fault than we’d like to admit.

Rebecca: I think we can all probably experience the idea that you get something back here like, “Yeah.” “Yeah, yeah, you answered that, yup.”

Allison: Yeah.

Heather: Right. Well, and part of it too, just to follow up on what Allison said, is we ourselves were early career and we’re just writing our own writing assignments for the first time. As a TA, you sort of inherit the assignments that people write and you’re like, “Okay, yeah. We can work with that.” But then when it comes to create your own, there’s no roadmap out there at all, and so, you stumble into stuff and you write assignments that the students have no idea how to interpret. And so, on the one hand, it was seeing some things that were out there that we thought, “Wow. There’s problems here. There’s commonalities,” and we can imagine how to get out of that problem and part of it was self help.

Allison: Yeah.

Heather: We’re looking for a resource that didn’t exist and Allison’s brilliant idea was like, “Ooh, we could create that resource.”

Allison: Yeah.

Heather: And so, that was a huge part of it.

Rebecca: Faculty definitely want students to be good writers …

Allison: Yeah.

Rebecca: … but we expect students to come in with those skillsets often and faculty often see themselves as content providers but not necessarily writing instructors. And I think that we hear that a lot even on campuses where writing across the curriculum exists. What role do you see faculty having in helping students develop their writing beyond just the prompt?

Allison: I think that faculty have a really important role to play on writing, but I think part of it comes from knowing what it is that you want to help students improve and having reasonable expectations for what the class that you have set up can actually help students do. In doing our research, when Heather’s saying we had a hard time finding roadmaps as we dug into a lot of the Bloom’s taxonomy literature and trying to figure out if we’re writing prompts that asks students to take particular steps, are we actually providing students a roadmap for those steps.

Allison: So, one of the things that I struggle with a lot is the way in which I don’t recognize that I’ve been disciplined. So, I’ve been disciplined as a political scientist. I ask questions in a way that political scientists ask questions, and then, get mad when my students don’t understand. That’s part of my expectations. But I also never make that explicit in content, even in the content-driven courses. That the way I’m approaching this content is about a political science perspective and here’s how that might be different and here’s how those expectations should then influence the way that you write a paper or approach a question.

Allison: And so, I think that it’s linking up the expectations for helping students with writing to the expectations we have around content-delivery is I think where a lot of faculty should spend more time.

Heather: I teach political theory, it’s not really a testable subject, and I could do a test but I don’t think that’s a particularly helpful way to evaluate people’s engagement with the content. On some level, I actually think it’s a cap out when faculty members say that they’re only content providers in part because I think we learn through writing and it’s not until we’re actually able to write about things that we grasp the kind of significance and the meaning and all of those things and we actually have some research. And I think in being … engaging ideas, I could be wrong, she suggests that we learn as we write. It’s only in the process of actually trying to put other people’s words into our own context that we actually grasp what’s going on.

Heather: And so, to be effective content teachers, I think we need to figure out how to be effective writing teachers as well and I think it’s important to be clear when we’re asking them to summarize and when we’re asking them to analyze and when we’re asking them to evaluate and those are all different things. And we need to give them opportunities to work on those things before we have them write big final papers or we ask them to do all of those without any scaffolding.

Rebecca: So, speaking of those nice keywords, I know that I’ve had conversations with students and they can’t actually tell me the difference between describe, analyze, reflect, things like that. So, can you share a little bit about how you might frame that for students, what those words mean and how you structure that?

Allison: Sure. Now, I’ll say off the top, I think that faculty, a lot of the time, don’t know what they mean when they use those specific terms. And so, part of what we would actually see in the writing center is prompts that said describe, but we read them a no, that if you actually just described, you are not going to get a good grade on this paper. That that was the word that was in the prompt, but I would bet money, if you follow those instructions, you would have problems. So, I think, I occasionally, for students to actually define the terms that are in the prompt, if I’m asking you to analyze let’s walk through in class one day, what would be the difference between summarizing this content and analyzing this content, so, actually walking them through what the terminology is.

Allison: I also think that that’s where having sub prompts after a prompt can be really helpful, where you break down for students that I expect you to summarize or describe a particular amount of the content and then, analyze something so that they have to distinguish for themselves what part of this assignment am I addressing in different components of my paper.

Heather: Yeah, I think that’s great. I do things like I have students do small stakes regular assignments where I have them summarize and then, reflect, and then, ask a question. And so, they’re already thinking about the difference between summary and reflection and then, I actually, in class, will talk about what’s the difference between describing something and analyzing something and one example that I use, because I went to grad school in Seattle is I’m getting off of a plane in Seattle. Seventy percent of the people on the plane are wearing super awesome Gore Tex water repellent gear and 20% of them are wearing wool and 10% of them aren’t wearing coats. So, that’s a description of the situation. But analysis is telling me why that’s the case. That’s trying to explain what we see and to make sense of it.

Heather: So, I then ask them to come up with reasons why, what that description says makes sense or what stories they can tell about why that’s what they see. There’s also a great piece, it’s the Netflix … the new Sherlock Holmes, it’s the lady in pink where he walks into a room and he sees a woman dead on the floor, and then, Sherlock Holmes goes through and comes up with all the stories about the particular things that he’s seeing are what he’s seeing. And it’s a really effective tool for students to be like, “Oh, summary is really different,” right? And many times, prior instructors may have asked them to summarize, and so, they’re relatively good at that, but it’s the analysis part that they really struggle with. Again, I think it’s our job to help them figure out what analysis actually is.

Rebecca: One of the things that I’ve noticed in my own department, we’ve been talking about writing in our department quite a bit lately. We had a conversation … My department is made up of art historians, designers and studio artists that all makes up like an art and design department. So, it seems like it’s all one discipline but we all have really different cultures within that discipline, and that we talk through what’s some of the kinds of writing that we do on our department and discovered that we didn’t really mean the same thing.

Rebecca: And so, we’re working on developing a common language and sharing that out within our own department to make sure that we can be consistent between levels because I think that’s some of the confusion that our students are experiencing.

Heather: Yeah, I think that’s absolutely right, and of course, writing across disciplines varies greatly, so we may put these statements in different places if we’re in the Humanities or if we’re in the Social Sciences, we may approach quotes differently and whether or not it’s appropriate to use them or not appropriate to use them, what counts as evidence differs from discipline to discipline. And the way I set that up for students is to say, “You’re going to end up in jobs where you don’t actually know what they want when they ask you to write something, and you’re going to need to be able to figure that out, and that’s actually what we’re trying to give you here is the ability to approach a writing practice and figure the rules out. And there’s different rules in different disciplines and your job is to develop the facility to be able to move between those things as needed.”

Allison: Yeah. I’ve done something in class with my intro class which tends to be … it’s very frequently a general education class. There tends to be students from a lot of different majors and actually, just asking them how do you think you’re supposed to write paper. And you’ll get all sorts of answers about …

Heather: Right.

Allison: … what a thesis statement is supposed to be, you should never use I. Which is a thing in political science, it’s like “No, I’ve got a correct that right now.”

Rebecca: Yeah.

Heather: Right.

Allison: You need to tell me I argue X, Y and Z and students are so taken aback, but it’s so much easier if you start, at least for me, by getting them to tell you what are all the rules you think you know so that I know where I need to tell you that at least for this class in this space, that’s not the right rule.

John: But part of it is just being more transparent with students …

Heather: Yes.

Allison: Yeah.

John: … and making sure they understand what you expect from them in terms of coming up with good writing prompt. You mentioned scaffolding a bit.

Allison: Yeah.

John: How do you scaffold it in terms of the stages of writing? How do you break it up for students or do you have them just submit it in whole draft or what?

Allison: Yeah. I think it really depends for me on different classes. So, for the intro class, before their four-page papers, they write a couple of four-page papers, they do something called reading reflections but it’s really a worksheet where they have to tell me the author or authors, the title, what type of source is it using the Chicago style guide. It’s essentially breaking out for them, everything they would need to know for citations, they have to tell me the research question, what they think the thesis statement would be in their own words, which again, is to get them in this format of saying like Madison argues X, Y, and Z. A couple of good quotations and then, their own initial impression of the piece. So then, when they sit down to write the paper, they already have the stack of material that’s like, “Oh, if I want to argue X, who would I go to as evidence to support that claim?”

Allison: In my advanced classes, I tend to break it down more in terms of the annotated bibliography, so before they would ever touch writing a longer paper, I first want an annotated bibliography and I do it slightly different than a “normal annotated bibliography” I ask for one paragraph of summary, and then, for every entry, I need one more paragraph that tells me the relationship between that piece and at least two other pieces in the annotated bibliography. So, getting them to think through what are the relationships that help them categorize where a literature review could go before throwing literature review on top of what it is that they have to write. And I may have stolen the annotated bibliography from Heather.

Heather: It’s possible, [inaudible 00:13:03] annotated bibliographies, yes. So, yeah, I do some similar things. I started to use Allison’s reading reflection assignment that I’m inching closer and closer to that mostly because I’m a little overwhelmed by grading. I have them do seven of these reading responses, I call them, where they do summary, reflection, and then, ask a discussion question. So, that is getting them to train to summarize stuff, and again, the point is they have to do one of those for each of the authors that we read, so they actually have a pretty decent summary and they have the other 24 summaries from people in the class that they can go to when it comes to writing their own papers.

Heather: And then, for my intro class, I hand a paper out and they need two and a half or three weeks before it’s due and then, I require a draft on say Tuesday, they then do peer review in class on Thursday, and then, the final draft of the paper is due the following Tuesday. So, they have to have a pretty decent working draft a week before the paper is due. And if you make a good effort, then, there’s no deductions from your final grade so it’s not a graded assignment but it is one that if you don’t do it will hurt you, and the same thing for sub [inaudible 00:14:06] good faith peer review. I like that a lot because the peer reviewers catch really irritating things that when I see them time after time after time, I get angry.

Heather: And so, the peer reviewers catch a lot of that where they say things like, “You seemed to have a problem with paragraph structure,” and somehow, when they’re hearing that from their peers and then, they hear the same thing from me when I give them feedback, I then ask them to do a reflection on the feedback that basically enforces them reading the comments, where one of the questions is, “Do you any commonalities in the feedback you’ve received from your peers and myself?” and surprisingly, there often is commonality there. And so, then, I start to get them thinking about what their patterns of error and what can they do to address those patterns of error.

Heather: So, in terms of scaffolding, I make it due early and I make a little stakes draft and then, they have a week where they can talk to their peers, they can come talk to me on office hours, et cetera, so that the paper that they turn in has been seen by at least two other pairs of eyes.

Allison: Yeah. I should say I do in my advanced classes, I have a version of that where there’s a draft due two weeks before finals week and then, students do not get evaluated on their drafts, they get evaluated on the quality of their feedback.

Heather: … what level?

Allison: Like a 5% grade. Yeah, I think it’s 2% you turned in a draft and then, after that, I have a sheet, I was doing it not graded, just sort of the participation points and I would get feedback that was like, “I really liked what you did here.” And I was like, “No.”

Heather: No.

Allison: “This is not going to work for me,” and so, changed it to where there’s an actual rubric for me to evaluate the feedback that they provide each other, and that has gotten students to give much more direct feedback to many students.

John: That was something I was just going to ask, have you used rubrics, and what do you see as the advantage of using a rubric for assessment?

Heather: Because we have writing-specific classes, and then, we have ones that aren’t but frankly, all of mine would qualify for the W overlay just because of the percentages. Teaching, I really care about writing, so that’s a simple part of the course but not all of them are Ws and if there are Ws, they have lower numbers of students and I can’t offer only Ws for curriculum reasons. And so, generally, all my classes are really heavy on writing, and so, I’ve moved more and more towards pretty specific rubrics where I basically highlight and bold stuff, and then, have a relatively short comment section. And I’ve just switched to a new rubric this semester and I actually think I like it.

Heather: I tend to over comment on their papers when I’m not constraint by a rubric and constraint by space, frankly. And so, for me, I’m a big fan, right? “This is what an A paper looks like,” “Here are four different categories that I’m assessing you on,” “This is what a B paper looks like,” “Here are four different categories for that as well.” And so, I’m tentatively enthusiastic about pretty specific rubrics.

Allison: I like very specific rubrics for intro classes. I have a hard time using very specific rubrics in a lot of my advanced classes, and I think it’s because I struggle to write rubrics that I think are balanced, aligned on being detailed enough to be a value but broad enough to where students can really sort of flex their muscles when it’s an open research question.

Allison: And then, a lot of my advanced classes, it’s an open question. And so, then, I find I have a rubric but it ends up being like on these criteria, would you be rated as excellent, good, fair, weak, poor. And so, it tells them where they are and then, with comments, but it’s nowhere near the level of sort of fine green value of the rubric that my intro classes have where everyone’s writing on the exact same question.

Rebecca: You are both hinting at differences in the role that a faculty member might play in different levels of courses between intro, intermediate and advanced. Can you explicitly address that and what the faculty member’s role is in each of those kinds of levels?

Heather: Yeah. I think I do something similar to what Allison does with my upper division classes, which I just taught at senior seminar. I have them do essentially two kind of shorter papers that are kind of lit reviewee where I’m asking them of pretty specific question about some segment of the course reading. And then, I have a big where like, you tell me what your research question is and then, they go through a proposal, an annotated bibliography, a draft with a clear pieces and then, a final draft. And that starts basically from the fall … the first one of those is the proposal was basically due at the beginning of November and the final paper isn’t due until the middle of December.

Heather: And so, I’m also a really big fan of if you don’t like the topics I wrote, then, you write one and tell me what you would like to write on, in part, because I think we say this in the paper, I actually am really interested in reading interesting papers. I would much rather read a paper that incorporates the material from the class in a way that you find compelling and that you want to write about, than I would read your [wrote 00:18:40] response to my question that you found really stupid. And so, I do give them that freedom but with a caveat that they do have to come talk to me. And I give students actually that freedom for intro, all the way up to seniors and my most advanced classes. But I do think it’s different when I’ve got a cinema major versus when I’ve got Political Science majors. I talk about writing and use examples in different ways across those levels.

Allison: Yeah. And I think that I, in the same way that you wouldn’t deliver the same content from intro to American government to an advanced American government class, I think it’s the same in terms of writing skills. And so, I tend to focus more in the intro class on these statements. Trying to lay bare some of the relatively, I would think in some ways, rudimentary, the thesis statements are deeply complicated space focusing on the building blocks of being in the discipline. These expectations of writing and then, in the more advanced levels, focusing on the types of writing that I think are in different forms more likely to be both of interest to them, but then, also, let them test skills that are more likely to be relevant.

Allison: So, for instance, I don’t have any full papers in my advanced classes usually outside of the big papers that are due during finals week, but for every book we read in the class, I have them do a critical analysis. It’s essentially a book review, but I found that if I call it a book review, I get book reports, which is not what I want so I call it a critical analysis. And the guidelines are I want no more than a half page of summary and then, up to two and a half pages of analysis. I don’t read anything over page three, with the idea of you need to be concise. I don’t want it to be summary and I give a set of prompts about what you can … here’s some places you might want to go but they’re very open in terms of talking about the content you know from your broadcast communication class that I haven’t read or how does this book help you think differently about some event that happened on campus or is happening in the news.

Allison: Where I think that that type of analytical skills sort of more what I want my advanced students to start being able to do, this thing I read in the classroom connects to some broader literature in political science or literature from another discipline or just the way I interpret the words. And that’s where I see the writing in the advanced classes outside of the research papers as more my responsibility.

Heather: I was just going to follow up with I’m teaching seniors again and they’re on the job market themselves and trying to figure out why they just did a major in Political Science and trying to actually have them answer a question that is meaningful to them as opposed to like I need to know that you know how to read a book and find a piece of statement. And so, really, trying to create space for more advanced students to do more advanced interesting things.

John: In the classes where you use rubric, do you share the rubrics with students? Because I would think that would give them a little bit more scaffolding in letting them know what you think is important and helping them determine how to structure the papers and things.

Allison: Yeah. I would say sometimes I do, and sometimes, I don’t. In introductory classes where I have students that are already very concerned about doing things “right”, I actually tend not to because I find that they then hew to the rubric in ways that are actually really counter-productive. I’ll give them more of what I would consider the left-hand column of the rubric. So, I’ll take into account, when I’m grading, your citations will be 10%. Your grammar and style will be up to 10%. Your thesis is going to be worth 20% so that they know how points are distributed. But I don’t actually like to give the specific boxes that are sort of it’s going to be an excellent if there are x criteria, because I found that that tends to lead to really I think counter-productive conversations about well, how do I meet the standard of that box, as opposed to what makes a good analytical argument.

Heather: I don’t put percentages on my rubrics. I’m a big fan of the visual rubric where I’m like there’s a lot of things in the C columns and what’s a C. There are a few things in the A column but there’s mostly things in the B column, that’s a B+. I’m a political theorist, we don’t really do quantitative things particularly well. I’m a big fan of not sharing that because I don’t actually know how to do that. Some rubrics, I share with them. I share rubrics about their participation with them, like here’s what I expect a good participant in this class to be able to do, and I assess them on that, but I just started using this new rubric so I didn’t share that with them at the beginning of this term, but now, I think maybe I should have. I don’t know if I think it would help them or hurt them, I wonder.

Rebecca: I have detailed rubrics that I use for grading and I just started using our learning management system to use the rubrics and I found that that actually can be really challenging because when they do that on paper, I sometimes circle the line between things.

Heather: Yes, right. Me to. Me to.

Rebecca: And then, you have to pick one …

John: But if you’re doing it in Blackboard or some other learning management system, you can always override the …

Rebecca: Yeah. Well, I was just going to …

John: … if someone works outside the box.

Rebecca: … which I have done. Yeah. And I sometimes will make a comment if I put it in the C column, the comment is to why it’s there if it wasn’t one of the criteria I had originally come up with, and so, it’s very clear. So, I’ve been experimenting with that a little bit. I tend to share the rubric, but I also find that students tend not to look at the rubric.

Heather: Until it comes back with a letter grade on it, and then, they’re like, “But why? What happened?”

Allison: Yeah. I will say that I use the point rubric in Blackboard for classes where the size or the amount of papers … and so, it’s basically just for intro where it speeds grading.

Heather: Yeah.

Allison: Right? That’s when I do a points rubric in Blackboard, but even then, the idea that Blackboard defaults you to having three categories and that I always go in and have like no, I definitely need a couple of more point variations, yeah.

John: I usually have four or five on that in mind.

Allison: [inaudible 00:24:20] Yeah, yeah. I have to go in and sort of add because I tried doing it with three ones, and I was like, “Why is everyone getting a 30?” It’s like …

John: Well, you can pick whatever categories or …

Allison: Yeah. Yes, and so, I had to go in. That was my first experience using it last year and I was like, “Well, that’s wrong.” Let’s go back and then, regrade it, and that changed all the rubrics. But okay now, yeah, it takes a little learning.

John: But it can be an iterative process …

Allison: Yes.

John: … where if there’s some work, you can modify it.

Rebecca: Yeah. That’s also why I often don’t put percentages for the categories upfront is because I sometimes see what I get back to see if I need to adjust what I thought the weights were going to be to make it more fair.

Heather: I struggle with the percentages because writing is hard to do in any sort of objective fashion and I worry about the kind of thesis is a percentage, because sometimes they write a not so great thesis and have a brilliant paper, right? And so, then, you’re like, “Well, okay. So, you got 75% on the way there on your thesis,” but your argument at the end was actually really good and so, my feedback is write a clearer thesis because your argument’s really interesting, but it’s hard for me to figure out how to do that.

Rebecca: One of the things that I think we haven’t addressed but hinted at a little bit is not only is there disciplinary ways of approaching writing but there’s cultural ways of approaching writing too. And so, when you’re talking, Allison, about needing to write really concisely, that’s something that’s popular in design as well.

John: And in economics.

Rebecca: Yeah.

John: Like economical writing.

Rebecca: Right. But I often have students who want to write with very flowery language or think the academic writing looks a particular way, and usually, it’s very convoluted, very complicated sentences that don’t make any sense.

Allison: Yeah. Yeah.

John: I wonder where they get that.

Allison: Often, I train political science but I always, in my intro classes and occasionally, in my advanced classes, depending on how many students it would be a repeat for pass out a piece called How Texas Teaches History from, I believe Ellen Rockmore. It was an op ed in the New York Times a couple of years ago about the high school textbooks that had gone out in Texas where all of the “benefits” of a slave-holding society, which is a deeply-problematic framing. Masters taught slaves Christianity has an active phrasing, and then, all of the brutalities of slavery are framed in passive ways. Slaves were beaten, slaves were assaulted. And so, you excuse any actors, and I passed that out to students before we do, sort of when I complained about your grammar, when I correct grammar, I’m not doing it because this is a pedantic exercise and I just want you to meet these standards. I do it because in political science, it is incredibly important that we are accountable for who the agents are that act, and the only way that I know who your agents are is when you tell me who the agents are.

Allison: And I think that sometimes, that tends to help ground at least conversations to about flowery language where it’s a slightly different point, but I can often say, “What you’re doing here is actually obscuring for me who is acting and what they’re doing,” and the most important thing that I need to know is who’s acting and why they’re acting and why it matters. And so, I found that piece actually really hit students in a way, it’s like I never thought about it before, I never thought about why it mattered before, and I found that to be really helpful.

Heather: We both teach in political science and I think that this is particularly true in politics. Instead of the something must be done, well, what needs to be done and who needs to do it, right? And in politics, I think that’s a pressing question in ways that it may be less pressing in other’s field of study.

Rebecca: I find that one of the comments that I read a lot for design students is like you haven’t said anything actually. “There’s only one sentence here that says anything and the rest can go.”

Heather: I do spend a fair bit of time talking about my own writing practice actually in class where when they’re working on the first drafts of their paper, I will tell the story of my first published article, I was like all done and, “Oh, yay, I’m about to send it out,” and then, I realized that the word count was 4,000 words less than the words that I had, so I needed to cut 4,000 words from my manuscript in order to send it in.

Allison: Oops.

Heather: Exactly. Geez, it was the first ignorance. And then, I tell them I got rid of all of the adjectives and all of the adverbs and I cut several paragraphs/pages in total and it made it better. You read the draft that I thought was finished and the draft that was submitted and the second one is way better because I had to be economical with my language, I had to be really clear, I had to be direct, I had to say what I wanted to say and move on as opposed of lingering, loving over the words because they are so pretty. Which is what we as people who write as a part of our job eventually realize that, but they haven’t had that drilled into them in the same way and like, “That’s my job.”

Allison: Yeah.

Heather: It’s like, “You kill your word babies.”

Allison: I definitely am a fan of showing students my writing process. So, for instance, when I teach the annotated bibliography, Heather, you may not know this, I actually showed the part of the annotated bibliography I sent you for the Bletchley Circle paper.

Heather: Oh, my God, that’s awesome.

Allison: When I was in charge of doing the lit review for a piece that we co-authored, what I showed to students on how to do a lit review is like, “So here’s the thing I sent to my co-writer. This is when I was doing work with someone else, this is how you do it,” so I showed that. I’ve actually taken to showing my annotated readings in class.

Heather: Me too.

Allison: So, in classes where I want students to annotate, I actually just put my work up on the dot cam, instead of doing like, “Everybody, to page 57,” and then, I just can only see my book. I want them to see that part of writing is also the annotating. So, getting as transparent in some ways about my process as possible I have increasingly done.

Heather: One of the first pieces I assigned in my intro class, it’s a four-page piece. I think his name is Lee Braver. It’s in the journal or the teaching journal of the American Philosophical Association or something, that’s how I mark up texts, right, how I mark up philosophy texts. And so, part of it is just getting them to pay attention to how they’re reading, and in many ways, that gets them to pay more attention to how they’re writing and to how … when they read a text, that they leave thinking, “Oh, I understood that,” it’s usually because the writing is really clear and you want everyone to leave, to finish reading your paper in the same way and have that sense of like, “Oh, I know how it was argued.” And if they don’t have that, then, it’s your job to actually fix it. In the same way that we can read authors and say, “Gosh, I wish Thomas Hobbes [used to 00:30:44] do our words.” But he’s dead, you’re not. You can do better.

Rebecca: What are some tips that you have for faculty who are running their … assignments for the semester or getting ready to write ones for the fall?

Heather: Right. Yeah. We’re definitely working …

Allison: Yeah.

Heather: … on our fall prompts. I think it’s really helpful to have other people look at them. I actually think it’s really helpful to have people map in your field and not even in your sub field, look at them, so I will occasionally ask my partner to read prompts and she knows how to do what I do. I’ve definitely sent assignments to Allison to just be, “Does this makes sense?” “Do you understand what I’m asking?”

Heather: I’ve had former students read prompts as well to see if it’s clear what I’m asking them to do. I think a huge part of it is time, like not writing it right before you hand it out, and then, getting other people’s eyes on it who you …

Allison: Yeah. So definitely, yeah, Heather and I send prompts back and forth before the semester starts. I’m also a huge fan of having all of my assignments done before the semester starts. I have everything loaded in Blackboard in the assignments, every assignment is in before the semester begins. And that helps me know, partially, it’s for me with planning a syllabus. If this is what I expect students to be able to do, where do I have to be. What do they actually have to have in order to do this assignment. So, for me, it’s just part of the planning process.

Allison: I also increasingly have a sort of stable rotation of assignments that I like, that I figured out packages for, and I, particularly in the advanced classes where it’s that more sort of open, I want them to be able to do what they want to do. I think figuring out assignment structures that get refined overtime and work well, and then, if they’re open enough, you can reuse them pretty frequently. And the thing I like about that a lot is that, then, the students start interpreting it for one another. It helps them become teachers for each other.

Allison: So, for instance, with those critical analysis assignments, occasionally, when it’s students that I’ve had for the first time, they’ll ask a question. I’ll try to answer it and then, another student will raise their hands and be like, “Dr. Rank, I got it,” and they’ll be like, “So, the thing she wants from you is this,” I’m like, “Great. Thank you for that.”

Heather: I’ve also have taken to, as I’m doing the grading, particularly at this point, for my intro class, I give them two or three options for which topic they want to answer and I will switch generally one of those topics each time because I’ve realized that it’s not actually asking something that’s important for them to think about for the course or it’s really poorly phrase or it’s not directing them to actually answer what I want. And so, if you’ve got something that’s worked relatively well, tweaking it as you’re grading it, you get your first five papers, and you’re like, “Oh, nobody answered the … I thought they were …” it may be me. Like maybe it’s not my students that are … maybe I misstated what I actually wanted.

Heather: And so, I will, as I am finishing grading something, if I realized that I wrote it wrong or that I wrote it unclearly, I immediately go in and fix it because I know I won’t actually remember the next time I use this that I did it badly.

Rebecca: Heather, I’ve also found that to be a really good procrastination technique during finals week.

Allison: Oh, my God. Oh, yeah.

Rebecca: So during finals week, I do so much planning for the following semester.

Allison: Oh, that’s when many assignments get written to go in Blackboard for the next semester, let’s be honest.

Heather: That’s true.

John: Going back to the thing about having a portfolio of assignments that you can rotate in, how do you deal with things like Chegg and Course Hero and other sites where students upload those materials?

Allison: Sure. I think because or what I’m talking about, the prompts are so broad that honestly, if students have my assignment for how to write an annotated bibliography ahead of time, bully for you. I’m really excited that you’re reading this annotated bibliography guideline …

John: Well, what I’m more concerned is …

Allison: … prior to being in my class.

John: … for the final projects and so forth, what prevents students from submitting mildly revised version of something that someone submitted two or three years ago?

Allison: I think that becomes about how often you teach the same class and using the same package for the same class. So, I’d say I have probably three packages of assignments that are scaffolded, that work for different types of classes. And I tend to not use the same package the second time I teach a class. So, you’re getting a good chunk of time between assignments, and again, I don’t teach large enough classes where I’m super concerned about not noticing, if that makes sense. Because they have to give me a research, for most of these, they require what your research topic going to be and then, I’m going to give you some feedback, and then, you’re going to turn in an annotated bibliography and I’m going to give you some feedback.

Allison: So, to some degree, if those are all coming back and you’re pulling them out of something like Chegg, I feel relatively-confident that I’d be able to tell.

John: You’ll recognize it.

Allison: Yeah.

John: Okay.

Heather: And I also, even for the classes that I recycle topics for regularly, I will often realize between iteration two and three that I asked for three sources from the course and actually, that was the un-doable and so, I lessen it to two. And I actually caught somebody who used the three sources prompt for the two sources assignment that I had given them, and then, I switched the readings out as well. And so, previously, it had been the full book and this time, it was an excerpt, and I don’t think you read all of [Espinoza 00:35:47] to write this paper.

Heather: And so, those minor adjustments that you’re making to your syllabus, it’s relatively easy to catch, and I’m not teaching 250 students a semester. I have maybe 50 or 60 or 70, and so, it’s relatively easy for me to catch the minor variations that I’ve made in my assignments or that I’ve made in the syllabus that they might not think I’ll catch.

Rebecca: What response have you gotten from students about your assignments in the way that your assignments are structured?

Allison: I’m not sure if this is about how the assignments are structured, though in the advanced classes, it might be. I get a lot of feedback from students that my classes are where they get the most feedback on their writing. That they never get as much feedback on writing as they get from my assignments. And I think that is partially because I think … I’m sure this is true for Heather too. If you’re a professor who cares a lot about writing, you give more feedback on writing. But I think it’s also because of that, so many of my assignments are staged. That I feel a real obligation to give a lot of feedback, and then, to give do overs in some ways, right, where it’s just the lower stake stuff first, and then, you can fix it.

Allison: And so, I hear a lot from students about appreciating the amount of feedback. I also hear a lot about appreciating the variable points that I assign for certain assignments. So, I have assignments that are structured so that if you improve more than more than one-letter grade between the two assignments, the point value of the second one goes up and the point value of the first one goes down, and students also talk about really appreciating that.

Heather: I certainly have students who say in my evaluations and I also ask them to do a final portfolio that reflects on their learning as a writer, what are your greatest strengths as a writer, what are your greatest weaknesses, or I don’t say that, I say what are your areas to work on improvement. And so, I ask them at multiple points throughout the semester to do that kind of metacognitive reflective stuff because I actually think it makes them better learners, which means they become better writers. And so, my experience as an undergrad was I got a lot of papers that had a letter grade and like the occasional “Good” or “What?” in the comments, and that was pretty much it. I had no idea what I needed to do to get an A and I wanted to get an A.

Heather: And no one actually took the time to tell me the steps that I needed to take and so, I try really hard to say, “Here are the four things you did really well on this paper,” like, “Oh my goodness, you used those sources really well. Great. Clear possible interpretations of the authors. You have a beautiful writing voice. Your citations were perfect,” and then, follow up with this sort of areas of growth and improvement.” And I also end up always being a cheerleader that I’m like, “You have one more of this,” “You can totally crush it.” And so, students, even if they get a grade that they’re not particularly excited about, I am on their side. I want them to succeed and they know that and they also know that because I didn’t just give them a letter grade.

Heather: The drawback, of course, is this is incredibly time-consuming, and I’m not sure how sustainable it is and I hope it is because I really care about it and it’s one of the places that I find the most satisfaction when I’ve had a student in two or three classes and I look at their first paper that they ever wrote for me and I look at the last paper that they wrote for me and there’s much difference, and I value that so much. But it also just takes so much time, and that’s why I rubrics, obviously. As I’m moving more towards rubrics that have less space for me to write, that becomes a little more feasible.

Allison: I feel like one of the things that I really like about and keep in my brain from the paper that we wrote is to always give that sort of a big question that lets students …

Heather: Yeah, yeah, it’s true.

Allison: … the difference between a prompt and sub prompts. So the prompt is the question that you could write a dissertation on, and then, the sub prompts are the space where you tell students, “To get a good grade on this paper, I’m going to need you to do the following three things. I need you to summarize the framer’s argument or justifications for x component of the constitution. I need you to analyze the differences between the interpretation of the constitution when it was put in place by the framers and the interpretation of it in the wake of the new deal. And I need you to interpret changes and who is allowed to be a citizen or considered to be a citizen in the United States.

Allison: So, there’s a really big question at the top that you could write a dissertation on and then, there are these cues that help students understand what are the important parts of answering that question. Because I often think that’s where students have a hard time distinguishing. They could give you lots of answers for the big question but those of us who are in the field would be like, “That is the least important thing you could say,” like, “That’s the least relevant way you can answer that question,” but it’s an answer to the question. And so, it’s maybe not fair to hold them accountable for that.

Allison: And so, giving the sub prompts helps cue them to really pay attention to the particular things that matter in a way that they might not have before. And then, again, having that ahead of time helps me, as the professor, know what I need to make sure they’re hitting in class. If they’re not bringing those things up on their own, I need to make sure I do it with them so that when the paper comes out, it’s not a surprise.

Rebecca: Sounds like there’s focus on scope and a focus …

Allison: Yes.

Rebecca: … on values.

Heather: Yeah. I think that’s absolutely right. That is one of the things that I continue to do when I’m writing assignments is that … The paper that just came in from my intro class was why is political obligation important for a community. That’s a huge topic and then, I’m like, “According to Socrates, you need to tell me what Socrates says and you need to tell me what Hobbes says and then, you need to make them talk to each other and then, you need to make an argument for why one is better than the other.” And that’s what those sub prompts do that I think is really helpful.

John: I guess our next question is for each of you, what are you going to do next?

Allison: Heather.

Heather: I will take that one. I’m really interested in writing a piece on what analysis is because I feel like we tell them all the time more analysis but we don’t actually clarify what we mean when we say analysis, and I don’t think they have any idea of what they mean when they say analysis. And so, I’ve started including an appendix in my ridiculously-wrong syllabus that is like what is analysis. At some point, I just need to write that up because I think most of us struggle with communicating to our students what we mean when we say that word, and I think being a little more clear about what we mean would actually help them learn to do it better.

Allison: In terms of teaching, the project that I am hoping to work on right now is something that I started maybe two years ago after I went to the Digital Humanities Summer Institute on using annotate with students as a way to help them with the group annotation. I’m trying to work through those annotating skills moving towards better writing. And so, that’s for my American politics classes that I am hoping to get better at. I get frustrated by the tech quickly, and then, drop it.

Allison: So, that’s one of the projects I’m hoping to work on this summer and I’m hoping to do some comparisons in different classes with how students do with group annotations versus annotating on their own.

Heather: Can you explain that? When you say group annotations, they’re all reading the same PDF and then, marking on it?

Allison: Yeah. So, it’s essentially, you load the PDF online and then, you can assign small groups of students to all work in the same version of the PDF.

Heather: Interesting.

Allison: And so, they actually can go through and put comments on each other’s annotations and say, “You could find someone else’s interpretation and let them either deepen it, disagree with it, link it to some other part of the text where they can start flagging for each other and having a conversation that is deeply in a relatively small section of a text.” Heather, I’m thinking about this for American political thought, an African-American political thought.

Heather: That’s wonderful. I love that idea.

Allison: Where it’s like beyond, I get some value out of collecting their annotations which I also do in American political thought where I show them how to annotate. I give them that same piece from Braver, and then, for the first couple of weeks, I actually collect their annotated readings and hand them back, and then, I’d like to start trying this group annotation as a way for them to start thinking of reading and working through texts is more of a collective exercise in conversation.

John: What was the software you use for that?

Allison: I believe it’s called Annotate.

Heather: Is it iAnnotate?

John: iAnnotate is an iOS app or an Android app, but is it-

Allison: It’s not an app. It’s a…

John: Web tool?

Allison: … a web tool.

John: Might be hypothesis.

Allison: Maybe it’s that.

John: I know a lot of people use that for …

Allison: Yeah.

John: We’ll check on that.

Allison: Yes.

John: We’ll add that to the notes.

Allison: Yes. You should. I found it very interesting to use and then, the one time I tried teaching it, students had all sorts of questions and I basically was at the front of the room like, “I don’t know,” and so, then, I scrapped it and just need some amount of time to go back in and play with it and anticipate more of what the questions would be.

Rebecca: Well, what you did was gather the questions.

Allison: Yes. Let’s …

Heather: That’s right.

Allison: … call it an …

John: A research exhibition.

Allison: Yes. Yes. Exactly.

Heather: Because it turns our much of teaching is not being successful.

Allison: Right.

Heather: Trying things that didn’t work very well, you’ll do it differently next time.

Allison: Yeah.

Rebecca: It’s an iterative process.

Heather: It’s exactly right. Exactly right.

Rebecca: Well, thank you so much for the insights that both of you have provided. I think that gives a lot of faculty food for thought.

Heather: Thanks so much for having me. I’m honored to be a part of the SUNY Oswego crew.

Allison: Yes, I was excited to be back.

John: If you’ve enjoyed this podcast, please subscribe and you may review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.