429. An Educator’s Guide to ADHD

Each student brings individual strengths and challenges to our classroom communities. In this episode, Karen Costa joins us to discuss ways to help students with ADHD thrive.

Karen is a faculty development facilitator specializing in online pedagogy, trauma-aware teaching, and supporting ADHD learners. Karen holds graduate degrees and certificates in education and education leadership; trauma and resilience; trauma-informed organizations; and neuroscience, learning, and online instruction. She is the author of 99 Tips for Creating Simple and Sustainable Educational Videos, and has served as a facilitator for the Online Learning Consortium, the Online Learning Toolkit, and Lumen Learning. Her most recent book, An Educator’s Guide to ADHD, has just been released by Johns Hopkins Press. Through her business, 100 Faculty, Karen offers supportive, fun, and engaging faculty support and development to faculty from all over the world.

Show Notes

  • Costa, K. (2023). 99 tips for creating simple and sustainable educational videos: A guide for online teachers and flipped classes. Taylor & Francis.
  • Online Learning Consortium
  • Lumen Learning
  • Costa, K. (2026). An Educator’s Guide to ADHD. Johns Hopkins University Press.
  • 100 Faculty
  • Clear, J., & Clear, J. (2018). Atomic habits: Tiny changes, remarkable results. Penguin Publishing Group.
  • Costa, Karen (2022). Systems Aren’t Scary. Medium. October 31.
  • Hallowell, E. M., & Ratey, J. J. (2022). ADHD 2.0: New science and essential strategies for thriving with distraction–from childhood through adulthood. Ballantine Books.
  • Dr. Ned Hallowell’s TikTok channel
  • Fehmi, L., & Robbins, J. (2008). The open-focus brain: harnessing the power of attention to heal mind and body. Shambhala Publications.

Transcript

John: Each student brings individual strengths and challenges to our classroom communities. In this episode, we explore ways to help students with ADHD thrive.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Karen Costa. Karen is a faculty development facilitator specializing in online pedagogy, trauma-aware teaching, and supporting ADHD learners. Karen holds graduate degrees and certificates in education and education leadership; trauma and resilience; trauma-informed organizations; and neuroscience, learning, and online instruction. She is the author of 99 Tips for Creating Simple and Sustainable Educational Videos, and has served as a facilitator for the Online Learning Consortium, the Online Learning Toolkit, and Lumen Learning. Her most recent book, An Educator’s Guide to ADHD, has just been released by Johns Hopkins Press. Through her business, 100 Faculty, Karen offers supportive, fun, and engaging faculty support and development to faculty from all over the world. Welcome back, Karen.

Karen: Thank you. I’ve been here before a couple times. It’s been a while, and I just want to say I’m so impressed with your continued work, that you are able to put these incredibly helpful podcasts out into the world. I think the first time we talked was around COVID lockdowns, about trauma, and you’re still doing this, and I find that so impressive. Thank you for all that you do.

John: In fact, your first podcast with us during the pandemic was our most popular podcast ever, and it remains so, it’s still number one on our hit list of the most popular podcasts.

Karen: Well, that wasn’t me. That was because people were trapped at home and desperate for some kind of simulation. So I can’t take credit for that.

John: You provided some help that they desperately needed during that time.

Karen: I’m glad you said that. I do get nervous about these things, but I think you all are my first sort of podcast stop. I’m talking about this book, and it’s a little different, because the book’s not just about helping students and helping teachers, but it’s about me and being somebody with ADHD and late diagnosed and all that comes with that, and I speak pretty openly about it, but there is an element of vulnerability to that. So I gave myself a little extra TLC this morning, reminding myself that it’s normal to feel that vulnerability. I’m really glad to be having this first initial conversation with the book with you two who are so kind and welcoming and making me feel right at home.

John: And speaking of TLC, do you have tea today?

Karen: I don’t think I’ve ever had tea when I’m with y’all.

John: You’re one of many.

Karen: I don’t pretend, and I like to think I’m a truth teller. So I’m drinking a nice, crisp, cool, Polar lemon seltzer, because when I do podcast interviews or talks, I do tend to get warm, and there’s a non-zero chance these days that that could lead to a hot flash. So there’s no tea, and I’m keeping things cool. So to our listeners who also need to keep things cool, you are seen and supported.

Rebecca: You can have iced tea.

Karen: Iced tea, yeah, I have to limit, while this is another ADHD story for another day, but yeah, I got to be careful about the caffeine. There’s probably a decaf. I’ll explore it.

Rebecca: Maybe next time. Today I have a Kenilworth pekoe tea.

John: Very nice. Was this one you picked up in England too.

Rebecca: This is from Scotland.

Karen: You went to the UK recently.

Rebecca: It was last summer, but I stocked up.

Karen: Oh, I’m sure. I lived in London in college for study abroad, so I never made it to Scotland. That’s on my list. I’m jealous.

John: And I just have a mint tea, which I picked up in the local Wegmans.

Rebecca: Still important. All right, now that we’re all hydrated, we invited you here today to discuss An Educators Guide to ADHD. Can you tell us a little bit about the origin story of this book?

Karen: I always find that an interesting question, because we could go pretty far back, generationally speaking. There’s some stories there, but we don’t have that much time today, so [LAUGHTER]I’ll come into the nearer future. A few years ago, I read a book called Atomic Habits by the author, James Clear. I love reading nonfiction. I’m usually reading, because I am ADHD. The way I read nonfiction, I usually read four or five at the same time and sort of cycle through them, because that’s how my brain works. I really enjoy self- development type books. I know how to take what I need from them and leave the rest, because they often have things I don’t agree with and things that aren’t helpful, but I can usually take a nugget or two, and the message I took from Atomic Habits was to focus on systems more than habits, which is something I had already been doing, but it kind of named that for me in a new way. I got excited about it, and I talked about it a little on social media, and I got some, not a lot of, pushback, but some of the pushback was of the don’t do that, don’t read books like that. You’d be better off taking a nap and resting, sort of like people who don’t really support or like that, quote, unquote, business bros, type of advice. And I found that sort of pushback really concerning, because systems for people with ADHD are really life saving, sort of like assistive devices. And so I wrote an article about it called “Systems Aren’t Scary,” and I think it came out on Halloween, which is we’re recording this the day before Halloween. That’s why I called it “Systems Aren’t Scary” because I’m so clever. And Jim Lang and Derek Krissoff, who were at the time, at West Virginia University WVU press. I was talking to them about writing another book, and they saw that article, and they said, “Oh, can you write this book instead?” And I said, “Okay, sure.” Long story short, a lot went down at WVU a few years ago, and Jim and Derek moved on, and I moved on, and was really happy to end up through the leadership of Greg Britton at Johns Hopkins University Press. The book is coming out. At the time of this recording, it’s October, so I think you’ll be hearing it, right around comes out January, 20, 2026 it’s been a long and winding road, changing publishers like that. For somebody with ADHD who struggled with finishing things, it was interesting. I’m proud of myself that I stuck with it. Writing a book is an incredibly long process, slow process. I like to do things like immediately and quick. So book writing is sort of out of my comfort zones, but I really want to have this in the hands of people. I want the generations coming behind me to have it better than I did, and I really want people to understand ADHD for what it is, and to make the world a better place for my fellow ADHDers, so I’m glad that I stuck with it and got it out into the world.

John: You begin the book with an analogy between ADHD and living in a house that’s open to the elements, in terms of both the advantages and the disadvantages. Could you share this analogy with our listeners? Because I thought it was a great introduction to your book.

Karen: Oh, thank you. I changed that introduction 17 times. It’s lovely to hear that. It’s such a funny thing. Book writing is such a solitary process. So just starting the stage where I get to hear from learners who’ve read the book is very exciting. So give me a minute here to be like, “Oh, that’s so cool, John read my introduction.” I like analogies. The analogy is, imagine that you’re living in a house with the windows thrown open, no screens. You are just sort of at one with the elements. You have a certain layer of protection, but you can hear birdsong. Maybe birds even fly into your house. Maybe the cute, adorable forest animals come into your house as well. You sort of got like a Snow White vibe. Your neighbor walks by the house. You can call out to her while she’s walking her dog, and say, “Come over and have a chat. Have a coffee.” You can pass her coffee through the window. We can imagine sort of this combination of those really lovely positive aspects, but also you’re going to have the not cute forest animals and the mosquitoes in your house. It’s probably going to be noisy. You’re much more exposed to the elements. So it’s this combination of strengths and challenges. We can imagine the neurotypical house as being more similar to a house with the windows closed, so you have certain protection from those elements. There is sort of a more fixed or closed or rigid nature to it. I’m not using those terms as positive or negative, just trying to describe that experience. And there’s positives to that, right? It keeps out the pests. However, you’re not going to get to hear the bird song either. There’s pros and cons. Another analogy, I don’t think this is in the book, but I’ll share with people that I often have been thinking about lately, is if you want to cook spaghetti, you use a pot, and that’s where you boil the water and you cook the pasta in the pot, and then you want to drain the water out of the pot. So you use a colander or a drainer. And one of the ways we can think about ADHD as like the colander or drainer, and we might think of the neurotypical person more like the pot. You wouldn’t say, like, oh, the colander is better than the pot, or the pot’s better than the calendar. They’re different. They have different benefits and different drawbacks. You’re not going to cook the pasta in the colander. You’re not going to drain the pasta in the pot. So there’s lots of metaphors that we can think of ADHD, but this more sort of open fluidity. And a word I’m using a lot lately is porousness, because I think that word has less, sort of like goodness or badness attached to it. It’s kind of more like it’s just the nature of the item. So thinking about the ADHD brain and neurotype as more porous, which comes with challenges, but it also comes with strengths, so that more fluid or porous brain or neurotype or experience has a lot, a lot of benefits, which I talk about in the book, and it has a lot of challenges. And that’s where we can meet folks with ADHD.

Rebecca: A growing share of students are diagnosed with ADHD. Can you talk a little bit more about whether or not this is because more people are being tested?

Karen: When I wrote the book, I didn’t expect ADHD and autism to be in the news, as they have been recently this past year, in quite that way. There’s a lot of conversation happening on this, a lot of interest, and that boggles the mind. We don’t have any evidence, despite what you may have read or seen on the news, we don’t have any evidence that ADHD is growing. It’s kind of presented as like this monster that’s out of control and we have to contain it. That’s not what’s actually happening. We are seeing things like a decrease in stigma. A lot of that is due to social media, and a lot of that is due to specifically conversations that happened during the pandemic where a lot more folks sort of came online and came onto social media and were speaking more openly about some of those challenges. We also know that, particularly later in life diagnosed folks, crises or massive changes will often precede a diagnosis. So a lot of the people who [LAUGHTER] were in a structured work environment and then got sent home with all their stuff one day, [LAUGHTER] and had to live in a very unstructured environment that is often cited as one of the reasons a lot of people realized something’s going on here, because that external structure of the workplace went away. So, that didn’t create the ADHD, It revealed it. We have no data, and there’s no evidence that this is something that is growing because of… I don’t even know. I try not to listen to these people. I don’t want to give them too much attention. But there’s no evidence that it’s growing. It is likely due to an increase in awareness, a reduction in stigma. Social media is motivating people to realize things about themselves, and many folks have been saying, “I want to get tested,” or folks are self diagnosing. So we’re seeing sort of a rise in awareness about ADHD, which I think is a really good thing, because I talk about diagnosis as a map in the book, and whether we can talk more about this formal clinical diagnosis or self diagnosis, diagnosis can be a map, and it typically can be a map to support ,whether that’s through medical treatment, through lifestyle design, or to community. And for me, that was one of the biggest pieces of the puzzle, was finding people like me who I can talk to about this experience and know that I’m not alone in that. So we see that, when done right and supported, diagnosis can lead to a massive reduction in shame, which is one of the biggest challenges for people with ADHD. Diagnosis is increasing more because of awareness and access, and generally speaking, it can be a really good thing because it connects ADHD or their communities.

Rebecca: Folks feel less stigmatized who have ADHD is becoming evident in university environments because people are speaking out more openly about such a diagnosis and also asking for supports and having really healthy conversations. I know that I’ve had some of those conversations with students in ways that I wasn’t having previously, because the students brought that conversation to the table, which I think is really great.

Karen: I think that’s great, and two things can be true. So I think we are seeing a decrease in stigma, and I think like [LAUGHTER] we were starting from a really, really bad place. So like, any improvement is good, but I will tell you, I get messages from people all the time, private messages, like through social media or email, and it’s from people who are working in higher ed who either are ADHD or suspect they’re ADHD, and they’re coming to me privately to ask for advice because they don’t feel they can speak about it publicly. I talk about it because I try to use the privileges that I have to, again, make things better for people who don’t have the privileges and protections. I like to be honest, the biggest reason I’m able to talk about this is because my husband has a stable job that provides me access to quality healthcare. As an adjunct. If I didn’t have that access, would I be speaking about this publicly? I don’t know, because I can’t live without… I don’t really think anybody can, but I have chronic health issues, and I need a lot of support with my various diagnoses, so if I didn’t have that, I wouldn’t be able to do this. Yes, I think two things are true, and we’ll never be bored, because we have so much work to do and so many areas of opportunity. So we won’t be bored, is the good news.

Rebecca: I really appreciate you also just addressing and drawing attention to the differences of experiences often that students might have from faculty and staff too. This is true around accommodations and all kinds of things related to disability.

Karen: I want to speak on the accommodations piece. That was one of the most intense parts of the book. There’s a page in the book where I write about the process to get a diagnosis. Because right now in higher education, our accommodations model, in order to access learning through formal accommodations, students have to come in with a formal clinical diagnosis. And I forget exactly how many steps I wrote out in that process, but it was like 18 steps. That made me feel pretty sick about that, that that’s the model that we’re using, and obviously a lot of folks are designing around that, through Universal Design for Learning and focusing more on accessibility. That’s how I teach. I know so many folks listening to this teach that way. But the idea that the system that we, who claim to be some of the brightest people in the world, have developed has 18 steps in between students and access to learning is preposterous and infuriating. So again, though, the good news is no boredom, because we have plenty of work to do.

John: And your book is a nice, important part of that. And the issue you just mentioned results in a lot of inequity, because those steps take time, as well as some diagnosis that’s not free, typically. The people who are more likely to get diagnosed are people who do have the privilege of good health insurance and more free time and other things. So the impact is quite inequitable on students.

Karen: Absolutely, it’s not just the fact that you are asking somebody who has executive function challenges to go through an 18-step process, which is again preposterous. It’s that even if the person with ADHD can get through that, we know that there are massive inequities around race, class, and gender in terms of access to a formal clinical diagnosis. Girls and women are massively under diagnosed. I was one of those girls who slipped through the cracks until I was diagnosed at the age of 40. We know that black and brown kids are less likely to be diagnosed. They are more likely to be misdiagnosed, often with personality disorders, or they are funneled into sort of like the punitive discipline process, instead of getting access to their actual diagnosis that can lead them to community and to support. You’ve got this incredibly inequitable system that’s just people are getting misdiagnosed or not diagnosed at all. You get into higher education, and you say, in order to access what you need to learn, there’s all these barriers in place, and a formal diagnosis can cost upwards of $3,000 if your insurance doesn’t cover it. In addition, the most recent diagnostic appointment I had was eight hours long. I had to find somebody who could do the work. I had to drive into Boston, which is, as the crow flies from where I live, it’s like 30 minutes, but Boston traffic, it’s like two hours. So eight hours out of work and life, and go in and making the appointment. I can go on and on here. So just the casualness of how people talk about like, “Oh, you just need a diagnosis,” is just so wrong. It’s so, so wrong, and we’ve got to do better.

Rebecca: You talked a little bit already today in our conversation about how diagnosis can provide a little bit of a roadmap. Can you expand upon that a little bit, maybe from your own experience and how that provided you with a roadmap?

Karen: I have had the experience my whole life of feeling like an alien. This is actually a pretty common thing among folks who identify as neurodivergent. There’s some writing on this, which I learned about after I was diagnosed. And I was like, “Oh, that’s cool.” This feeling of wrongness, of this deep shame, of like, why can I do these things but not these other things? One of the things I talk about in the book that’s really challenging for people with ADHD is personal care and dental hygiene. This is the vulnerability I was talking about earlier. One of the things I struggle is remembering to brush my teeth, and not just remembering to brush my teeth, but remembering if I brushed my teeth. So if I leave the house, which I don’t too often, because I work from home, but anytime I leave the house, I don’t know if I brushed my teeth. I can’t remember. Speaking back to James Clear, I have a system that I’ve learned to create, and if my husband’s home because he works from home, I’ll text him and I’ll ask him, can you check my toothbrush to see if it’s wet? And he knows how to help me, so he’ll check. And most of the time, he says “yes,” but sometimes he says “no,” and I drive with a toothbrush and toothpaste in my car and a bottle of water. If I didn’t know that I had ADHD, that’s a really, sort of, like, scary and demoralizing and shameful thing of like, what’s wrong with me? Why can’t I figure this out? Right? Like, I just wrote a book, I have a master’s degree. I speak publicly, and most of the time I think I know what I’m talking about, but I can’t remember to brush my teeth. And that’s one of like, a million examples. When you get the diagnosis and you learn to understand your brain, and I understand now why I don’t remember to brush my teeth. And I understand how to design a system, and I understand that the most important part of that system is if I remembered or if I didn’t. I say to myself, “Karen, I’m so proud of you. I’m so proud that you checked with your husband and now you have your toothbrush in the car. I’m so proud of you,” or if I did remember I say “I’m so proud of you that you were…” like either way to learn how to speak kindly to myself. It was just so confusing before when I didn’t know what was going on. And those skills have been learned through therapy and working with coach and doing tons of study on ADHD and things like that. There’s work involved in that, but yeah, I think that shame reduction piece is so powerful, and you find people like yourself, like one of the books I talk about most in my book is the book ADHD 2.0 by Ned Hallowell and John Ratey, and they’re both medical doctors, but they both also have ADHD. And Dr. Ned, when I was first diagnosed, seeing him say… he would do these little TikToks about what people with ADHD, and then he would say, people like me. And when he said people like me, this doctor who has written multiple books, little pieces of me would start to heal. It was the “people like me,” not even the content that he was talking about. It was when he said, “people like me.” And that is, for me, the beauty of what diagnosis, whether clinical or self, can do, is it can help you find people like you.

John: So you received some really nice support from those videos. What can faculty do to better support students who do have ADHD, and you mentioned in the book that faculty need to unlearn what they think they know about ADHD. Could you talk a little bit about some of the misconceptions that people have?

Karen: The first problem we have in front of us is the name. ADHD stands for attention deficit hyperactivity disorder, and I’ve had very lovely sort of people who are ingrained in the clinical or medical model say to me, “that’s not deficit based,” and this has happened to me, so it has the word deficit in the name disorder. Yeah, so that’s not what it is. I still use the term ADHD. I don’t use the term attention deficit hyperactivity disorder because it is obviously deficit based, but it’s just we’re learning now that it’s incorrect. ADHD is not a deficit of attention. It is now increasingly being understood as an abundance of attention. And Hallowell and Ratey are on the leading edge of this. But we’re also seeing this influx of researchers who have ADHD coming into the field, and the framing of research studies and how you approach them, obviously impacts the outcomes. I know I’ve got a lot of research folks that are listening, and when you bring folks in who’ve actually lived with this neurotype, you open up this whole world. We are learning so much more about what ADHD is and isn’t, and one of the biggest misconceptions is that it is either deficit or superpower. So I use this model of strengths-based, challenge-aware, which I think is a much more middle path. It is not fully a deficit; i t does include challenges. I don’t like when people say it’s a superpower, because my family and I are fans of superhero movies, and all the superheroes are intensely miserable, and it has ruined their lives. I talk about Tobey Maguire Spider Man meme in the book, where he’s just the most miserable human on the planet. Superheroes are not generally healthy and happy, well-balanced people. So I like to really bring people into this middle path. This is a neurotype. Another way we can think of it as a brain style. It is part of the natural, vast variety of life and ways of being in the world, and it comes with challenges, and it comes with strengths. And I go into this in the book. We are taught that there’s one right way to be, the neurotypical way of being, and that anybody who diverges from that is both weak and threatening. The reason we do that is so that we can then deny those people resources, its not rocket science. So it’s really quite a radical thing for folks to say, “Wait a minute, I’m not weak or threatening. I’m part of the vast variety and natural neurodiversity that is part of the entire world.” And if you, as an educator, can shift your mind into that, you will not only be better supporting your students, but you’re really also sort of working toward a brighter, better future where we can move beyond some of those limitations. The biggest thing is to also bring in those strengths. And I want educators to know this is like a win-win. Your ADHD students are the creative geniuses of the world, and they are little engagement fireworks that you can be igniting in your classrooms. That’s the number one thing educators tell me that they want to do in their classroom is to engage their students, and your ADHDers are just like human vacuum cleaners who love to consume information, love to learn, have so many ideas, they’re gifts and assets to our world and to our classrooms. The most important thing that you can do as an educator is to get your mindset in check about what ADHD is. Get rid of the deficit. Get rid of the idea that there’s something wrong with us. We are part of the natural variation of the many neurotypes and ways of being in the world. We have challenges. We need help, we need support, but we also come to your classrooms with so many strengths. So if you can come into your classroom with that mindset, you’re going to reduce shame, and you’re going to design inclusive learning experiences for your ADHDers.

Rebecca: Can you talk a little bit about some practices in the classroom that can help lean on some of those strengths and help address some of the challenges.

Karen: The first rule of supporting folks with ADHD is externalize everything possible. Our working memory, as ADHDers, I mentioned this word before, is very porous. An example of that is if I leave this room of my house that I’m in now, and I go into another room with the intention of putting on a different pair of socks. When I get into that other room, I will have no recollection of why I’m in there. If I need to do that, what I will do is I externalize it. As soon as I say “I need to change my socks,” I will walk into the other room and I will repeat to myself, “change shocks, change socks, change socks.” I’m making it external, because I’m not able to hold change socks in my working memory, so that when I get to the other room, I can take that action. This saying “in one ear and out the other.” is how my working memory works, [LAUGHTER] or doesn’t, for that matter. It’s porous. The thing is that, I want to say this quickly, that porosity is the reason behind our creativity. Let’s go back to the open windows. So that’s why we’re getting birdsong. That’s why we are getting to play with the squirrels and have all that fun. That flowing creative energy is Hallowell and Ratey, call it “the paradoxical pairs.” The flip side of that is we can’t hold things in our working memory. So this is what you got to do. Everything is external. Everything gets written down. What you don’t want to do is like, if you’re on site and your students are walking out of class, time is up, you yell out the door, like, “Don’t forget, I want you to read pages 72 to 77 next week,” or whatever, in one ear and out the other. The second it’s out of your mouth. It no longer exists to me. Things need to be put into emails, put into task lists. We need to be putting things in multiple modalities so that it’s something that we can see and touch. I use multiple alarms. So I use like things that I can hear, things that I can see, things that I can touch. And there’s always multiple layers. Task lists are really a lot of bang for your buck. What do you want people to do this week? And it’s got to be something I can see and touch. Couple other things I’ll mention, metacognitive, or what I talk about in the book, is meta learning. So metacognition is learning about learning. Meta learning is sort of just like a more expansive way of not being quite so brain bound. Giving students a chance. I’ll name the elephant in the room, the AI elephant in the room. I’m seeing a lot of folks who are linking metacognition to some of the pedagogical strategies we can use in the AI era, having a greater focus on process rather than product, and metacognition and meta learning is great for this. So asking students, how do you plan to do this? And then after they do it, what did you learn from what you did? It’s great for everybody, but it’s particularly great for ADHDers, because we get to explore our process and what works best for our brains, and then the reflection afterwards, we get to see if it worked, and then we can redesign as we go. That’s a much better process than you imposing what you think your ADHD students should be doing. You’re putting them in the driver’s seat of your learning. This metacognition should really be much more intensely built into all our courses, but it’s a particularly powerful strategy for ADHDers. Fun fact, somebody who got their hands… I didn’t end up having to work with this person …[LAUGHTER] on an early copy of my book, who was from the clinical field, said pretty much word for word, “ADHDers can’t do metacognition,” and there’s reams of data of how good metacognition is for ADHDers. We absolutely can do it, and it benefits us, but there are still so many people out there who have this idea of like ADHDers can’t do that. That very ableist mindset is alive and well. But read the book. There’s tons of research in there to support this. The beauty of this is you’re not trying to make them be a neurotypical student. You’re empowering them to figure out what works for them. I could keep going, but I know we don’t have all day, so I’ll stop there. Read the book for more ideas.

John: Those techniques you’ve talked about are good for everybody, and they’re parts of inclusive teaching practices, which suggests that a lot of inclusive teaching practices will work for a wide variety of students with a wide variety of backgrounds [LAUGHTER] and needs, desires, interest and motivation. This is really good advice for all students. One of the things that you’ve talked a little bit about is that deficit mindset, and one of the things you suggest is that, as you have already mentioned, the term ADHD is a misnomer. And you suggest an alternative ON, or the open neurotype. Do you think using a more neutral term like that might help shift people from the deficit model to a more accurate representation of our students?

Karen: I thought I would have some fun with it and say, If I could rename ADHD, what would I name it? And I landed on open neurotype, because ADHDers, we are often “on,” I don’t know if we have an offsetting that’s one of the best parts about us. We have what the French would call a joie de vivre, a joy in life. One of our challenges is never turning off. It can be quite exhausting. “On” worked for that reason, but it also worked in that idea of a neurotype is one of the words I use to describe these, I don’t like the word conditions, but some folks are still using the word conditions. It’s a better word than conditions. It’s not a disorder, and it’s not a condition, it’s a neurotype. It’s a brain style, and then open is talking to that fluidity, that porosity, that flow, that is the best part of our ADHD learners, and it’s also the root of many of our challenges. I don’t know how we get from attention deficit hyperactivity disorder to something like ON. I do know that if the word deficits in the name, then it’s deficit based. So again, the way I’m doing it is I’m very transparent about my language choices, and I talk in the book and I say, like, I’m going to use ADHD, I’m going to tell you what it stands for, and then I’m not going to use that term again, and that’s what I do in my work. And I hope that, again, as we have this influx of ADHD researchers coming into the field and a lot more advocacy going on, I hope that we start to lean into more inclusive terms, just as one other example in the ADHD 2.0 book, they use the term VAST, variable attention stimulus trait, that is their alternative term for ADHD. And you’ll notice variable attention is speaking to the fact that ADHD is, once again, I’ll say it, not a deficit of attention. And stimulus trait is a much more flexible way of looking at this, not as a disorder, but as a collection of traits.

John: And the use of the VAST term, too, points out that there’s a whole spectrum in which people may have their attention. It could be very narrowly focused, or it could be much more open. And we have a wide variety of people, and we should be trying to address the needs of everyone.

Karen: I talk in the book about there’s this relationship between neurodiversity and biodiversity, which I find very fascinating, because we’re seeing that biodiversity is under attack, and a big part of some of the worst climate outcomes we’re seeing are related to the loss of biodiversity. And some of the original creators of this term neurodiversity were modeling it off of biodiversity. And the idea is that if you look around the world, you see millions of ways that life expresses itself. There’s not one way to be alive, and neurodiversity is lined up with that, there’s not one way of being a human being and expressing yourself and experiencing the world. There’s a seemingly infinite number of ways. So that vast variety is why life persists. Life didn’t put all its eggs in one basket, [LAUGHTER] sort of like why life is here. It’s a good thing. There is strength in diversity. I want to say the word diversity like 100 times a day, because we’re being told we’re not allowed to say it. It’s why we are here. Diversity is strength. So whether it’s neurodiversity, racial diversity, gender diversity, diversity is strength. And I just want to say it a million times, because people don’t seem to get it. And neurodiversity, biodiversity, we need to celebrate it all.

Rebecca: I wanted to circle back to the idea of attention. One of the phrases you used earlier and you use in your book, is that, like abundance of attention, I find really delightful. But we also hear a lot from faculty of concerns about paying attention or distraction or focus, all those kinds of words. Can you talk a little bit more about this range of attention and why maybe this narrow focus on how the term is used is problematic.

Karen: This one could probably be a book in and of itself, one of the first things that people can do if they want to explore this, a fascinating exploration is look at the different words and terms we use across various languages to describe attention. So I’m studying French later in life, which is a very ADHD thing of me, and the French translation is not to pay attention, but to make or do attention, which is a very different way of looking at it. Isn’t it interesting that in America, attention is something that we pay, so there’s a little like, “Hmm, let’s be curious about what’s actually going on here.” There’s a book I talk about in my book called The Open Focus Brain, by this guy, Les Fehmi, and his argument is that this idea that attention, it should be this very like you’re zoning in on one thing, is very much tied to capitalism, and that’s how you are efficient. That’s how you’re a good worker bee. That’s how you produce the most, is by zooming in on one thing. Zooming in on one thing is also what we want to do if we are like under threat. And he argues that by really cultivating a single pointed type of attention, we are training our brains to believe that we are under threat whenever we are sort of in that super- focused type of attention. He presents this alternative called “open focus.” And anybody who’s listening to this, who’s gone through meditation, I’m like a lifelong… this is another podcast… yoga student teacher and meditation student teacher. When you go through the meditation process, you often learn to start with single pointed meditation, focusing on an anchor or your breath, and then you learn to expand out, and you learn to sit in meditation and be open to all of the experiences and all of the noises and all of the sensations. One of those isn’t better than the other, so open focus is the idea that we are able to be taking in everything that’s happening around us: the other students in the classroom, the sound of the air conditioner, the smells in the room, ADHDers tend to have that sort of more open focus that can make it more challenging for us to be in that sort of productive, efficient mode, but it offers many gifts as well. That’s why we’re so creative, and why we are able to connect seemingly disparate ideas in really interesting ways. So it’s, again, that blessing and a curse. The other thing that we will do, ADHDers, when we’re in a setting where you’re saying, I’ve got to zone in and my focus is open, is that we will do something called stimming, which is things like doodling or fidgeting or turning around in our chair, things like repetitive motions, rocking. Both ADHD and autistic folks do this. We’re not doing that because we’re distracted. Doing that helps us to focus. So that’s one of the big messages I guess I’ll leave folks with is whatever you want to call it, you don’t know when people are paying attention. Attention is very complex. It’s very personal. It’s very diverse. Stop trying to police other people’s attention. The way that I pay attention. I mean, my desk is like covered with little toys and gadgets that I use behind the scenes [LAUGHTER] in Zoom meetings to keep myself focused. If you think attention looks like sitting still and not moving, you’re going to say she’s not paying attention. I actually am. I’m deeply paying attention because I’m doing all of those other things that give me those external structures that I need to focus in on what you’re saying. So stop trying to police people’s attention. Know that open awareness and open focus is another way of being in the world, and incredibly valid and important, and we probably need a lot more of it, and trust your students to do what they need to do to be present in your classrooms.

John: A friend of mine is an autism researcher, and we’ve talked about it quite a bit in terms of the type of focus that people with autism have. As I was reading your description of ADHD, I was seeing so many connections in terms of the different types of attention, and you mentioned that there’s an emerging recognition of an overlap between ADHD and autism. Can you talk a little bit about the ways in which they’re connected, or the similarities that occur in autism and ADHD,

Karen: I can. So here’s a fun little story for the listeners. In the middle of writing this book, I got a second diagnosis. After getting late in life ADHD diagnosis, I then got a late in life autism diagnosis, so am doubly blessed. And remember the winding journey of writing the book I mentioned earlier? Didn’t see that one coming either. So yes, I can speak to this from both lived experience and the research. AuDHD is sort of like this new wild frontier that we are seeing rise up. I’m really excited for this understanding of the overlap between autism and ADHD. I think we’re going to be hearing that term, AuDHD more and more and more in the coming years, there’s a lot of unknowns. So I have seen data that up to 80% of autistic folks would also qualify for an ADHD diagnosis. At that level, [LAUGHTER] you start to wonder what’s going on. And there are some folks who are saying, “Are these like the same thing, and they’re manifesting in different folks differently.” There was a time, up until several years ago, where you couldn’t get both an autism and an ADHD diagnosis through the DSM. They said these two things cannot overlap, and now they’re saying 80% of folks with autism have ADHD. So we are learning a lot. There are days when I feel like these are the same thing, and AuDHD is like a thing in and of itself, and there are days when I can feel them as separate entities [LAUGHTER] inside of my head. And it’s hard for me to explain that, unless you’ve lived with these neurotypes, it’s a very sort of fluid situation. I don’t know if it’s a question we’ll even be able to answer through research. I think it’s more something we’re going to move through and have to experience and really listen to people’s lived experiences. But you’re going to see a lot more AuDHDers in your classes, in your meetings. The joke is in our community that if you got a late in life ADHD diagnosis that you can just chill out for a couple years and then you just go and pick up your autism diagnosis as well, because there are some folks who believe the reason that they mask each other, and that’s why folks are getting late diagnosed. One of the ways [LAUGHTER] that I’ve heard it talked about is ADHD is the gas pedal and autism is the brake. So you’re driving a car pressing down the brake and the gas pedal at the same time. It makes for an interesting ride, but there’s some evidence that they’re masking each other, no less challenging for people. They are no less in need of support, but there’s some interesting overlaps going on there. If you’re listening to this and this is something that interests you, we just need a ton more research on this, a ton more support. We need people who are diagnosed and have lived experiences leading the charge on this. But yeah, AuDHD is going to be pretty big and very interesting for us to continue to learn about.

Rebecca: So earlier, you mentioned about Universal Design for Learning and also accessible design. Can you talk a little bit about how those methodologies, frameworks, might really help all students, but ADHDers in particular?

Karen: Yes, absolutely. So. I am a student and a facilitator of UDL, a practitioner of UDL, a wonderful approach that helps us address those accommodations, limitations, fantastic. Two things can be true. My concern about UDL is that sometimes I feel like it’s used to ignore disability or ignore difference, and I’ve heard sometimes UDL practitioners say things like, “We don’t even need to talk about your disability or your ADHD in the classroom, because we have designed this accessible learning experience.” Well, those are parts of me, so UDL is not intended to wipe that away. When done well, it should be inviting that into the experience. So what I think might be helpful, I’ll give an example. For me as an educator, I feel like I’m on the right track when I hear from a student who says I have accommodations, and I look at their letter and I am able to respond to them and say, “I’m so glad you sent me this. All of this is designed into the course already for all students,” that feels like I’m on the right track. And I will often disclose, depending on the situation, and let them know that this is something I live with, and I’m proud and happy to be a role model for them, and I am there to do anything I can to support them. I am not a mind reader, so please keep me updated on anything I can do. So that’s my approach to it, like I want to notice and I want to celebrate their identities and reinforce that throughout the course. I teach, primarily online, asynchronous, so it’s woven into my discussions, into the conversations we’re having, into my grading, into the types of resources I share with students. And it’s not disability I want to eliminate, it’s the need for that letter eventually, right? Like everything is built into my course already, I feel like I’ve done the right thing when I’m able to notice and celebrate and thank that student for sharing that with me and let them know it’s baked into the course for all students. The last thing I want to say about this is one thing I don’t like about UDL is when we say, here, “We can do this for ADHD students,” and people say, “Oh, and Isn’t it lovely that it’s good for everybody.” Two things can be true. Let’s let it be just good for ADHD students, that’s good. That’s good on its own. It’s great on its own. And I recognize that we want to motivate people to see the many benefits of this. And it’s good for everybody is a lovely thing. But I also want us to say that, like, “If it was just good for ADHD students, that would be wonderful, too.” So those two things are happening at the same time for me, and I think that’s important to mention when we talk about UDL.

Rebecca: I appreciate you pointing out these complexities, that multiple things can be true at the same time. Now that comes up a lot in the accessibility work that I do as well. So we always wrap up by asking, what’s next?

Karen: This is coming out in January, so I’m really excited to get the book out into the world, and hearing how people are going to react to it and use it will be exciting for me. The other thing I’m working on now is I’m shifting a bit from climate change mitigation to adaptation in my focus, which is to say, I’m always very mindful when I talk about the climate stuff. We need folks who are working to prevent climate change, and I want people who are doing that to keep doing that. I am feeling called to focus more on adapting to some of the worst impacts that we’re seeing already happening. And I’m really feeling curious about things like the fact that we have hundreds of colleges in coastal areas that we don’t know if they will be able to stay there. So I’m starting to take some classes on things like forced migration related to climate change, because my sense is nobody in higher ed is talking about this, or very few people, which is very understandable. I feel called to do that. I want to support campuses that might need to relocate, students who might need to relocate due to climate change, and I very much want to, in particular, help support disabled folks who are part of those changes, and to recognize that our ADHDers are, I think, the creative folks that we need to be leading a lot of the changes that we will need to be making in the next few decades, in this very volatile era. So one of our greatest strengths is that creativity piece. We don’t just think in terms of what has already been proven. We design and create new worlds and new ideas. I think that’s why I’m able to do that work well, and it’s a little disconcerting for some people, but I feel like it needs to get done, and it’s something I’m feeling curious about. So leaning into that more as well.

John: Certainly an important issue to be focusing on in a country where we need far more of a focus on climate change right now. Well, thank you. I very much enjoyed reading your book. I think it would be a great book for reading groups, for any faculty member to learn more about a non-trivial share of the students they’re working with.

Karen: The number that I use in the book is 5 to 10% but I’ve had classes where up to 40% of my students disclose ADHD, so I think that 5 to 10% is the tip of the iceberg, and I also think the vast majority of our students stop outs and dropouts are probably undiagnosed ADHD. If I’ve got anybody out there who’s connected to enrollment management, I think you’ve got a huge opportunity here to help students persist and complete by considering that lens of undiagnosed ADHD and higher ed’s failure to adapt to that as a big, big opportunity.

Rebecca: Well, thanks so much for joining us. We always enjoy talking to you, Karen, and sharing your work.

Karen: I am so grateful for both of you, again, the fact that you are able to persist and keep doing this is such a testament to who you are, and so needed in our world and our very rapidly changing higher ed so thank you for doing the work that you do. Action is the antidote to despair. So everybody’s got to find their small actions or medium or big actions that they can take to help make a difference. You don’t have to save the world, you just got to do your part.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

428. The Science of Learning Meets AI

The widespread adoption of and the rapid evolution of generative AI platforms have created substantial challenges for faculty in how we assess student learning. In this episode, Lew Ludwig and Todd Zakrajsek join us to discuss a new resource they have created that is designed to help faculty use AI to efficiently support teaching practices based on the science of learning.

Lew is a Professor of Mathematics at Denison University, where he served as Director of the Center for Learning and Teaching from 2020 to 2025. Much of his recent work has focused on innovative methods for utilizing generative AI. Todd is an Adjunct Associate Research Professor at the University of North Carolina at Chapel Hill, from which he just retired a few months ago after 17 years. He is also the Director of the International Teaching Learning Cooperative and the Director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published six books (so far) in the past five years.

Show Notes

Transcript

John: The widespread adoption of and the rapid evolution of generative AI platforms have created substantial challenges for faculty in how we assess student learning. In this episode, we explore a new resource designed to help faculty use AI to efficiently support teaching practices based on the science of learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Lew Ludwig and Todd Zakrajsek. Lew is a Professor of Mathematics at Denison University, where he served as Director of the Center for Learning and Teaching from 2020 to 2025. Much of his recent work has focused on innovative methods for utilizing generative AI. Todd is an Adjunct Associate Research Professor at the University of North Carolina at Chapel Hill, from which he just retired a few months ago after 17 years. He is also the Director of the International Teaching Learning Cooperative and the Director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published six books (so far) in the past five years. Welcome, Lew, and welcome back, Todd.

Lew: Thank you.

Todd: Thank you, Rebecca,

John: Our teas today are:… Lew, are you drinking tea?

Lew: I am. I’m usually a coffee drinker, but I have a lovely Cha Atiso. This is a Vietnamese artichoke tea, which a student brought back for me from the summer, and I’ve really, really been enjoying that. So that’s my tea for today.

Rebecca: What does it taste like?

Lew: It’s got a little sweetness to it, which I like. I don’t put anything in it, but I really enjoy it. And it’s supposed to be healthy, no caffeine, all that sort of stuff.

John: And Todd, are you drinking tea?

Todd: Yeah, I went with a roasted dandelion spice tea. It says detox on the package, and I can always use a good detoxing. So I’m doing that and sipping it from my favorite cup. Mmmh.

Rebecca: That’s right, the tea for teaching mug.

Todd: I have my tea for teaching mug here. I never leave home without it. Actually, I never left it home. [LAUGHTER]

John: And Rebecca?

Rebecca: I have a Swiss Breakfast tea, John.

John: Very nice… on a chilly morning here in upstate New York. And I have a ginger peach green tea today.

Todd: Wow. Quite a variety.

Rebecca: Nice, this big variety. So we’ve invited you here today to discuss your newest book project, The Science of Learning Meets AI: A Practical Faculty Guide to Purposeful Integration, Student Engagement, and Ethical Practice. Can you tell us a little bit about the origin of this book?

Lew: Yeah, sure. I can get us started on that. So as you know, Todd and I both have backgrounds in teaching centers of faculty development. And our national conference is something called POD, Professional Organizational Development Network. And in 2024 we were at that conference, and here ChatGPT had already been out for about two years, and we were still kind of struggling with the idea of, how are faculty supposed to use this? A lot of the questions we were getting is, “Hey, how can I use AI in my classroom?” And it wasn’t until this conference was up in Chicago in 2024 where we realized we were asking the wrong question. It’s not really how to use AI in the classroom, but instead, maybe we should be thinking about, why are we using AI?

Todd: Yeah, and actually, I’ll jump in too, because Lew went right to that technical part and everything. Another thing that happened, which is why I love it when people get together, is Lew asked me if I would help out or just talk to him a little bit about some of the science of learning, because he wanted to really work on this project with bringing AI into this, explaining why it’s important to use it. And I said, “Well, that’s interesting, because I just started to outline a book that I want to do on AI, and I was going to check with you about the AI part, because I got the learning part down.” [LAUGHTER] So we just said, “What the heck, we’ll do it.” And Lew didn’t know me very well at the time, so he said, “Yes.” [LAUGHTER] Anyway, we made it through. But the big thing was that we really wanted a book that would teach faculty and help them understand the learning theory behind why the learning happens and everything else, and use AI to augment that. So this isn’t like using AI to do things, it’s using AI to help us do the things that we’ve been doing.

John: You start the book by acknowledging that faculty didn’t ask for this AI disruption or for any of the evolutions of this. Why start with this instead of with excitement about AI’s potential?

Lew: Oh, it’s called learning from one’s mistakes. So again, ChatGPT came out in November of 2022, by August of 23, I remember on my campus, we had a fairly large across the board faculty meeting about AI. It was about a half-day session. So administration was there, IT was there. And as the Teaching Center Director, I gave probably a 90-minute session on, “Hey, here’s this new technology and how we might be able to use it in our classroom.” And what happened was, just by saying, “Hey, here’s how we might be able to use it,” some folks labeled me as AI pro. So I was just pro-AI, hey, you talked to Ludwig, and he’s just going to be all in on AI. And that was almost like my scarlet letter [LAUGHTER]that I was wearing around campus to some folks. So I learned from that. So any of the workshops, any of the things I do since then, I start off with this exact phrasing, just to kind of level the playing field here, I say “You didn’t ask for this. You didn’t request an unregulated, untested, and rapidly evolving technology to suddenly upend education and nearly every other sector of society.” Now, I’ve been delivering that, like I said, quite a few workshops. I did it back in August for a group of high school teachers in Richmond, Virginia. They applauded me. [LAUGHTER] So it’s just kind of getting it out there and acknowledging it that, yes, this is a disruptive tool. We kind of have to acknowledge that grief and then kind of move on from there.

Todd: Yeah, I would say too, is really, I really thought Lew had a good handle on that part of it. I wouldn’t have put that in there, quite frankly, for the book here, and Lew was pretty insistent that it go in there. But as we started talking it through, I realized I always started any COVID workshops this way. When we were doing emergency remote teaching, if you just started talking about emergency remote teaching and didn’t acknowledge just all of the stuff that happened with COVID at the time, on the number of deaths, for instance, you can’t just jump into something and ignore it. So I think that Lew had a really good point here, and then I started pulling together what I did with that, with what he was talking about. And I think the whole concept is we started thinking within the book of, how do we do this? To recognize people are grieving, that they’re struggling, and we say it’s hard that we see you there, the frustration is real. Teaching is changing, just as it did with COVID. We don’t have a choice of that, and AI isn’t going to go away, so let’s recognize what’s going on, and recognize even some of the impact of AI, and then from there, then move on with a sense of acknowledging what’s going on.

Rebecca: It’s almost like you’re modeling some good teaching practices there. [LAUGHTER]

Todd: Kind of. Periodically it happens.

Lew: We try.

Rebecca: This adapt, create, embed, progression, the ACE model, is used throughout the book. Can you talk a little bit about what this looks like in practice for a faculty member who’s never used AI?

Lew: Yeah, so this work kind of, again, developed that starting point back in November of 24 after we had that initial meeting in Chicago and started to realize we needed some early wins for faculty. So we got a lot of faculty out there who are kind of hesitant about this new technology. And if you gave them something like a prompt or something that didn’t work, they’d be just like, “See, I told you, this AI thing, it’s a disruptive thing. So we’ve been workshopping this quite a bit early January onward, and refining the model. Originally, you ready for this? It was, instead of adapt, create, embed, it was improve, create, integrate, which, as an acronym, is icky. [LAUGHTER] We didn’t think that would land very well, so we went with ACE instead. So for example, the adapt, that’s kind of the easiest win. So what you do is, you take an existing assignment, you have that assignment that it’s a big project, but for some reason, it just doesn’t land with students. They always hand in something a little bit different than what you asked for. Well, that assignment’s begging to be TILTed, right? So Mary-Ann Winkelmes has this idea of transparency in learning and teaching, and it’s a technique that you can do for your assignment to make sure that your students understand why they’re actually doing the assignment. And it’s a kind of a three-step process. And AI is great at TILTing your assignment, and faculty members can take an existing assignment that they have. They’re experts on that assignment. AI can help them TILT that with this framework, and then they can see how that might better fit into their language for their course, and help improve what their students are doing. And the TILT framework has been really good. It kind of raises all student boats. But in particular, it does well for underrepresented and first-gen students do exceptionally well with this. So again, just kind of an early win with the adapt model.

Todd: Should mention quickly that we kind of started out and we based a lot of this on Ruben Puentedura’s model of SAMR, which is the substitution, augmentation, modification, redefinition for all of technology. If you have any new technology out there, you started with substituting it. How can we do this instead? You can augment it. And that’s a really interesting model. Didn’t quite work with what we were after. And so Lew had lots of workshops he had been doing. And so we kind of used that as a foundation. But as Lew just pointed out, the adapt, which would be similar to that original substitution, is to get you rolling. And I’m going to go ahead, Lew and take I’ll take the other two real quickly. So next is create. And essentially, it’s looking at how you can do something that you really couldn’t do before. And there’s lots of things that are out there that we could use for this one. We’ll talk a little bit more about those, probably later on, as we’re looking at some specific examples. But when you think about what you can do and couldn’t do before, one quick thing is a daughter, one time, was struggling with something, and she said, “Can you help me?: And I said, “I’m busy right now. I can help you a little bit.” 15 minutes later, I said, “I can help you now.” She said, “Oh, took care of it.” I said, “How’d you do that?” She says, “Oh, I just got on a group, a small online group, and then we solved it together.” And I said, “Who was it?” And this was years ago. And she said, “Oh, there was a student from Germany, one from South America, somebody from Nebraska, and myself.” That was never possible before. And so many things in AI do things like that. They’ll set up things we could never do before. So the “create” is, how can we do things different? And then once you’ve adapted it and created it, the next thing is, just embed it. Just make it part of the regular aspects of your course. How do you do that? And what we like to point out over and over again is that, you do this wherever you’re at. If you’re just getting started, you just adapt some things you’re doing. And that’s where Lew is really good at setting some of those things up. If you’re a little bit nervous, just take something you’ve always done, do it with AI, do a little bit better, faster, and then just get started, but don’t get overwhelmed with it.

Todd: And the other thing about doing with something you’ve already done is then you’re the expert, and you can critique what AI is doing for you. I wouldn’t go off and learn a brand new subject you learn using the AI, because I’d have no idea. But again, coming from that wheelhouse of your own expertise, don’t forget you are the expert with the content as well as your students and context. So keep that in mind whenever you’re dealing with AI.

John: And that example you use of the TILT model is a good one. I’ve given workshops to faculty about it, and a lot of people were reluctant to put in the effort to make all those changes. But when you show them how AI can do that, or better yet, if you share perhaps a prompt, or if you share a GPT or a chatbot tool that they can plug in their assignments to and their learning objectives, and it can do it for them, or at least give them a starting point on it, it makes it so much more likely that people are going to do that. So you talked a little bit about the ACE model, which is used throughout the book. But could you tell us a little bit about the organization of the book?

Lew: Yeah, sure. So the book’s kind of broken up into three parts. The first part, we call it “Creating your own Gen AI roadmap.” And again, chapter one, you didn’t ask for this, but here it is. So just kind of, again, acknowledging that situation that we find ourselves in and how to move forward with it. Then in chapter two, we start talking about what it means to think about shifting our teaching in the world of generative AI, so the idea of being transparent with our students and how we’re using it and setting up good communication with them, and then also the idea of as faculty members, as we’re embracing this, we really have to be curious and flexible as we work with this technology. We’re always pushing our students to take risks, be flexible, be curious. Well, guess what, that shoe is going to now fit on our foot as we try to move forward with this model.

Todd: So that kind of got things up and running, and then we kind of ran right into the learning aspect of it. Just to give you an idea of the book, I guess is probably the easiest way to do this is. Chapter four is on universal design. I’ve been a long time advocate of universal design, and I think AI can be now done to do things that were really challenging before. So we did dedicate an entire chapter to universal design. Then chapter five, we really thought community was important. So chapter five is about building classroom community and trust, looking at belonging, engagement, and integrity, kinds of issues. Chapter six comes into backward design, a huge aspect of teaching and learning, kind of knowing where you’re going as your goal. And then chapter seven is looking more at learning science of cognitive load, attention, spacing, interleaving, retrieval practice. And that’s the “building the foundation section.” And the last section of the book, section three, is for kind of deeper learning. We looked at some things about teaching and learning that just we know works well. So chapter eight is guiding students from novice to expert. We put together in that section zones of proximal development, scaffolding, and the gradual release of responsibility as ways of helping students kind of go into deeper aspects of learning. Chapter nine is on strategic learning. So we looked at transparency, TILT, cognitive apprenticeships, metacognition and those types of issues. And then we wrapped it up with chapter 10 of persistence and motivation. So there we pulled together the concept of self efficacy, growth mindset and self-determination theory. So the chapters have two or three strong learning principles to them, but they’re put together in ways that not only help explain the individual things, but also overall this concept, like persistence and what kinds of things go into that. And then the last thing I’ll say, since I’m on a roll right now, is overall, I think we really thought it to be helpful, repetition is just helpful for learning, right? We know that. And so we did is we built a book that has strong repetition. So chapter after chapter, we basically started with chapter four, and then we go all the way through chapter 10, and it’s set up so that there’s always an opening story or anecdote to set it up, which we’d want to do for just getting the learning focus there. Then from the model, there’s an improve activity, and then we look at the science components, and from there we might be looking at metacognition or Bloom’s taxonomy or something. So that comes into the second two parts of the model. Here we have a create and then embed, those activities are in there. And then we talk about the ACE model, and we have more activities that come into there. Then we look at risks and guardrails. And again, we’re going to acknowledge everything has risks to it. We’re not going to just pretend like it’s all great. And then we have a conclusion, pull it together, and we finished every chapter with discussion questions and reflection questions. So if you’re a faculty developer and you want to use this as a book group, for instance, we’ve got your discussion questions already built in, but for having faculty work through it, there’s reflection questions in there. The last thing I’ll mention is that we talked really extensively about how to cite this stuff, big APA kind of person, but we really wanted this readable, and so we went with a model where we put everything in as end notes, so as you read through it, you don’t get the standard citations with dates all the way through it, it just reads like a novel, almost, and I think it’s really easy to read. So you don’t have to read it cover to cover, although you should, [LAUGHTER] maybe three times. You can just go through and pick out what you like, but if you do go through, we’ve actually built it so that you start with simple kinds of examples and move to more complex ones. And Lew is masterful at putting together how to like, ask questions and do prompts and specifically with a separate font in there, we just have paste this in and add your components, and it spits out the answer. Pretty cool.

Lew: So what Todd’s explaining there is, we have a lot of different techniques. We have things called activities as well as examples. So each chapter has six activities, which are kind of page-long discussions of how you might use AI. And it’ll start off with some prompting, like he said, in that different font, but you put it in your context. We don’t do this: “Oh, a biologist would do this. Now you do it for yourself.” You actually, in the context of this thing, put in your information so you’re directly dealing with it. So we have, like I said, six of those for seven chapters. So 42 of those things. Each one could be a workshop. And then also we have smaller examples for seven chapters, we have 12 smaller examples, which is another 84 prompts. Those aren’t prompts, those are just more ideas. But Todd mentioned this idea of meta prompting. We give you a way to turn those examples into a prompt for yourself. So you’re using AI to create your prompt for you, and I think it’s called meta prompting, so that’s what he was referring to there.

Rebecca: There’s a lot written about the science of learning and Todd, obviously, you’ve written some books on the subject. How did you decide what science of learning concepts to include?

Todd: I just put a whole bunch of stuff in there. And then Lew started saying, “Nope, nope, nope.” [LAUGHTER] No, we got along great on these things. It basically boiled to Lew does tons of workshops, I do workshops, I think I’ve been at 300 campuses so far. So I looked through the different presentations I’ve given, and when things kind of hit and when they didn’t hit. Cognitive load is something a lot of people just don’t quite understand, and is a huge foundation of learning, because within cognitive load, you’ve got automaticity and schema activation and all kinds of different things that are practicing retrieval kinds of things. So I just went back to the warehouse of stuff and picked the things that worked, which is what I still think is really, really valuable about the book, is that these are things that work, and it’ll help explain them as you go. Big things like the metacognition, Bloom’s Taxonomy, cognitive load, automaticity, practice, retrieval, interleaving, the kinds of things we’ve heard. And it was hard, I will say, Rebecca, it was really hard. There’s a lot out there. I mean, obviously I got what is it a 270 page book on learning. So how do you just fold in some of that? And so, it doesn’t have everything, but I think it has the big things.

John: So you mentioned before that AI allows us to do things that we couldn’t easily do before. Could you give us an example and walk us through that in terms of how AI could be used to create learning activities that could not be done as easily in the past.

Lew: Sure, individualization of instruction. I’m at a pretty small school here at Denison, but I could imagine, wow, what if I was teaching a class of 400. So I might end my class here at my small school with, you know, a muddiest point, I get 20 responses. I look those over, what did students struggle with that day? And then next class, I can kind of address that. Imagine if you had 400 students, what could you do with that? Well, now, with AI, you can take this classroom assessment technique of muddiest moment and, on the spot, have students type their prompt into, say, a Google form or something like that, and as maybe the students are off working on some kind of project or discussion together, within two or three minutes, you could use AI to organize those muddiest points and give me the top three things that these 400 students are struggling with, something that was just was not even feasible before. So I think you know, at scale, this thing can be really useful for those types of examples. And with regards to CATs, the classroom assessment techniques, Todd’s got a book of like, 50 of these things, right? So there’s so many things that you can kind of do on the fly that AI can then help you with, kind of the managerial stuff, just the paperwork thing of keeping track of that stuff so you can respond in the moment for what your students are doing.

Todd: And Lew, I appreciate you mentioning that book, it’s interesting, got me thinking real quickly here. So that book is Classroom Assessment Techniques, which not too many years ago was like the standard. Everybody had a copy of this thing, and they did because the first edition was 1988 I believe it was, second edition was 93 though, and in 93 was right about the time we started talking about learning versus teaching, and how would you know if students are learning? And along comes Angelo and C ross, their book was out, I guess, already, and so it explained it. I mentioned this quickly because I then worked with Tom Angelo to do the updated version, the third edition, which is now out 30 years after the second edition. And the second edition was still selling, but we were removing things like “hand out papers to the students, have them respond on the muddiest point, and then take them home, but make sure you don’t ask questions that you don’t have time to respond to. Don’t ask elaborate things because it will take too much time.” Lots of stuff about CATs was about being careful not to overwhelm yourself. And if you think about that in the old days, that’s what you do. Pieces of paper were handed to, you’d read through them and everything. So we took out things like “share your results on the overhead protector,” and those are gone, and then we replace them with things like Lew was saying is, just put your students into a small think-pair-share, have them respond to this thing with a PollEverywhere or something, and then while they’re talking, then come up with the four muddiest things going on, and then teach those in the second half of class. And so, all of those classroom assessment techniques that we’ve been using for 30 years can now be made easier with AI. So I’m really excited about this.

Rebecca: You mentioned earlier about the chapter on Universal Design for Learning, and I know you’ve been a long time advocate, Todd. Can you talk a little bit about how faculty can use AI to design courses that are better for diverse learners and really have kind of access in mind, rather than retrofitting accommodations.

Todd: Yeah. So number one, I think the thing to point out here is, so I teach, this Lilly teaching conferences. They’ve been around for a long time. There’s different types of sessions that I’ll include in the program, and some of the sessions, and I talk to the presenters about this all the time, some of the sessions are just hot topics that you know are just going to go: engaging students in asynchronous classes, it’s going to fill a room. How to do active learning strategies, going to fill a room. They always have. Sometimes you have topics, though, that you put on the program because you have to talk about them. And if you don’t put them on the program, even though they’re not popular, then it doesn’t get picked up. And I remember putting sessions in the late 1990s, early 2000 about diversity that would have four people in a room, and individuals would say “there’s only four people in there.” And I’d say, “We’re still putting it on the program.” UDL was another one, anything on universal design, I can tell you I’d put in the smallest room in the conference, because it would be a great session, but it would draw like seven people. And one of the reasons it didn’t draw many people, and quite frankly, across all of higher education, people might love the concept, it’s really hard to do. If we’re going to do things like multiple means of expression, which is a key factor of universal design, it can’t just say that you can share your results in one way. What if you’re not good at putting text together, but you really have the idea down. So maybe you create a video, or maybe you have a play, or maybe you get in small group and have a discussion. Multiple means of demonstrating the information, so presentation, how do I explain this in a way that people can get it that’s always been hard, it is now being made really easy because we can go to AI with the correct prompts, and we can come up with multiple ways of expression, multiple ways of presentation. Lew was talking about having a class of 400 with the students in there. You could say to the students: “Here’s what I want to do. We’re teaching metacognition. Everybody think of something you love to do now, type into AI. I love to blank. And I want to learn more about metacognition so that I could explain it to my friend.” So I could type in I love long distance bicycling, and I want to learn metacognition. Rebecca might put in that she loves rock climbing. John might put in like he likes baking, and Lew might probably put in there he likes street fighting. [LAUGHTER] So you put those prompts in there, and then what pops out is everybody in the class gets to learn it in their own… I mean, what could be more universal than that? And so I’m just really excited about it. And we did the whole chapter, and Lew and I talked about it, and I think at the beginning I wasn’t even going to do a whole chapter, and he said, “Yeah, we need to do a whole chapter.” And so I think AI will open the gate so that people can finally do universal design in a way that’s meaningful without overwhelming them.

John: Faculty have been overwhelmed, first with COVID and now with AI, and many other things that are happening externally. How does your book address the reality that faculty don’t have enough time or energy to learn a whole new technology on top of everything else they’ve had to learn in the last several years.

Lew: Fair point. I think the thing we need to start with is the idea that we want to be careful and not this chase the technology, because if you do that’s going to make things worse, you’re going to get frustrated. What you want to do is try to use the tool to make things better. So I often talk about there’s 2, t-w-0, too many shiny objects out there. And one of the ideas with the too many shiny objects is there’s all these little prompts that people: try this, try that, try that, and that’s really frustrating for faculty, because it’s just a little bit beyond their skill set of what they want to do. It’s a biology example, and they have to bring it down into what they want to do. The other thing is, with the shiny object, is the idea of treating AI like this big shiny hammer, and then you’re running around trying to find all these curricular nails to try to beat down of how I’m going to use AI to do that, that is just such the wrong way to address AI, letting the technology lead first, you’re just going to make things worse and you’re going to stress yourself out even more. So what we want to do is we want to go back to basics. We want to think about backward design: at the end of this course, what should my students know? What should they understand? Then how am I going to assess whether they understand that? And then finally, how would I go about teaching that? And that’s where we decide whether AI might fit in or not. So again, it’s not the idea of trying to get AI in the classroom. I think as faculty members, we want to be really careful about how do we use it to be a creative assistant for us and kind of level up and improve our existing teaching practices. And the thing is, the book, all these teaching practices that we’re doing, these are well reasoned research techniques that have been laid out in chapters four through 10, backward design, ZPD, TILT, all the fun letters, UDL, all that stuff. But again, it’s trying to use that in a practical way, trying to develop your teaching skills more, and then just using AI as the tool to help you with that guidance along the way.

Todd: I wrote an article years ago. I was like, we just want to be well, that’s all we want. [LAUGHTER] So this concept of what we’re doing, if we think about what we do as faculty members out, as Lew pointed out, we’re chasing things a lot of times, and that’s just a problem. And this goes back to… remember when clickers came out, student response systems. Everybody’s like, “Oh, I want to use a student response system because my colleague’s using it,” and as faculty development directors, we are constantly trying to talk people out of using technology unless they actually needed it. Backward design. Why do you want to use it? Student response systems are always fun, because it’s like, I want to use student response systems and my friends and I would say, “Okay, so why?” “So we can gather student responses” and I’d say, “Okay, you know, you could just have them raise hands. Like, how many of you think this?” They say, “Well, yeah, but I want to know like anonymous.” I say, “Okay, have them close their eyes and raise their hands well.” “But I want to tally them.” “Alright, well have them close their eyes, raise their hands and then count them.” I mean, these things are not necessary. But then when they start to say, “Okay, but I want to know some qualitative aspects, what are they thinking and why?” Now we can use the technology. So I think that’s helpful. And so when you go back and look at what really takes time in teaching, those types of issues do, I can’t, like have everybody talk about muddiest point, because I don’t have time to read through those. Now you do, because AI can skim through those quickly for you. If you’re prep timed, you put in exhausted in there. I was exhausted because when I taught history and systems of psychology the first time, I would be up from 10 o’clock at night till two in the morning putting together a lecture. And I knew the stuff generally, but I had to go look for it. But I spent all my time looking. I was looking for the information. Then I’d find it, and then I would embed it. With AI I’m going to get the same kind of information, I’m just going to get it really faster, and because I can pull that together with the knowledge base I have, then I can put it out there. So prep time is going to drop, grading time is going to drop in terms of figuring out how to read through these types of things. Underpaid, by the way, I think you might have mentioned that one, just so we can’t do a lot about that. So sorry, yeah, for the underpaid part. But I want to finish with one quick thing here. There’s a book that I did actually get accepted for publication that I have never written. The contract was signed in 1994 and the reason I mentioned this is that what I wanted to do was take a stats book, but I thought we’re teaching statistics without kind of framework. So what if we took the old psych studies that were really the cool studies, took the data set, retrofit a data set basically that would give you the same data. Have students analyze the data, come up with what they’ve got, and then talk about what the data means, and then explain to them how that fits into like, that was the Bobo doll experiment, where Bandura had models hit a bobo doll, a blow up doll, and then they give the same doll to kids and see if the kids would do the same thing. You could replicate these studies while learning statistics. And I was told I couldn’t write that because most faculty members teaching statistics don’t know those studies. And so at the time, there was no way to do it. Now, with AI, you actually could, it would be very easy to say, I want to know more about Bandura’s Bobo doll study. And so I think that’s some changes that we’re going to have that are really going to be helpful.

Rebecca: One of the things that you’ve both underscored is leaning on the faculty member’s expertise because they can validate what’s coming through the AI tools. [LAUGHTER] So you’re not worried about the reliability, because you have that expertise, and the things that you’re suggesting people use are relying on that expertise to some extent.

Todd: Yeah, and that’s the thing. They’ve got the expertise. How can we help them with it? And help them by putting together a book that’s very modular. You can just go through and work through these things. Actually, the activities are set up, and Lew did this, I thought it was fantastic, they’re put in boxes. You could pull the whole thing out and make a workshop out of it. Very, very easy to pull this stuff out and do that, but by making these things very consistent, we know that reduces cognitive load, which is going to reduce some of that exhaustion and extra work.

Lew: And the other thing is, the examples were set up in a way, again, that they relate directly to what you’re doing in your classroom, your assignment, your example, not something that, again, the cognitive load that you have to translate from somebody else’s existing example. And again, that’s one of the neat things that AI allows you to do. Because now, instead of me saying in a book, do this example, I can put a little bracket there and say, insert your example from your discipline here, and all of a sudden you have individualized learning, individualized application for the teaching technique that you’re trying to do using the power of AI to do that.

John: And one of the things I’ve seen is that we have a lot of people attend workshops on AI just to learn how they can block it and how they can ignore it. But when the focus is on activities that actually allow them to do their work more productively or more easily or more efficiently, they get a bit more excited about it and then start thinking about how they could be using it to improve their courses and to improve their lives in general. And I think that’s a nice way of introducing people to AI who are initially somewhat resistant to it.

Lew: Yeah, it’s critical. I don’t think you can walk into a classroom and say, okay, students, we’re going to use AI today without understanding it yourself. And you hit the nail on the head there, the best way to do that is to do something that you are the expert in, and then learn to explore this technology on your own. It’s going to take a couple years for us to kind of get used to what’s going on. I think in a few years, we’ll be ready to think about how we might use this with students and things like that. But as a faculty writ large, we’re going to have to have a better understanding ourselves of just how to use those tools. And I think to your point about students misusing it and things, we can now have better conversations with them if we better understand how to use the tools ourselves for our own work.

Todd: And how people learn. [LAUGHTER] So we have tht published in the book too. That’s what it does. It helps you understand how these processes work, and so you learn about things like cognitive load. So then you can not teach cognitive load, but using that, build assignments that have lower cognitive load. That was one of the things that Lew came up… I love that one assignment that you came up with, Lew, is basically it’s you load the assignment in and says, “Where are students going to have high cognitive load in terms of really struggling with this assignment? And then AI says, “Oh, they’re going to stumble here, here and here.” And then you start rewriting it. It’s not a lot of extra work for you, and it helps the students. You don’t have to go through examples and have all those students lined up outside your door in office hours, saying, “I don’t understand why this thing works.” And so yeah, I think it’d fantastic.

Rebecca: I appreciate the idea of meeting people where they’re at, for sure. You mentioned something about a couple of years, maybe we’ll be ready to really integrate it with student learning. In five years, maybe, what does good teaching with AI look like?

Todd: This is kind of a fun one. I actually was asked to do a workshop recently that they were looking for a speaker, and they said they were interviewing different people, and they said, “We just want somebody come in and talk to us about what teaching will look like in 5 to 10 years.” I said, “If somebody tells you that they know what teaching would look like in 5 to 10 years, I would not ask them to give that presentation.” Because I don’t know, we used to talk five years out. Tom Angelo’s book with CATs was good for 20 years and still kind of worked. That stuff’s not going to handle anymore, because just in the last three to four years. Can you imagine, just before AI took off, go back just a few years and predict that it would look like this? So I don’t think we have a clue about what overall it’s going to look like. I do think we can look at some trends, though, and we can look at some directions we’re taking, because we can’t just throw our hands up and say “we don’t know,” but we want to be careful about what it’s going to be and it’s just going to be difficult, but there’s lots of good stuff coming out. And I’m going to actually ask Lew to do this, because earlier today he was explaining to me an article he’s reading. I think this stuff is fascinating that’s coming out.

Lew: Sure, I’ll jump in. So one of the things, not only do I keep an eye on higher ed and what’s going on with AI, but also the K through 12 space, and I follow a lot of folks on substack, it’s interesting. I think a lot of the discourse right now is going on in substack, as far as with AI and teaching and learning things. But anyway, there’s a fellow I’m going to try to pronounce his name, Nick Potkalitsky from Ohio, and he’s working with our Ohio Public Schools and writing about that. So the state of Ohio has mandated that by July 1, 2026 all school systems are supposed to have some kind of AI policy in place. So Nick is working with some of the larger school systems, Cincinnati, Columbus, Cleveland and things like that. And in this article he talked about on substack, he’s like, “Listen folks, this is going to be a three to five year rebuild. This is not going to be a quick policy fix. Rebuild means rethinking instruction, assessment, and curriculum, basically from the ground up. And how might we support teachers to make that happen and things like that. But teachers clearly need to be in that conversation. They need to be in that room.” This is the K through 12 space, and I think that’s only going to translate to higher ed as well. So the sooner faculty can get out there and start engaging in this technology and seeing what it works for them, then they will be able to kind of think down the road of how that’s going to affect their students in their classroom.

Todd: We have an epilogue in the book we did too after chapter 10, there’s a section in terms of what we think is coming along. And I just want to add one more quick thing here, when I think about what good teaching is, throughout my career… so I started teaching in 1984 so I’ve been teaching for a minute… and the concept here is that throughout all of that time, when I think about things I wanted to do, and what frustrated me were things that I couldn’t do because I didn’t have time or I didn’t have a way to do them. Universal design was just one example of that, but lots of things. With 400 students, I would love to ask them an essay question, but there was no way to look at that, even to get a sense of what they were writing. I would like to put students in small groups and looked at multiple versions of what they were talking about. There’s so many things out there. And now, as I start looking at what the future of teaching might look like, it’s kind of like one after another we just remove those barriers. So I don’t know what it will look like, but I know it will look better because it is removing things that we can’t do. And I will just end with my chunk of saying this is that we so quickly, when this came out, went right to cheating. Just immediately, the first articles out were, how are we going to stop students from cheating? How are we going to stop them? Students were cheating before AI came along. Don McCabe did a lot of work on this, and the proportion of students cheating in some way was off the chart. It made it easier, no doubt about that, and it does create a real problem, because if students can just jump in and do AI, then there’re many that will do that. But I don’t think we have to go there. And Lew and I talked a lot about this too, is that we shouldn’t just go there. We should start looking at opportunities. And I just talked to someone the other day who said, like, “How are we going to know when we can fully trust AI when it gives us an answer?” And that’s the kind of stuff that just drives me nuts, because when we talked about hallucinations, and so many faculty immediately say, “Oh, I put that in and it made a mistake.” So do I, sometimes. There are times when I make mistakes. I’ve heard my colleagues say stuff and I say, “That’s not true.” And so it’s no different than humans in that respect. We’re going to be able to trust it when we can look at it and we know it’s right, but it is getting better all the time. And Lew said it earlier in this discussion we’ve had here today too, is we can’t focus on what it was. We have to focus on what it can be. And I think looking forward for teaching, there’s some really cool stuff coming.

Rebecca: I think one of the things that you’re highlighting, both in your book and how you responded, is it remains anchored in the science of learning and just how we’re using the technology, and things might shift, but if we keep focusing on good teaching and learning practices [LAUGHTER] and just get them better, it’s not a surprise.

Lew: Well, it’s interesting, especially at the higher ed. We’re all trained in a very specific discipline, a very specific area. We’re experts in that field. We were never trained to teach, or very few of us maybe had that one course as we were TAs or something like that. And again, what the AI is allowing us to do is to further our expertise and our teaching and to kind of push the boundaries where we just didn’t have that training before. So I think, as Todd pointed out, I think there’s going to be a lot of opportunities just to make things better, hopefully.

Todd: Before we wrap up this question too, I’m just going to kind of put a charge out there, because Lew mentioned a little bit earlier, but in the model we have for adapt and stuff, for those of you who are listening, if you haven’t done it yet, just try dumping one of your assignments in for a topic or an area that you know students struggle with a little bit put into the AI. And the book’s got some examples on how to do this, but you could just say, “Where will the students struggle with this?” When I teach statistics, distribution of sample means, always give them fits. It took me years to figure out how to get around that, but if I was a brand new teacher and never taught before, I could put distributional sample means in there and say, Where will my students struggle? And it will tell you, here’s some places. And then, if you want to set that off to the side and then teach it and see where they struggled and make a comparison. And if it kind of lines up a little bit, you know it’s a gateway to kind of figuring out where the problems are going to be before they emerge. Wouldn’t that be cool?

John: So this is the first time you’ve worked together on a project like this. Could you talk a little bit about the working relationship and how that evolved?

Todd: Yeah, let me jump in there real quickly. Number one is writing a book is hard. I mean, it is really, really hard. And AI actually has helped us out with this too. So I wrote a book, this New Science of Learning first edition took two and a half years to write that thing. This book was nowhere near that, six, seven months, and we put a lot of time and energy into it. But the point is, it’s hard. Working with somebody is even harder because now you got to talk about what you’re doing and look at different points of view and everything else. And I think what was really cool with the way Lew and I kind of set it up is that we were coming from different directions. I had the learning theory. He was coming in from the AI, I would write a bunch of stuff, and he’d say, that’s really cool about all that metacognition. Maybe we can change it from like 3000 words down to, I think, 42, one time. And I said, I’m not explaining metacognition in 42 words. And then he gave me like 150 so we had to compromise and go through that. I would just say, working with him, he gets stuff done fast. He is really, really good at things in terms of understanding AI component. His prompting was amazing. I don’t know he’s a little mathematician-ey for me sometimes, but, you know, it was alright working with him. So all in all it was a positive experience. And I think Lew would say the same thing. So what’s the next question? [LAUGHTER]

Lew: No, it was a lot of fun. I really appreciated, as you’ve already mentioned, Todd is a very prolific author, so he was able to help us navigate all of those rough and challenging waters. So I was so much appreciative. But I think we complemented each other well, as far as what we brought to the table, and it’s exactly you know, what the book needed. And in particular, I remember Todd that GLCA, the Great Lakes College Association talk that you gave, what was the title of that? You remember something about neurodiverse learners?

Todd: It was basically how AI can help neurodiverse learners to be better learners. The title is way better than that.

Lew: Right, and Todd, he graciously used himself as an example, I will say. So a lot of things made sense to me [LAUGHTER] after that presentation Todd, [LAUGHTER] I’ll just leave it at that.

Todd: So, let me just say this. Yeah, I explained a little bit about ADHD and some of the characteristics of people who have ADHD, like time blindness, where you think something’s gonna take 40 minutes, but you don’t have a clue? Might be 10 minutes. Might be 17 hours. Who knows. You set up your task list, but you’ve never, ever finished a to-do list. I have never finished to-do list. And yeah, I went through all of those types of things and a couple of other things that I kind of struggle with. And then Lew when we finished, he says, it’s been great working with you, but I’m not sure if I’d heard that presentation first I would have signed the contract. [LAUGHTER] Well, you did it.

Lew: No, it was fine. It was fine.

Rebecca: So Todd, some of our long-term listeners might be curious what role opera played in this book. Didn’t you listen to a lot of opera last time?

Todd: Yeah, you know what? Let’s just throw this in there real quickly, because this is part of the ADH stuff that I’m learning, to whatever extent this fits in with this whole interview here. So ADHD is one of those things that, much like dealing with gluten intolerance and several other things, people think, if you’ve really got it, you must have known it your whole life. There are people who find out later in life that they have this thing. I always knew that kind of general things with ADHD which always struggled with a little bit. And I knew I had some of it. But, I mean, I was 60 years old when I was diagnosed, and so that concept there is suddenly I started to realize and read bunch of books about things like time blindness and some other things, but I had a terrible time writing. And a person, Tamara Rosier, who wrote a fantastic book, Your Brain’s Not Broken. It’s been reviewed in the New York Times and stuff, she said, “listen to opera, like, crank it up. Don’t do ear damage, but crank it up and do that while you’re writing.” And I said, “that will never work, because I have to have it quiet.” And she said, “Just try it.” And I put opera on and cranked it up, and I churned out work like I’d never turned it out in my life. And she pointed out to me that my brain’s always looking for stuff. Part of it would settle on that, and then it just kind of left this focus for the rest. Still had to have absolute quiet when I read. But for writing, that worked. I got put on some ADHD medicine, and two days after I started the medicine, I cannot listen to anything while I write. [LAUGHTER] To me, that’s fascinating, because what it now means is my brain’s actually kind of slowed down enough that that becomes a distraction. But that to me as a cognitive person, that differentiation is fascinating.

Rebecca: That is fascinating,

John: And that’s a nice preview of our next podcast that will be coming out next week. We’re talking to Karen Costa about ADHD and her experiences with that.

Todd: She’s really good.

Rebecca: So we always wrap up by asking, what’s next?

Lew: So I’m a mathematician by training, and I was just at a recent large gathering of many of our professional organizations, and for years now, they’ve been calling for, especially in like the high school setting, that we need to teach more statistics, data analytics, modeling, in other words, creating mathematical pathways, kind of beyond the calculus curriculum. The rub is, is that the way folks are trained, they’re basically trained for the calculus curriculum and not these other things. So what I’m doing is, I’m actually working with some organizations like Math for America to consider how we might take frameworks to increase the number of teachers that were able to develop these courses in these different areas. So what we’re doing is we’re looking at something I like to call AI-assisted retooling, so teachers are learning new content alongside an AI partner to expand their mathematical pathways. So, that’s kind of a really exciting thing that I’m starting to work with. We got a few case examples that we’ve been doing. I’ve been a case study myself of doing that. So, I’m really excited about that. And of course, you know, I’m continuing to work with teaching centers on workshops and things around the book.

Todd: Nice. For me, so I got another book started, because why not? These are always fun. This particular one’s going to be different, though. So I’ve probably said about as much as I can in terms of some of the teaching tips and kinds of things we do. I really think that what’s going to happen pretty soon is there’s so much stuff coming out with AI, real life stuff is going to become something I think people are going to cherish. So the project I’m working on right now is looking more at how I’ve taught in different aspects of it, but what it really is going to boil down to are things that I experienced in the classroom. They’re really kind of unique, interesting things, and then how it played out and how it could be different. And so I had a student one time have a grand mal seizure in my class. And the quick example here is I quickly ran down to a nursing station. It was just around the corner. I ran over there, got somebody, brought them back, and the nurse came back, and we came into the classroom. And I, right now, would tell you… that was my first year of teaching that happened… I would never leave a classroom when an emergency is going on. I should have said to somebody, “Hey, go to the nursing station.” So what I want to do is talk a little bit about what I’ve done, some of the ways I’ve done it, how I would have done it differently, and then how that ties into learning and different aspects. So I really want to make more of a personal story. I don’t know if anybody will be interested in it, but, you know, we’ll give it a try.

Rebecca: Sounds like an interesting book. I’d read it.

Todd: Good. I just sold one. Oooh.

Rebecca: Yeah. [LAUGHTER]

John: Maybe two. [LAUGHTER]

Rebecca: Yeah, maybe two.

Todd: No worries, but thank you, appreciate that.

Rebecca: People have anxiety about, like, what if X, Y and Z happens in my class, and hearing personal stories about how those sorts of things have been addressed and strategies for addressing them, I think eases some worry for folks, so it could fill a gap.

Todd: Yeah, I appreciate you saying that, and I think one of the big ones is that it’s been about 40 years of teaching, and the 40 years of teaching, I think the framework I’m gonna go with, I’ve made almost every mistake you can make, and I still am seen as kind of a person who’s written these books. I do know a lot about teaching, and I’ve learned a lot about it, but you don’t become a better teacher without making mistakes along the way. I don’t think anybody can. And so I’ve done that. And I probably should point out, too, as Lew said, I run around some campuses. I love meeting people and going to campuses. I’ll still keep doing that.

Lew: So Todd, three years ago, I did a column for the MAA, it’s the Mathematical Association America. It’s called Still Learning From My Misstakes. And mistakes was spelled with two S’s, anyway, but yeah, exactly that, that idea of, I think, showing your vulnerability, especially at the part in the career that he’s at, like, oh, Todd Zakrajsek does everything perfect. I think peeling back the curtain and showing the mistakes that we made for the folks who are coming up, that is so refreshing to see that and what we did well and what we didn’t do well, and then kind of examples of moving that forward. So yeah, maybe three books, you might sell three. [LAUGHTER]

Rebecca: Definitely worth the investment now. [LAUGHTER]

Todd: Lew, that will cover my next tea. [LAUGHTER] But thank you, appreciate that.

Rebecca: Well, thank you both for joining us and for sharing your work.

Lew: Thank you so much. Appreciate it. It was fun.

Todd: Appreciate the opportunity.

John: Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

427. Open Exchanges in College Classrooms

Ideally, college classrooms provide students with a comfortable but challenging environment in which diverse ideas and viewpoints are openly exchanged; the reality they experience, though, is often quite different. In this episode, David Laibson joins us to discuss how Harvard University is attempting to identify and address barriers to this ideal.

David is the Robert I. Goldman Professor of Economics and a Faculty Dean of Lowell House. He has published dozens of heavily cited articles on a wide range of topics, including behavioral economics, self-regulation, behavior change, household finance, and aging. David is a Research Associate in the Aging, Asset Pricing, and Economic Fluctuations Working Group at the National Bureau of Economic Research, member of the National Bureau of Economic Research, and serves on numerous advisory boards. He has received Harvard’s Phi Beta Kappa award and a Harvard College Professorship in recognition of his high quality teaching. David is also a co-author of popular textbooks on introductory economics and  a co-editor of the Handbook of Behavioral Economics.

Show Notes

Transcript

John: Ideally, college classrooms provide students with a comfortable but challenging environment in which diverse ideas and viewpoints are openly exchanged; the reality they experience, though, is often quite different. In this episode, we discuss how one institution is attempting to identify and address barriers to this ideal.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is David Laibson. David is the Robert I. Goldman Professor of Economics and a Faculty Dean of Lowell House. He has published dozens of heavily cited articles on a wide range of topics, including behavioral economics, self-regulation, behavior change, household finance, and aging. David is a Research Associate in the Aging, Asset Pricing, and Economic Fluctuations Working Group at the National Bureau of Economic Research, member of the National Bureau of Economic Research, and serves on numerous advisory boards. He has received Harvard’s Phi Beta Kappa award and a Harvard College Professorship in recognition of his high quality teaching. David is also a co-author of popular textbooks on introductory economics and a co-editor of the Handbook of Behavioral Economics. It’s really an impressive list of things there. Welcome David.

David: Thank you. Glad to be here.

Rebecca: Today’s teas are: David, are you drinking tea with us today?

David: Well, I don’t actually have my tea cup with me right now, but I do have my preferred tea on hand [LAUGHTER] because I knew you would ask. So my favorite tea right now is a Korean matcha called Osulloc, and it comes in a very handy format where I can basically take my matcha tea bag, drop it in my water and have my lovely cup of tea, but I don’t honestly have one right now. [LAUGHTER] So, I thought you were gonna ask about what I like to drink, as opposed to what I’m drinking.

John: And I am drinking a Twinings oolong tea today.

Rebecca: Oh, nice. Must be the end of the semester is getting a little intense with the black teas, huh?

John: It is. [LAUGHTER] I think this is my third or fourth cup today. [LAUGHTER]

Rebecca: I have a decaf Irish breakfast this afternoon.

John: That’s the difference being in the administration. [LAUGHTER]

Rebecca: I have about a week, have about a week. [LAUGHTER]

David: Well, good luck reaching the endline.

John: So, we’ve invited you here today to discuss your work as co-chair of the Classroom Social Compact Committee, which we had read about in an October 6 article in The New York Times. Can you tell us a little bit about this committee?

David: Sure, the committee did most of its work in 2024 and issued its findings report in 2025, early in the year. Maya Jasanoff, another faculty member here at Harvard, is the co-chair of the committee, and our report was both a description of the state of affairs here at Harvard and a set of recommendations for addressing many of the problems that we uncovered or documented. The committee was charged by the Faculty of Arts and Sciences Dean, Hopi Hoekstra, and the charge, I won’t read the whole thing, but it included describing the nature and purpose of the faculty of arts and science classroom and developing practical recommendations for engendering a vibrant learning environment in the classroom.

Rebecca: Can you talk a little bit about some of the primary issues that were raised in the report?

David: So, there’s a long standing recognition that the classrooms here have a aspirational role to play, that perhaps we’re falling short on on many dimensions, and let me review some of them that were discussed and documented in the course of writing our report. So, one thing that we’re concerned about is the degree to which students come to the classroom not completely comfortable sharing what they think when they raise their hand, if they raise their hand. There’s a fair degree of self-censorship, an extent to which people, for various reasons, may not feel comfortable speaking up. It may be that they’re shy and don’t want to say something that might be wrong or that they don’t want to say something that may be viewed critically by other students. And so one aspect of the classroom that is problematic is the degree to which people are not comfortable bringing authentic voices into the classroom setting. Another significant issue is the degree to which our students have, in some cases, focused energies outside the classroom, to the extent that it actually undermines the academic center of their lives here. Now I don’t want to imply that that’s the fault of our students. I think in many cases, that’s a reality that our faculty are not creating classroom experiences that are sufficiently stimulating and therefore engaging our students’ interest. And so I think the un-centering of the classroom, which has been occurring for decades here, is a two-sided phenomenon. On the one hand, it’s students that are focused elsewhere. On the other hand, it’s faculty that may not be creating the most stimulating and exciting environment intellectually in the classroom. And I think we’ve all agreed, unfortunately, that that’s okay, and this report is part of reversing that consensus and reasserting the principles that I think we would agree with, but haven’t always been our practice here at Harvard.

John: And I think what you’re describing is something that we’ve seen at pretty much all institutions, to some extent. But one of the things that really impressed me about the report was the extent to which you gathered data and the ways in which you gathered data, because that could be a useful model for other institutions as well. Could you talk a little bit about what types of data you gathered and the process of assembling that?

David: So we did a year of data gathering, and as you point out, it was pretty aggressive and extensive. We wanted to make sure that we really had our finger on the pulse of the many different communities and the many different stakeholders, so we held well over 30 group listening sessions, and then I would say, hundreds of one-on-one listening sessions, where you sit down with a student and have a one- hour lunch and get a full download from them in the most confidential setting, where it’s just a one-on-one conversation, as opposed to someone volunteering an opinion with 40 other people in the room, which can be a little daunting. We also consulted with colleagues and consulted with many different committees that sit here in the Faculty of Arts and Sciences that oversee our curriculum. So the Educational Policy Committee, the Graduate Policy Committee, the General Education Committee, a place called The Bok center, that is a sort of mentorship for teachers unit, and we talked with department chairs. We talked with divisional deans. So we really went out there and gathered a lot of data. In addition, we looked at 11 different surveys that were fielded either by us or by other groups inside Harvard, measuring undergraduate opinions, graduate opinions, faculty opinions, alumni opinions, and that is the maximally confidential space for people to give answers, because a lot of these surveys are done without any ability to identify the respondent other than knowing that they are, in fact, a Harvard affiliate, because they got into the survey using a Harvard ID. So we looked at five undergraduate surveys, three graduate student surveys, one faculty survey and two alumni surveys, and by that point, we had a lot of data that would enable us to, I think, have a fairly precise understanding of at least what the community thought and what they were willing to share with us in these different formats.

Rebecca: So, you mentioned the self-censorship issue that bubbled up across all of these different audiences that you surveyed and collected data from. What are some of the consequences in terms of student intellectual development?

David: So there’s a lot of adverse consequences. One is the obvious problem that students don’t feel, in many cases, that they belong, because their perspective is not welcome, and that, I think, undermines their ability to thrive. They’re also not developing the critical skill of learning how to speak publicly. If they’re constantly suppressing their voice in these public settings, and the classroom setting, then that’s a skill that isn’t being nurtured. I think perhaps one of the most important problems with a classroom where people don’t feel that they can say what they really think is that there’s this false impression of consensus when in fact there are enormously divergent viewpoints. They’re just invisible because most people that are speaking are speaking in a relatively narrow domain with respect to maybe the most sensitive issues. So when you have a classroom that is a place of self-censorship, and I want to emphasize many classrooms don’t have that problem. But when you have a classroom where people are self- censoring, you get a false impression of homogeneity. You get a set of skills that are not being developed. You get many students feeling that they’re just there on a performative basis, and it undermines the whole intellectual enterprise. We’re here to learn and really compare ideas while all that’s out the window when you have the impression, and again, this is only in some classrooms, that there’s a kind of acceptable viewpoint, and anything that sits outside that viewpoint you better keep quiet about. But I want to emphasize, a lot of people think that the way the law is laid down is through faculty and graduate student teaching fellows. Indeed, the students told us that mostly they were concerned about what their fellow students thought. So, if anyone has the impression [LAUGHTER] that this is a sort of monoculture problem that is driven by or primarily by instructors, I want to, right from the start, clarify that overwhelmingly, the students said that in many cases, it was fellow students and their judgments and what would happen after class in the dining halls that they were most concerned about with respect to saying something that might be unpopular in class, and not the bad grade they might get. Indeed, I was quite happy to hear that the faculty were the people that the students were the least concerned about. So I think there’s more concern about some of the graduate students that are grading their papers in a big class, but especially there’s concern about their fellow students. Now that’s not to say that the faculty don’t have any role to play in changing this equilibrium. Definitely, the faculty need to change norms and need to lead with their voice this new approach to a classroom where more perspectives are sought out and encouraged and welcomed, but it is, I think, important to emphasize that it’s not just the faculty that are contributing to self- censorship, it is fear of what other students are going to say in the dining hall after class.

John: One of the more striking statistics there was that in one of the surveys, 33% of Harvard’s college students reported that they felt free to express personal feelings and belief, as you’ve noted, a lot of students are self-censoring. However, there were some other statistics based on individual course evaluations that suggested a much lower rate. How was that explained in the report, that discrepancy between the reports from seniors and the individual course summaries.

David: Yeah. So just to highlight the tension, if you ask people about their overall academic experience, only a third say they were comfortable speaking openly, but when you ask them about the individual course that they just took, so ask them about one course, and you get about 90% saying that they’re comfortable speaking openly. So clearly, what we’re learning here is that this is not a problem that people are experiencing everywhere, all the time. It’s a problem they’re experiencing in some places. So when you ask people, as they’re leaving after four years, what was it like? Did you feel the need to self-censor? They say, “Yeah, sometimes.” But when you ask them about the class they took last semester on 19th century history of Brazil, they say, “No, actually, in that class, I did not feel the need to self-censor.” So, the distinction, I think that’s important to highlight, is that it’s not a problem that’s occurring all the time for our students. It’s a problem that’s occurring frequently enough that it’s a real concern, but not so frequently that it’s affecting every curricular experience. And I think that speaks to a couple issues. First, the problem is somewhat contained. But secondly, there probably is a degree of intellectual self- segregation that is contributing to the sense of, “Well, it’s not a problem in my class.” Sometimes I ask students, “Well, what happened in your class? What was the curriculum?” And they’ll describe a relatively homogeneous student body in that class. And then I’m not surprised that if the classes has advertised itself as having a particular viewpoint, and the students get that message early on, and the students that want that viewpoint stay, and the students that don’t want that viewpoint move on, you can easily see how people might say, “Yeah, in that class that I took, I had no trouble saying what was on my mind.” But then one realizes that in parentheses, that’s because the class ended up attracting a relatively homogeneous set of perspectives in the student body. And so I do think all of these considerations make it challenging to know exactly how deep the problem is. I definitely think we can over-exaggerate it if we just cite the 33% number of people that feel comfortable speaking their mind. But I think we can also kind of over minimize it [LAUGHTER] by saying if you look at individual courses, 90% of the students say that in the particular course they are evaluating right now, they feel free to speak their mind. We don’t know how much of that is a kind of self- segregation into curricular environments that they feel comfortable in.

Rebecca: What are some strategies, maybe both structural or also within a class, to work against this?

David: Well, I think you start with awareness as an instructor, I think instructors don’t realize the degree to which their students feel pressure from other students to accept a kind of student-endorsed viewpoint. And I think faculty are trying very hard to have real conversations in their classrooms, and are often working to prevent their own viewpoint from becoming, in some sense, the only acceptable viewpoint. They’re not necessarily jumping out on day one and saying, “This is what I think,” they’re creating space for student perspectives. They’re working hard on that front, but they don’t realize that it’s not just that students are trying to glom on to what the faculty member thinks, it’s what the other students think that is a real issue. And so I think faculty need to be explicit in saying, “Hey, in this classroom environment, we really encourage diverse perspectives, and I’m going to be thoughtful in drawing them out.” And so I’ll give you examples of how we do that in my class. You probably have both have the experience of that wonderful student who is always quick to get their hand up right away, and if we always call the students who get their hands up in the first five seconds when we ask a question, or when we open it up for questions, we tend to get a relatively homogeneous set of answers because it’s the same students that we’re privileging. They’re the students who get their hands up the quickest. And if we call them, once there are a few hands up, well, we’re going to over sample the same students. And so one thing that we do in my classes, we, by that, I mean me and my co instructors… we try to let the question settle and not pick someone immediately. And I have something called the 10-second rule, where once I ask a question, I’ll count to 10 in my head, quietly, and only then, when I’ve allowed 10 seconds to pass, do I finally start picking from the hands that are up. And at that point there’s a lot of hands up, and I have my pick of students, and I can go for students that have not been contributing so far in that lecture, or so far, even in that semester. So I love the ability to bring in this far more diverse set of student voices by just giving them time to get a question together. And I think there’s lots of ways that faculty can launch a more diverse conversation by not allowing a small set of voices to become the dominant voices, and by looking for opinions that lean a different way than the first opinion you got. So you can ask a question, and some student says, “Well, I think A,” and you can ask, “Well, who thinks not A?” And then you can draw in the range of opinions, rather than allowing the first student to say, “I think A” and then the second student to say, “I think A too,” and then the third student “I think A three,” and everyone thinks a. And that’s pretty much the herding effect you don’t want. And you can be very intentional in creating an anti-herding effect by only asking for questions or perspectives that create the heterogeneity that you’re trying to engender, rather than allowing students to herd on the first answer that comes out of the room. So I think there’s a range of strategies, and they had to be launched on day one to get the equilibrium right in the classroom. Students need to hear the message early on and hear it repeatedly, that this is a place where we’re looking to hear from different perspectives, and we’re not trying to all agree on something. I am very disappointed when I hear [LAUGHTER] faculty telling me about how much agreement there is in their classroom and how excited they are by that. I think when everyone’s agreeing on something, it’s probably a mundane thing, or they’re observing false agreement. There’s enormous disagreement among our students, and if you know how to draw it out, you’ll get it. And if you get everyone to agree together that something is true, well, I ask you, what does everyone agree on that isn’t something that’s utterly trivial.

John: One of the things I’ve done in my classes is I do a lot of polling, but also I have a Catchbox, one of those little cubes with a microphone in it. And when I’m asking questions where there is likely to get a lot of diversity, I’ll wander up to the back of the room because that’s where the students are most likely to sit who are least likely to volunteer and either hand it to them or toss it down the aisle a bit, because the same students always sit up front with their hands raised all the time, and students in the back, at least in my class, are much less likely to ever raise their hand, no matter how many other people have done so. I think one of the things you recommend perhaps using the Bok Center to have people work on alternative teaching strategies to draw out some of these things.

David: Yeah, we do some of the things that you just described. We also sometimes kind of insist on questions from the back of the room. And we also have something called a pre-lecture activity where we get everyone to anonymously answer polls or play games or do something that we’ve created before the lecture. And it’s amazing how much variation you get on these pre- lecture activities. And then everyone in class is astounded when we show them the histograms of responses, and they realize, “Wow, I thought everyone agreed with me and now I realize half the class has a totally different viewpoint.” And it’s not just now a hand or two, they’re seeing data that represents the entire distribution of people in the class, and everyone says that’s the thing they remember the most from the semester, seeing their own data and realizing how different their own perspective is from that of everyone else in the class. And it’s an eye opener.

Rebecca: One of the other things you mentioned earlier was the self selection into classes. Do you have some strategies to encourage folks to explore more things? [LAUGHTER]

David: Yeah, so certainly that’s advice that I give all my students. I’m not sure that that’s [LAUGHTER] the best way to get that advice to be accepted, just sharing it. I certainly take the view that if you are taking a class that’s just going to tell you what you already believe about the world, you’re less likely to be learning than by taking a class that might challenge your preconceptions. That said, I think one thing that instructors should do, in my view… of course, they have academic freedom, so they’re free to decide whether they want to do this… but I would encourage it as good pedagogy, is to offer a range of perspectives in a class and to, themselves, offer the alternative view on an issue, as opposed to the view that is the maybe main thrust of the course. Indeed, I do this myself, I’ll give students the range of evidence on an issue, and I’ll say some scholars think this, and here’s why they think this, and some scholars think this, and here’s why they think this, and here’s the body of evidence that we have, and you can see all the different empirical estimates that have been generated in scholarship in the last 15 years, and so we’re going to be very candid with you about the limitations of what we know and how scholarship has yielded different answers, rather than picking the study that I most like, that is most aligned with my personal viewpoint and teaching it that way. I think good pedagogy goes hand in hand with acknowledging the range of viewpoints that exist in the scholarly community and being forthright that there isn’t a single viewpoint that happens to match my own that is the right approach. And we even have a lecture in the introductory economics class on epistemics, where we talk about the limits of our knowledge and all the ways in which our own views have evolved as we’ve professionally been exposed to more data, seen new research, changed our minds, and so we try to model for our students this concept that there isn’t a set of truths carved on stone that we’re going to hand them. And I would recommend to colleagues that they teach their classes that way. And I think students should also think about learning that way. If they think that they have already discovered the truths carved in stone at age 19, [LAUGHTER] well, they’re in for a big surprise about how the world works. They’re never going to find those truths carved in stone, and they certainly don’t have them already at age 19. So I think we need to message that more in our classrooms.

John: One of the things you’ve mentioned is that students don’t always feel free to express their opinions, but there was also in the report a discussion of student motivation, student engagement, and the relative emphasis that students put on the classes versus the social life and extramural activities. In the report, there’s a number of recommendations, one of which you’ve already discussed in terms of changing how we teach in the classroom. But what are some of the other recommendations of the committee relating to these two issues in terms of getting more students engaged in class and putting more of an emphasis on class, as well as perhaps being more free to express their thoughts in class.

David: I don’t want to create too strong an impression of certitude here we are saying in the report that recentering the classroom, recentering academics, is an important goal going forward, and indeed, we’re already making progress this year where I’m seeing tangible changes in this current academic year, which is really giving me a lot of encouragement That said, the extracurricular life is rich and serves many purposes. So we’re not trying to convey the message that all of that should be shunted aside and people should focus exclusively on their academic lives. I think it’s about finding the right balance, and for some students they’re already well balanced, and for other students, maybe they’re unbalanced with a search for courses that ask very little of them. They’re called gems here at Harvard, g-e-m, and that means low work, high grade, and so maybe not so many gems should be offered, and maybe students shouldn’t seek out a course that is called a gem, and if people feel that they’re so busy out of class extracurricularly and they’re searching for gems, that that’s a sign that maybe the balance hasn’t quite been found for that student. So I think that we should, as a community, ask ourselves, “Well, what courses are famous for having a low lift of work and a very high grade distribution?” and then sort of ask, “Is that the sort of course that we want to be offering?” And that means that for that kind of curriculum, maybe we ask more of our students. Maybe the readings are more demanding, maybe the grading is tougher. So I think that there’s some suggestion to the community of instructors that we might think about raising our standards. And there, we’re thinking about what we ask our students to do out of class, what we ask our students to do even in class. So there’s the cell phone that’s inevitably taking the student’s attention away from what’s happening in the classroom and then 75 minutes have gone by, and you ask the student as they’re leaving the room about what just happened, and because they were parallel processing through the whole thing, they actually didn’t get very much out of it, and 75 minutes were spent kind of scrolling some social media app, and the student might think they got a lot out of that 75 minutes, but in fact, it’s very hard to do two things mentally at the same time. So I think we need to create more stimulating classrooms, more demanding classrooms with respect to assignments, we need to not create a classroom where there are so many A’s that the student can essentially only distinguish themselves by joining the consulting club outside of class and putting all their energy into that. So I think that there’s a lot to be done by instructors in creating a more exciting classroom, a classroom where excellence is acknowledged with higher grades, and it’s not that the norm is an A. I think students might also ask themselves, is the best use of their four years here to focus on things that are happening outside of our classes rather than inside our classes. And if the answer from the students is, well, actually, it’s better outside the class, then I think the faculty need to double down on making the in-classroom experience better. So a lot for us to think about as teachers, and our obligation to create an exciting curriculum that makes students thrilled to come to class. That should really happen. I mean, students should not come to class grudgingly. They should come to class with excitement because something interesting is going to happen, because there’ll be a conversation that will be memorable, because they’ll have an insight that they’ll take with them for the rest of their lives. I mean, classrooms are some of the most remarkable, extraordinary experiences one can have in a lifetime. And if it’s not that, then it’s on us faculty to up our game and create these experiences that are so memorable, that are so transformative, that are so important in the development of our students, that students are actually eager to be there and are not going only because attendance is being taken. Though I might take attendance if that’s a problem too, but for sure, the goal is a classroom that is a wondrous experience, that is the highlight of one’s life, that is sad as you leave it behind and graduate from college. Graduating from college should be, on one part, an exciting moment to move on with your life, and on the other hand, a very sad moment when you’re leaving behind this incredible opportunity to sample intellectually four courses that are life changing every semester. Well, let’s create those courses together.

Rebecca: I think a life changing course, seems like some faculty might have some imposter syndrome [LAUGHTER] around being able to achieve that. Can you talk a little bit more about some of the strategies in the class to make that engaging experience where students maybe don’t want to interact with their phones and they want to fully participate in class.

David: Well, I think students, by and large, do want to participate in class. I think the phone is sort of like a drug and everyone says, “Well, I don’t want to be a heroin addict in five years. I’m going to quit between now and then.” And of course, it’s very hard to quit the addiction. I think phones have a lot of those addictive properties. You enter the classroom thinking, “Gosh, I really need to focus.” I mean, this is my own experience as a student. There weren’t cell phones, but there were plenty of things that one could let one’s mind wander to. So I always enter the classroom intending to take good notes and really focus and not get distracted and learn intensively, and often I found myself drifting off. So we’re all easily distracted, and the cell phone is easy distraction on steroids. So I think students want to learn, and I think we need to create environments that facilitate that learning and make it easy. There’s so many classes that I’ve attended where literally, I was on the edge of my seat through the whole thing and excited because there was something important being said, some conversation with students, among students, between students and faculty, that kept me engaged. And I think students want that. I don’t think they are eager to spend these four years blowing off their courses. I think that is something that happens as a unintended consequence of all the other things they’re doing, like the conversation with a roommate that starts at 11pm and doesn’t end till 3am and after four hours of debating every single philosophical issue under the sun, you finally go to bed, and then, for your 9 am class, you’re very sleepy, and you get to the 9 am class having not gotten enough sleep, and you’re barely awake. That student was excited to be debating philosophy at two o’clock in the morning, and unfortunately, got to class in a drowsy state because of it. They’re not intellectually unengaged, but they’re still not going to really thrive in that classroom because of the sleeping that they didn’t get. So my sense is that students want to learn, and we just need to help create environments where it’s conducive to learning, where being in class isn’t going to put you to sleep, it’s actually going to wake you up, because so many exciting things are happening with respect to what is being said, debates that are taking place in real time, opportunities to ask questions, moments where we’re all on the edge of our seat because a student is actually pushing back on a faculty member on some issue and there’s a interesting dialog happening, maybe students also interacting with each other on some issue that is being discussed in the class, and so there’s a conversation, there’s issues that are surprises, there’s intellectual turns that were unexpected, there’s data that’s surprising, there’s ideas that are engaging. I mean, these are all aligned with what our students want. And I think the problem is often they come to class, and actually it’s a pretty boring lecture, and that’s not their mistake, that’s ours.

John: And along the lines of mobile devices, one of the things recommended in the policy is that mobile devices be either banned or discouraged, except in the cases of documented disabilities or when they’re being used actively for some component to the class. And in your class, the article mentioned that you had a device- free zone, but many people chose not to sit there. How can we encourage students, perhaps, to not use mobile devices?

David: I want to be clear, the policy in our class is no cell phones anywhere, so that’s a universal ban across the entire classroom, but we do allow students to use laptops and to use iPads if they want to. Though we also have a iPad and laptop free zone, I struggle with the question of laptop and iPad use, because for many students, they, I think, legitimately view it as an aid for learning, just a regular academic experience, I like to take my notes on an iPad, for example, as opposed to any need to use an iPad. So, I’m open to students doing that. I do warn them that it’s a slippery slope. Once you have the laptop open, you’re one click away from all the distractions and temptations of the internet, and that will really undermine your teaching. And what we do in our class is we actually hand out photocopies of the slides so they can commit to keep their laptop closed by simply using our photocopies and taking notes with a pen on our photocopies. And most of our students do that. That said, I think there are students that want the hybrid experience, and probably some who are taking advantage of being in a laptop allowed zone or iPad allowed zone to occasionally check out and check into their email or everything else that’s happening. So I’m torn about exactly how to handle the classroom environment. I’m open to professors that are going to do a complete ban. I think that’s fine. Obviously there’s accommodations, and we, I think, need to be mindful of that, but a complete ban outside of accommodations. But I’m also open to saying, “Hey, these young people need to learn how to develop this skill of living in a distracting world, and maybe part of developing that skill is realizing that after 75 minutes in the classroom, they have gotten nothing out of a lecture because they found themselves doing one thing after another on their laptop.” And so there is an upside to the distraction, which is a little bit of growing self awareness, which emerges as people begin to discover that we think pretty serially and cannot think consciously on parallel streams, and consequently that skill set of learning how to operate in this distracting world is developed in a room that allows them to use these electronic devices, though I think the cell phone is just so obviously destructive that I’m happy to nix that.

Rebecca: The policies that completely ban all devices except for accommodations… are we kind of skirting or dancing around issues of privacy when it comes to students’ disability status and things like that.

David: Yeah, so that’s a real issue. I mean, I think I’m still comfortable with having that policy, but I think you make a good point.

Rebecca: So we certainly encourage folks to read the full report, to dig into the details, but we always wrap up by asking, what’s next?

David: You don’t change the momentum of a university overnight. So I think writing a report is a great step, and now there’s lots and lots of work to be done to both evaluate the merits of the arguments in that report, and to the extent the arguments have merit, to change our culture, and some of those changes come with legislation, which indeed, Harvard has passed. There’s new language in the undergraduate handbook, which is in the report and mentions a lot of the things that I’ve discussed, including other things that I haven’t discussed, like Chatham House rules in class to give people more comfort speaking authentically. Chatham House rules, as I think your listeners know, is a policy of non-attribution so people can say things in class, experiment with new ideas without being afraid that it’s going to show up on a social media post five minutes after class ends with their name attached to it. So legislation, policy making, culture change, rules that align with the messages that I’ve said, like, how much do we allow pass/fai… Harvard is restricting the degree to which students can take pass/fail courses, not eliminating it, but reducing the freedom to do that as frequently as students did so in the past… finding ways to implement and enforce policies that ban electronic devices in the classroom or at least limit their usage, figuring out where the sweet spot is, and all of that, changing the norm so students get comfortable with difficult conversations, not just in the classroom but also in the dining halls, creating a classroom culture where we’re excited to talk with people who disagree with us, rather than horrified to interact with someone who doesn’t share our viewpoints. I’m seeing all these things actually happening in real time. There’s more and more willingness, just in the last year, for people to say what they think, and we have a lot of work to do. It’s not that culture change happens overnight, but we’re, I think, broadly, moving in that direction, even though there’s individual instances where I’m quite unhappy and still concerned about the freedom of speech on our campuses. So it’s sort of two steps forward, one step backward. But we didn’t get here overnight, and we’re not going to get out of the place we are in overnight. Let’s keep working on these issues and thinking about how we make our academic environments healthy and productive and engaged, and how we create classrooms that people are super excited to have the opportunity to enter and sad when the 32 courses are over and they’re off to the rest of their lives. College should be an extraordinary privilege and an academically delightful one, and getting back to that place, if we were ever there, is going to take some work.

John: Well, all the things that you’ve discussed, I think, pretty much all colleges are facing and it’s nice to see the actions that you’re taking. And I think a lot of institutions could benefit from exploring these approaches.

David: Thank you.

John: And we will include a link to the report in the show notes, as well as a link to the New York Times article that provoked our interest in this topic.

Rebecca: Yeah.

David: In the New York Times article, I will say one thing, it puts a fair bit of emphasis on the students. I don’t think that’s the right balance. I think we need to put a lot of emphasis on what the faculty are or are not doing to create a stimulating classroom environment that will, in this dance between the student and the instructor, create a combined commitment to all the things that we’ve discussed, and just saying the students need to be more academically inclined, I think, is the wrong message, and the New York Times article seemed to have a little bit of that tone.

Rebecca: I appreciate your emphasis on actionable items, and as members of our institutions, we can only control the actions that we can take. [LAUGHTER]

David: That’s right, we’re going to make all of this better by creating classrooms that are exciting to be in.

John: And at least move in that direction as much as we can.

David: Yeah, I’m pretty optimistic, actually. I think there are some people that are just brilliant, charismatic teachers, and it’s impossible to copy and paste that. But there’s also a bunch of low-hanging fruit that doesn’t involve being a charismatic instructor, that can create the classroom environment that’s exciting to be in. I actually think that we can create experiences that are very meaningful for our students by sort of working down a long list of check boxes for classrooms that have the right ingredients for engaged learning and active learning.

John: Well, thank you. We very much appreciate you joining us.

David: Thank you. Great to meet the two of you.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

425. Authentic Voice in the Age of AI

Student use of AI tools presents challenges for faculty teaching writing. In this episode, Anna Mills joins us to discuss when and how AI tools can be used to help students develop their writing skills.

Anna has been a leader in exploring effective strategies for integrating AI into higher education in a manner that fosters the development of student critical literacy. Anna serves on the MLA Task Force on Writing and AI and as a lead advisor on the instructional design for MyEssayFeedback.ai. She also has served as the only educational specialist recruited by Open AI to test GPT-4 pre-release. Anna is also an OER advocate who has released numerous OER resources including two OER textbooks, one on How Arguments Work: A Guide to Writing and Analyzing Texts in College, and the other on AI in College Writing: An Orientation. She is also one of the developers of the PAIRR process in which students develop writing skills through feedback from peers, AI, and individual reflection.

Show Notes

Transcript

John: Student use of AI tools presents challenges for faculty teaching writing. In this episode, we examine when and how AI tools can be used to help students develop their writing skills.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Anna Mills. Anna has been teaching college writing courses for 18 years and currently teaches writing at the College of Marin. She has been a leader in exploring effective strategies for integrating AI into higher education in a manner that fosters the development of student critical literacy. Anna serves on the MLA Task Force on Writing and AI and as a lead advisor on the instructional design for MyEssayFeedback.ai. She also has served as the only educational specialist recruited by Open AI to test GPT-4 pre-release. Anna is also an OER advocate who has released numerous OER resources including two OER textbooks, one on How Arguments Work: A Guide to Writing and Analyzing Texts in College, and the other on AI in College Writing: An Orientation. She is also one of the developers of the PAIRR process in which students develop writing skills through feedback from peers, AI, and individual reflection. Welcome Anna.

Anna: Thank you. I’m excited to be here.

John: We’re very happy to talk to you.

Rebecca: Today’s teas are:… Anna, are you drinking any tea today?

Anna: Oh, yes, I am drinking Yorkshire Gold. It’s my standby.

Rebecca: A classic.

John: That’s long been a favorite of some of our British colleagues, and I am drinking ginger peach green tea.

Rebecca: Nice. I have switched on to Jasmine green tea.

John: Very good. So we’ve invited you to here today to discuss issues that you’ve been writing about and talking about quite a bit related to fostering academic integrity while helping students develop AI literacy skills. First, though, can you tell us a bit about how you became interested in the possibilities for using AI in education?

Anna: Sure, so I actually grew up in Silicon Valley, and my dad was a software engineer working with the dinosaur age AI of the time, and so when I kind of stumbled onto today’s large language models, before ChatGPT, I was playing around with how well it would do generating essays that fit my assignment criteria, and I realized that it had made a point that I hadn’t thought of in critiquing a sample essay that I had written. And I just thought, this can complement my thinking. This can stimulate my own thinking, this can stimulate and complement what I already do. And if it can do that for me, then it might also stimulate my students, so it might support their learning, depending on how it’s used. But I think I’ve always been interested, growing up in Silicon Valley, in the divergences and convergences between human experience, human thinking, and mechanical processes’ logic and early computing. So I’m always interested in those kinds of tensions and questions, and I just was electrified when I realized that systems that can simulate thought are here. The Turing test has been passed, and that’s the strange age we’re in. [LAUGHTER] So, I think it’s an exciting moment.

Rebecca: Not all faculty lean in [LAUGHTER] necessarily in the same ways that you do, given your past history and your curiosities. One of the major concerns that faculty have about student use of generative AI platforms is that students might use it to substitute learning, rather than complement it, like you had just kind of noted. Many faculty have reacted to the availability of generative AI by simply banning its use. Is that a productive strategy?

Anna: I actually think it’s a pretty reasonable strategy. I would just say it can’t be the end, right? It’s a beginning, it’s not the full answer. And certainly, if we’ve designed a learning activity for the student to do, then it makes sense to ask them to do it, right? and not to have a machine auto-complete it for them. So as a first step, I think banning it for a learning activity that’s not designed for it makes some sense. I just think it’s sort of similar to the dawn of the internet, in that we might ban it for some things, but we also will start to build AI literacy curriculum to guide students as to how to understand this phenomenon, how to think about it, how to consider whether to engage with it or not, and we start to develop some assessments that might involve it as we figure out how it actually could support learning rather than getting in the way. So not everything has to involve it, and certainly we need students to do something, right? So we have to ban it in some contexts and in some ways, even if we want to work with it. So I don’t think instructors who ban it should be demonized at all. I think they just need a little more support to think about those next steps, as far as, how can I continue to guide students in this evolving landscape? How should I start to consider applications, limited applications and pedagogy. I don’t want to demonize anybody for banning it. I think it’s a very reasonable first step.

John: Would limiting or prohibiting its use on lower-level tasks be appropriate so that students could develop some basic skills in the discipline that they could then use to evaluate AI output? Because AI can be really productive, as you’ve noted, but it does also hallucinate, and it does produce output sometimes, which is not very reliable, but in order for students to be able to evaluate that, they need to develop some skill in order to be able to distinguish between what is quality output and what is not. So might bans be more appropriate as students are beginning to develop the skills and then perhaps progressively allowing more use of AI?

Anna: Absolutely, yeah, I’m completely with you on that, and I think there are parallels to calculators and computer use. And I’m very concerned about setting some guardrails, some guidelines, so that students still have these formative experiences, so they can develop the kinds of skills that they need in order to do their own thinking, make their own decisions, to interact with AI productively, to be skeptical of what it gives them, to improve on what it gives them. So absolutely, I want to do that, and that’s kind of why I’ve gravitated toward not inviting my students to use it in hybrid ways in their writing, I want them to have those experiences in college of developing an essay from start to finish, with their own words, their own ideas, their own organization. They’re making the decisions and AI is used in my pedagogy mainly for feedback, that they then reflect on critically. So, absolutely and I think, unlike the calculator, I think you have to wait longer before you’re really ready to be learning with AI in these hybrid ways. I think we have to protect space for students to develop their own skills, making their own choices.

Rebecca: We talked a little bit about disciplinary skills, but we’ve also mentioned AI literacy. Can you define what that is? I think we like vaguely have an idea of what that is, and we’ve been using that term over a couple of episodes, but haven’t really defined it.

Anna: Yeah, I think it’s hotly contested. Sometimes people use it to mean sort of technical knowledge about AI systems, and that’s not usually what I mean. But I think that you have to know enough to be skeptical, as John said, to know that there are errors and hallucinations, that these are fundamentally sort of pattern-matching systems. And beyond that, I think a lot of the things that help you interact well with AI are not specific to AI. So there can be general information literacy skills, reading, writing, and critical thinking skills, expertise in a subject matter, but I think they also involve knowing what current systems are capable of and some of the ways that people use them strategically, so some familiarity with this landscape of what people are doing with these systems and how well they perform. And that’s, of course, a moving target, but I think it is important for educators and students to be keeping up, to some extent, with what’s possible, because that will be part of workplaces, certainly. And at the same time, it’s not about the technical details. It’s not about learning Python and being able to build these systems yourself. It’s not even about prompt, quote, unquote, engineering. What we’re hearing is that, with the current systems, as they improve, you really don’t need these specific technical approaches. You need expertise and thinking and reading and writing skills. So I think any look at AI literacy should really emphasize those. But I think that there is also a certain familiarity that we need to support teachers and students to develop with what can you do now with these systems? What are people doing with them, and what are the pitfalls? What are the dangers as you use them?

John: One of the things that’s happened over quite a period of time is that more and more assessments in classes have been shifted outside of the classroom. That is especially true with online work, and if you are trying to restrict student use of AI tools while they’re developing those basic skills, one of the things that many faculty have tried to do is rely on AI detectors. How reliable are these? When is it appropriate to use them? And in what ways might it be appropriate to use them?

Anna: So they’re definitely not reliable, even the companies will acknowledge that those detection systems sometimes label AI text as human written and sometimes they label human written text as likely AI. So any use of them has to take that into account and not rely on them. That said, I do think they have a limited place in my practice, I have a lot of colleagues who strongly disagree with me, and I recognize some of the concerns that we don’t have rigorous testing of these systems for things like bias against English language learners, and I would like to see more testing and regulation around them, but that said, we do need ways to deter misuse, to make it less tempting to students, and I think AI detection does quite a lot in that respect when you use it to complement other strategies that reduce misuse, and when you reassure students that you know it makes mistakes and that you will take that into account, that you will meet with them, you will use other approaches to determine whether they’ve used AI. In my case, I’m not using it punitively. I am offering students another chance to rewrite. I’m giving them the benefit of the doubt where that’s necessary. If I meet with them and they can explain the essay pretty well, I’m not going to penalize them, even if the writing was flagged as AI. So I think there is a judicious way to use them, but I would also like to see a lot of guardrails around their use.

John: And we have seen a lot of cases where faculty just universally will accuse students of AI use when the detectors flag a high probability. But as you noted, that probably is not appropriate without some additional evidence, is that the approach you’d take?

Anna: Definitely, so I’m using detection, but I let students know that if their work is flagged, that means that we’re going to meet and talk about the essay. And when I meet with them, I let them know that I’m not trusting it, but that I am concerned. I want them to be learning, to be developing their confidence, their voice, and we can work together on that. If there is something that was more AI than they intended or they should have done, then that means they haven’t done the learning yet, and there’s still space for that in the class. And then I’m using it in association with a lot of other things. First and foremost, I’m emphasizing what can I do so that they feel engaged with the class and intrinsically motivated. I’m giving them a lot of choices about essay assignments. We’re talking about how writing helps us develop our thinking skills and how that’s applicable to many other things in life. And I’m checking in with them a lot about their engagement with the assignment. I’m offering a lot of support for their process, a lot of formative assignments along the way that are credit/no credit, a lot of tutoring support, scaffolding for the skills to help them build that confidence in their own voice. And I’m also offering ways to use AI that support their learning, that are not cheating and so kind of trying to guide them into that other path, where it stimulates their thinking rather than replacing it. So I think it all goes together. There’s not one right approach and one wrong approach. There’s a lot of things that can help. And you can get around detection, also. There are tools that will, quote, unquote humanize AI text to get around detectors. You can get around process tracking. You can simulate typing into a Google Doc and make it look like you’ve spent an hour on that Google Doc. So these approaches, they’re not adequate gotchas, and they shouldn’t be used in that spirit. But that said, we need accountability in most areas of human life, even if we mean well, sometimes we need help sticking to that, and we need deterrence for those actions that deviate from the path. And I think AI detection can be that, it can help with that. And as you said, we’ve got quite a situation with online learning, and we may need some proctored assessment to complement it as well going forward. But I think this is a little bit like cybersecurity, so it’s never going to be perfect, and yet you can’t stop working on it. And I think some technical mechanisms to discourage misuse can support educators, can be an ethical tool. So I think we should keep asking for ethical tools that support academic integrity.

Rebecca: Both you and John have talked about online learning and the challenge of assessments [LAUGHTER] in the age of AI, so other than oral exams or in class blue book exams, proctored exams, what else can we do in the age of AI to make sure that we have good assessments for students when these other ones might be a bit vulnerable?

Anna: Well, I think that there’s a great literature on humanizing online learning. People like Michelle pacansky Brock have really shared a whole bunch of wonderful practices that are good pedagogy anyway, so things like collaborative annotation, where students are commenting on the readings for class in a social way and responding to each other. Video and audio assignments, using platforms like VoiceThread or Canvas Studio, so building courses so that they are social in a variety of ways, including discussions, but beyond discussions, I think those things can help. Online learning doesn’t have to be sort of one direction, content delivery and then performance. There’s a lot more possibilities than that. I’m interested in things like lockdown browsers for a little bit more of a guardrail around some online assessment. But I also recognize that’s not secure either. Even with a lockdown browser, you can have another device and have aI going on that device. So I do think we might need some complimentary sort of proctored assessments. There’s this two-lane approach coming out of Australia that says we can have formative assessments where students may or may not use AI, but then we need to validate that there has been learning with some form of secured assessment. And I think we need to keep looking at what that looks like for online courses, balancing things like validity with student access. We got to keep going with online learning, because so many students would not be able to access their education otherwise, and we have to find ways to validate that, and that might mean various forms of observing student learning, close observation, whether that’s proctored assessment in person or some form of online process observation.

John: One of the things that I found somewhat troubling recently is the number of browser extensions that have been developed to automatically detect questions on the screen and to offer to answer those questions. In fact, Chrome provided one for a short period before taking it down. Would a lockdown browser be a strategy to prevent that, or would they not recognize the fact that there is this AI agent hiding in there that is doing the data input, rather than the students themselves?

Anna: Yeah, I have been wanting to test that. I’m not sure if lockdown browsers can block agentic AI browsers like Perplexity Comet or ChatGPT Atlas that will just move around and click and fill out forms for you in the learning management system. I think they might help. But long term, I don’t think teachers should have the burden of trying to design around this. I really think we need the learning management companies and the AI companies to help, to say to these systems, don’t complete work for students. It’s a very simple step that the big AI companies could take to drastically reduce this kind of threat to learning. I think we’ve got to pressure them to do that. And I think there can be mounting public pressure from parents who want their kids to learn, and they want their kids learning to count for something and not be judged against somebody else’s cheating. So I think it’s sort of a common sense societal interest that we’ve really got to push. We’ve got to make space for learning online, and that means don’t let the AI systems jump in and take over. Make sure that it’s the students. And there’s not a whole lot that I think I can do as a teacher to block AI agents. I might be wrong. There are educators experimenting with this. Tim Mousel just released a video where he shows how he built this little system to try to detect if it’s an AI agent and not a student. And maybe that can inspire the learning management companies to do something similar. But I think for most educators, we’re not running the Canvas or the Blackboard, and there’s not too much we can do. Multimodal assignments might slow down an agentic browser, like it’s probably not going to be able to generate a video with my face too easily without a little help. So yes, those things might help a bit, but I don’t think it makes sense to ask teachers to redesign our entire pedagogy when we really don’t have a lot we can even do to block this, and the big companies do have things they can do to block it.

John: We’ll include a link to that video in the show notes. I saw it this morning on LinkedIn myself and I. He was able to block two of the three things he tested with his work. But it’s an escalating battle. We have better things to do with our time as educators than to fight that.

Anna: And why should we throw out pedagogy that works and that is research based? Why should we deprive students of those kinds of learning experiences that we know how to provide? So I really want to protect things that we know are valuable. I want my 14-year old son to have those learning experiences, and teachers shouldn’t have to throw them away. That doesn’t make any sense.

Rebecca: You talked a little bit earlier about ways to preserve academic integrity, but also to motivate students. And you hinted at a few things like process, scaffolding, and things like that. Can you dive a little bit more into what that looks like, maybe in one of your classes?

Anna: Sure. So in writing pedagogy, we’ve been teaching writing processes and strategies for years. So in my class, that starts with collaborative annotation of readings for the class, so students are speaking back to the text in the margins and talking to each other about it, and that starts to lay the foundation for what they want to say in their essays and for making space for their voices. Then there are a lot of low-stakes assignments where they’re brainstorming or they’re writing drafts where they have permission for them to be terrible. We’re reading about writing strategies that help us get past those difficult moments. We’re acknowledging that thinking is hard and sometimes painful or frustrating, and that’s okay, and there are lots of things we can do to seek support. I’m assigning tutor visits as homework. I’m giving lots of chances to review each other’s drafts, to revise, to reflect on AI feedback before there’s a final draft due, and even then, I’m saying, “You can rewrite to raise the grade.” So this is an ongoing process where there’s support all along the way. We’re going to get stuck, but there are a million moves we can make to feel better and to ultimately push our own thinking further in the writing process. And you get credit for all of those things that you do. So I’m hoping that that’s really an affirming and a hopeful orientation to writing that carries a lot of empathy for the moments [LAUGHTER] when it’s hard and onerous, but that ultimately offers a lot of options for every step.

John: Many of the guests we’ve had on the podcast in the past have advocated alternative grading systems as a way of shifting the focus from grades to learning. Have you experimented with that? Or have you talked to many people who have tried this, and has it had an impact on improving student motivation in shifting the focus from just trying to maximize grades to maximize learning?

Anna: Yeah, I think it definitely does help. A proportion of my class is sort of ungraded credit/no credit for process assignments, and then I do have a portion that’s graded. I think it can help. I think you still have to have a certain minimum standard for passing the class, but I think it’s something I’d like to explore further, because I think it does remove a lot of that anxiety for students. The sense of inadequacy is huge in first-year composition classes and the sense of imposter syndrome, and “maybe I don’t belong here, and I should be using big, fancy words.” So, there’s so much sense of inadequacy and fear and imposter syndrome in first-year composition in community colleges where I teach. So you can say to students, “I want to hear your voice and you don’t have to use fancy vocabulary and all of that,” but it’s hard for that to really sink in. I think that ungrading can be part of just affirming whatever it is that students bring to class. But I think also sharing examples with students of student voice, and celebrating that, can be powerful as well, so that it really becomes the students’ space and more on their own terms, and I think ungrading can be part of that.

Rebecca: One of the ways that you’ve talked a little bit about doing this is through the PAIRR process. Can you talk a little bit about this process for our listeners?

Anna: Sure. So PAIRR stands for peer and AI review plus reflection. This was actually developed first at University of California – Davis by Carl Whithaus, Lisa Sperber, and Marit MacArthur, principally, but I was doing something very similar on a parallel track as a volunteer advisor for MyEssayFeedback, which is this not-for-profit app, and then I joined forces with the UC-Davis PAIRR researchers, and we’re now working together because it’s an approach essentially to bringing AI into what we already know about good writing pedagogy and building AI literacy in the process. So basically, you keep what’s good, including peer review and instructor feedback and students drafting, putting together their own draft, and then you bring AI in for feedback that stimulates further revision, and you encourage students to question that feedback and compare it to what they got from their peers, and think about how they feel about that, and what was their actual purpose with the writing. Did the AI feedback get it right, or did it misinterpret what they want to do? So you’re building this habit of skepticism of AI and also building some prompting skills, helping students get the most out of that feedback by pushing back in various ways and asking for more from the system. So that’s the core of the approach. But we also start out with some AI literacy readings, so that students have some basic concepts about what these systems are and AI literacy, including questions around the ethics of AI and how those ethics should influence our decisions about AI use. That’s a big piece of it as well, but really the goal is to support students’ sense of agency around their own words, their own thinking, and their choices around using AI. So it’s a really simple method, and I think, very complementary to existing pedagogical practices. So it’s kind of something that people who are against or afraid of AI can use, and that people who are excited about AI can use. I think of it as kind of a meeting place, this approach.

John: How have students responded to this approach, or the approach that you’ve talked about with MyEssayFeedback?

Anna: So the UC Davis researchers, they have a piece out in Computers and Composition. They found that students overwhelmingly preferred having both peer and AI feedback. They found them the most useful in combination. They didn’t want one or the other. Often the AI feedback reinforced the peer feedback. Sometimes it provided a complementary perspective. They also noted flaws in it, that it wasn’t always right, and so that’s overwhelmingly what I’ve seen as well in student response to MyEssayFeedback. I’ve used it for I think this is the fifth semester, and that’s what we’ve seen in other teachers’ observations and student testimonials on MyEssayFeedback, that it’s sort of like, “Oh, this is a way that AI can support my writing and not interfere and not get in the way. I’m still in a driver’s seat, but I’m getting help.” So I think it’s a really powerful paradigm for students that they appreciate. There’s the concrete help, but there’s also that paradigm shift around what AI can be for them. And we hear that again and again in student comments, and we’re still doing further research, so we’re still looking to improve the process, to improve the prompts that we use, the ways we request AI feedback. We’ve done a lot of testing on those prompts, and we shared them publicly as well. So I think we can keep improving that model and sharing it and inviting others to improve on it as well.

Rebecca: One of the things that I can imagine as a student that would be interesting is when you have peer feedback and you also have AI feedback, there’s sometimes conflicts in that feedback, but there’s also places where that feedback converges, and that happens when you have feedback from multiple people anyways, it’s not always the same, so sometimes that helps, maybe, prioritize what to work on or to focus on.

Anna: Yeah, I think sometimes students don’t know whether to trust the peer feedback or the AI feedback, so it is helpful to see where they overlap with each other. We also designed one of our prompts so that it would try to nudge students to speculate on what the human readers will feel or think or want, to kind of remind them that the point is the human readers’ response, ultimately, not so much does it meet the criteria or the grade, although that’s also part of the feedback, but what are your human readers going to want out of this? And that the AI can help them think about that and remind them of that framework, so it really can complement this human-oriented writing process.

John: We have a lot of faculty, though, who are still a bit reluctant to try anything related to AI, and they just like to bury their heads in the sand, or who are just giving up on a role for faculty in affecting the evolution of how we do this in education, because they feel that it’s being dictated to us from the big tech companies out there, and we don’t have any sort of say. How would you respond to those faculty who are either just ignoring AI or who are perhaps giving up any hope of being involved in how we use these tools?

Anna: Yeah, I would say “I hear you.” That makes sense. It’s a lot of cognitive load and labor to engage with it, to learn about it, to try out new strategies, to understand how people are using it. Those of us who are on social media a lot learning about AI and education often liken it to a fire hose coming at us. There’s constant change, and this is labor. I think it is possible, though, to support faculty, to compensate the labor to develop programs that make space for faculty to learn and to shape and reshape our pedagogy, our AI policies, our own practice in an emergent way, in a collaborative way, that is not dictated by the AI companies or by for profit ed tech. And there have been some great examples of this that I’ve been able to be part of. One program at College of the Siskiyous that has participants get a stipend. They attend meetings, they learn certain things, but they’re developing their own project that they’re later going to share and showcase that builds on what they’ve learned, but that is very much their own. And that’s the kind of community of practice model that has some learning, some outside instruction, but also a lot of faculty making decisions based on their own experience and practice. And I think that’s the kind of model that shows respect for faculty, time, energy, and labor that I feel good about saying, if we do this, then we really can ask faculty to engage, and that will feel very different from the AI company saying, “Look at this cool thing we built. You should be using it in X, Y and Z ways. So do a whole bunch more work.” It’s understandable that people would want to say no to that kind of ask, but I think it is possible to support faculty to engage on our own terms, and that’s what we need to do. I just think faculty empowerment around AI is really something I want to see more of. Those of us who are afraid of or hate AI, those of us who are excited about it, all of the faculty voices are needed to shape what is our society going to look like? What is learning going to look like? We should be the ones shaping that. So I want to see more involvement, more people listening to us, in terms of how these systems are shaped, how the learning management systems are shaped, where the money goes. And I think we don’t need to be intimidated. These systems are very straightforward to interact with. We have the expertise to work well with AI. We have the expertise to give guidance on how it should and shouldn’t be used, and what the systems should do and not do. So really, they need us. Society needs us. Let’s step up to that. Let’s claim that, a more powerful place in shaping our future with AI.

Rebecca: So we always wrap up by asking, what’s next?

Anna: Well, I want to improve on this AI orientation textbook that I’ve done and really share it, and I want to start making more student facing videos. I think one space where a lot of learning happens that both faculty and students haven’t yet tapped into as much as we could is social media, and so I do a lot of engagement with other faculty learning on social media, and I want to reach students on there as well, and be in dialog with students on platforms like YouTube and TikTok, because I think that this is a moment of great vulnerability, risk, possibility, overwhelm, change, and the only way to feel good [LAUGHTER] in those kinds of moments is to sort of turn toward each other, to support each other, and to keep talking through it, and to sort of make space for new practices to emerge. So that’s really where I get my energy and hope. And I think that my wish would be for more faculty and students to get to experience that with each other and find more spaces for that with each other.

John: Excellent. I’m not sure I’m ready to do any of the TikTok dances, but that would be a place to go to reach students. [LAUGHTER]

Anna: Right. And I’m all about like that vulnerable message to students of “I don’t have all the answers, and even I question sometimes whether I should use AI at all, given the ethical concerns, I don’t have the perfect prescription for ethical AI use, but, here’s what I’m learning, here’s what I’m doing now, and why and how you might critique that, but why I’m still doing it.” I think we can be honest and vulnerable with students, and it feels better to me when I do that. I think ultimately, as an educator, it is exciting that this is all changing so fast, and students want to talk about it, they want to learn about it, they want to develop their own opinions about it and responses to it. And that’s where we are, too, as teachers. So that’s the moment where we can really meet students in a really genuine way and be energized together. To be honest about this, I think that’s a good aspect about the moment. There are a lot of challenges, but that is a good thing about where we are today as teachers.

Rebecca: I really appreciate that idea of bringing people together to talk about the issues that are really bothering them or exciting them, or whatever emotion they’re [LAUGHTER] having about it, and that experts and the novices can be together in the same space and learn from each other.

Anna: Yes, AI keeps hijacking our department meetings in the English department. [LAUGHTER] Everybody wants to talk about it at every meeting, whether it’s on the agenda or not. So we need each other. We need those conversations. And there’s so much energy around it. We really can support each other and grow together given that great need.

John: And it’s hijacking pretty much every educational developer conference that I’ve been to in the last three years. [LAUGHTER]

Rebecca: Or podcasts for that matter, John.

Anna: Yeah. [LAUGHTER]

John: Well, thank you. It was great talking to you, and we’re looking forward to more conversations in the future.

Anna: Thank you for your wonderful questions and a wonderful conversation. I appreciate it.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

424. Teaching with AI

The rapid evolution of AI tools provides a challenge for educators exploring educational applications. In this episode, José Antonio Bowen joins us to discuss ways in which faculty and institutions can better prepare students for their future lives and careers in a world in which AI tools are ubiquitous.

José has a joint PhD in musicology and humanities, served for several years as President of Goucher College, and is the author and co-author of several books, including: Teaching Naked: How Moving Technology Out of Your College Classroom Will Improve Student Learning; Teaching Naked Techniques: A Practical Guide to Designing Better Classes; Teaching Change: How to Develop Independent Thinkers Using Relationships, Resilience, and Reflection, and a co-author, with C. Edward Watson, of the first and second editions of Teaching with AI: A Practical Guide to a New Era of Human Learning.

Show Notes

  • Bowen, J. A. (2012). Teaching naked: How moving technology out of your college classroom will improve student learning. John Wiley & Sons.
  • Bowen, J. A., & Watson, C. E. (2017). Teaching naked techniques: A practical guide to designing better classes. John Wiley & Sons.
  • Bowen, J. A. (2021). Teaching change: How to develop independent thinkers using relationships, resilience, and reflection. JHU Press.
  • Bowen, J. A., & Watson, C. E. (2024). Teaching with AI: A practical guide to a new era of human learning. JHU Press.
  • Bowen, J. A., & Watson, C. E. (2026). Teaching with AI: A practical guide to a new era of human learning. 2nd ed. JHU Press.
  • ChatGPT
  • Gemini
  • CoPilot
  • Claude
  • Hume AI
  • Sesame AI
  • Latimer
  • Mike Caufield (2025). “Deep Background GPT Released.” June 16 substack post.
  • Apertus – new Swiss AI platform
  • Storm (from Stanford)
  • weteachwithai.com (website accompanying Teaching with AI.)
  • Undermind
  • FutureHouse
  • AAC&U Institutes
  • Nano Banana AI
  • BoodleBox
  • Poe AI
  • Comet AI browser

Transcript

John: The rapid evolution of AI tools provides a challenge for educators exploring educational applications. In this episode, we discuss a resource that can support faculty in this process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is José Antonio Bowen. He has a joint PhD in musicology and humanities, served for several years as President of Goucher College, and is the author and co-author of several books, including: Teaching Naked: How Moving Technology Out of Your College Classroom Will Improve Student Learning; Teaching Naked Techniques: A Practical Guide to Designing Better Classes; Teaching Change: How to Develop Independent Thinkers Using Relationships, Resilience, and Reflection, and a co-author, with C. Edward Watson, of the first and second editions of Teaching with AI: A Practical Guide to a New Era of Human Learning. Welcome Jose.

José: Thanks. Good to be here.

John: We’re looking forward to this conversation. Today’s teas are:… Are you drinking tea by any chance?

José: Coffee. Sorry.

John: That’s not uncommon, that and water are probably the most common teas.

José: I had tea this morning already, though, so…

John: Okay.

Rebecca: Alright, so you’re not anti tea. Alright. [LAUGHTER]

José: I lived in England for five years. How could I be anti tea? [LAUGHTER]

Rebecca: Right? [LAUGHTER] I have a Constant Comment today.

John: And I have a ginger peach green tea today.

Rebecca: So we’ve invited you here today to discuss the second edition of Teaching with AI. We ran a very successful faculty reading group on the first edition of this book in fall of 2024,
and since the first release of Teaching with AI in late April 2024 we’ve experienced extremely rapid change in the development of generative AI platforms and tools. What were some of the changes that prompted this relatively rapid release of a second edition of the book?

José: Wow. Yes. I mean, first of all, the technology has continued its march, and so we are seeing pretty steady improvement and diversification of kinds of tools and platforms. So we initially started a website with a list of tools. There was the big three and a few others, and now they’re probably 18 to 20 foundation models that are all over the world. When we wrote the first edition, there were not all of these cultural models. There were not Arabic and Chinese models and Spanish models, and there’s a new Swiss model this week, which speaks 1000 languages, we didn’t have Latimer, which is trained on black and brown sources. So there are more models, there’s a lot more tools, and every week seems like we get more of them. But it’s also the case that people’s attitudes have changed, student use has changed, faculty use and resistance have changed. But a lot of it was also that we just spent a lot of time visiting campuses. So I think each of us visited 100 campuses this year alone. So that’s a lot of ideas for assignments, things we hadn’t thought of, that “Oh, I’m using it to do this. I’m trying it to do that.” So we always intended this as a practical guide. There are lots of assignments, lots of assessments, and there’s a lot more of that now. There’s a whole new chapter on chatbots, because that wasn’t really an easy thing to do when the first edition came out, and now there’s hundreds of platforms that do that. And there’s also an extra chapter on literacy and ethics. We were initially a little scared to write a chapter on literacy. I mean, partly, we didn’t have time. I mean, it was just we were like, “We got to get this out.” And also, I think we didn’t know what it meant at the time. It was brand new. And now we have literacy, we have fluency. So this time, there’s enough, I think, research and evidence out there for us to say, “Okay, here’s how people are thinking about this.” And so there’s a whole new chapter on literacy, and there’s an extended chapter on sort of role play and feedback. That’s less about tools, although the voice tools have really… we didn’t have that in the first edition, the ability for voice and also emotional voice tools like Hume and Sesame that really do understand the tension in your voice and respond in an appropriate way that feels and sounds human. So role playing is a bigger thing. I could go on and on and on, but so much has changed. I thought we should do it as an annual, [LAUGHTER] but I don’t think I can do that. I think that’s too much work.

John: And it will probably stabilize to some extent, as most other tools have, at some point in their evolution.

José: We thought so, but the first three years, it certainly hasn’t yet. I’m with you. I sure hope so, because it’s impossible to keep up.

John: In your earlier Teaching Naked books, you focus on the importance of maximizing human connections in the classroom and using appropriate technology for student work mostly outside of the classroom. Do you still advocate a teaching naked approach in the current environment, with the ubiquitous availability of AI tools?

José: Yes, mostly, I think there are going to be places in the classroom where you have to bring AI in, the same way that you can’t teach digital literacy if you don’t have the tools in the classroom. I do think there are a whole class of assignments where I want students to do things in class and talk to each other. An example that I’ve borrowed from Mike Caulfield’s great work, or sort of altered, but used his tool, is that I have students come up with a list of controversial statements. What are some things that might be true, or does the Earth have two moons? And then the students now write a prompt, an 800-word prompt, to test this group of ideas. If I said, “Write an essay. write a prompt,” sounds a little bit more meaningful. I’d let them go do that at home. Go write your prompt, and then they come back. And so now everybody has a prompt, and now I say “So now get up with a partner and pick a controversy and test it with your two prompts. See who wrote the better prompt that actually gets underneath and does this thing well.” Then I show them Mike Caulfield’s Deep Background, and I said, “so his is better.” So I don’t show them the prompt. I just say, “Can you alter your prompt to get it to do this, to make a table, to divide the question into sub questions, to trace back the origins. What are the other things that you didn’t think of? Do you need a chart that with a scale of “Well, some statements might be true on a scale of one to five? This is definitely true, definitely not true. But what about the maybes?” So there’s a lot of active learning in class, but the tool is still with us in class, in the same way that I don’t think we should ban PowerPoint from the classroom, but I do think we should ban PowerPoint slides that have 400 words on them. Put a picture, that’s good, and six words that’s, I think six words still works as a maximum, and then write that. You use that as the way to kind of focus the conversation. “Oh, that’s what we’re talking about. We’ve kind of wandered off, but there’s the picture and the six words.” So I think there’s a place for some use of technology in the classroom, like I always did. But I still think that if you’re going to have students drive to campus, get parking and take the extra investment to be face to face, rather than be online, you have to give them something different, something they cannot get online, and that’s the smell and the feel and the touch and just the presence of other people and the ability to have those kinds of conversations and look how people are responding. I still think we should be using class for that. I do think that, as with the internet and now with AI, we might not need to meet three times a week for 50 minutes, especially if you’ve got over 50 or 60 students. If you’ve got hundreds of students, you do not need to meet together with 600 people. My suggestion is, let those 600 sessions… let that happen online, tape your videos, your lectures, whatever, but then meet with students in small groups of 15 or 20. So maybe that’s only once a week. Maybe I have my 600 students and I have my four TAs. So maybe what we do is we only meet in groups of 12 face to face once a week. We could radically rethink what we do with that face time, and that’s kind of what I proposed in Teaching Naked and so I still think we could do that. And now, instead of those lectures, maybe it’s an AI tutor that does the out-of-class work, but I certainly don’t need the AI tutor in class.

Rebecca: One of the main concerns that faculty raise about generative AI is the concern over academic integrity issues, so faculty sometimes opt or wish to ban the use of generative AI or focus their efforts on trying to detect its use. Why is this not a very useful strategy, and what should faculty maybe be thinking about instead?

José: There are two things. So the first is faculty should be concerned firstly, with learning, not with academic integrity, right? Learning. So if students are not learning, the problem is that they’re not learning, and I need to rethink how they should be learning. The second problem, though, is that the phrase “academic integrity” implies that we have a special integrity. It’s not like your integrity, our integrity, right? And the minute you say that, it’s like, “Well, then in four years, I can forget about it, right?” It’s only academic. So why not just talk about integrity? Why not talk about authorship and the importance of human contributions? And by the way, what we call cheating, business calls progress. So the first thing to realize is that if the product your human students produce is no better than AI. You’re not doing them any favors because they won’t get a job. Yes, there are things… we have to learn how to add, even though the calculator will do that, but there are other reasons for that, right? We’re really learning number sense. But in fact, most accountants are going to use a calculator. They’re not going to add up long columns of numbers. The learning to add is part of the process to get us to a kind of thinking that moves us along. So if we’re clear about, oh, this is the reason you’re learning to add when the student says, “ Well, no accountant is not going to use Excel spreadsheets or computer or calculator, why do I have to learn how to add?” Well, the answer when I was a kid was, so you get the right change when you buy a candy bar at the drugstore, [LAUGHTER] and now they don’t even make pennies. Or they used to say, “Well, the patient will die. If you can’t do long division, you won’t be able to give the right kind of meds.” No doctor anywhere does long division in the hallway on a piece of paper. They all use an app. Now it’s so you don’t put as many zeros on your taxes, right? You want to be able to estimate so that, oh, I put the numbers in wrong. This doesn’t look right. This is too many milligrams. So the ability to have number sense still matters, maybe matters more. The process of learning to do those things matters. Is that really an integrity issue, or is that a learning issue? I’d much rather frame this as a learning issue and demonstrate to students that this learning will be useful. At the same time, I want to raise standards in the same way that a spell checker raised standards. It did not make me a better speller, but it did make my spelling better, and that was sufficient. So now one in three fortune 500 companies require Grammarly for every communication in and outside of the company. So the requirement that you don’t have spelling mistakes is now pretty common, but I don’t grade spelling like I used to. So those 10 points that I used to give for spelling, I now put on the thesis and the thinking, I put it someplace else. So AI is going to make us do the same thing. We have to rethink what is essential for students to do, and how will I encourage them to do it and motivate students to do it? And how will I raise standards so in fact the end result is something that neither AI nor student alone could do, in the same way that my accountant could not do my 700 pages of taxes without a computer program. She simply couldn’t do it, but she does know how to add, I think. [LAUGHTER]

John: But she doesn’t need to.

José: We do talk numbers all the time, and she’s estimating things. That kind of thing happens all the time. I’m glad.

John: A related concern, though, is that perhaps students are using AI too much at the early levels of skill development, so that they’re not developing critical thinking skills and disciplinary expertise to be able to evaluate the output of generative AI. What are some strategies that we can use to help students develop these skills while working on learning how to use AI tools effectively?

José: The first part, is yes, they absolutely are overusing it. But back to your cheating question, instead of calling that cheating, maybe we should call that overreliance or misuse. Is it really fraud? Is it really intentional deception? Cheating is a very loaded term, and so a student might not necessarily know when they’re overusing it or how they’re undermining their own learning by over reliance or misuse. On a practical level, there are two things that you need to use AI well. One is expertise does matter, that being able to say, “Huh? I don’t think that’s right.” I often say that AI literacy is fundamentally two things. One is asking better questions, and the other is evaluating answers. The two things that we all do, the core of a liberal education has been those two things for 1000s of years, so we all already teach AI literacy, but how do I get expertise if I have this tool that pretends to have expertise? It’s kind of the same question as the calculator problem. And I know the calculator is not a perfect example, but it’s a good example of this. I still need to know how to add to know when the calculator might have made a mistake, and so even though the calculator can do the work, I need to have that experience. So the calculator challenged us in two ways. One is, do I still need to teach long division and addition of long, long lists of numbers? The answer to both of those is not much. I should probably do a little bit, but I don’t have to spend six months like I did just adding up long series of columns. So I have to change what I teach, but I also have to think about the sequence. When do I introduce the calculator? So the question for us is not, do I introduce AI? The question is, when and how and what are the skills that I still need to preserve? Spelling is another one of those. I’m able to write books with a spell checker, and if I had to look up words in a dictionary still like it was 1970, I would be much slower… if I was still using index cards for my bibliography… So, I no longer have to teach the card catalog. Are there skills that I can not teach in order to add the other one? Academics, we love addition, we hate subtraction. [LAUGHTER] We don’t like taking stuff away, but we’re going to have to take something away. But that’s a big question. How do I do that? But I do think we’ve been teaching digital literacy for a while, and we should add AI to that. So the question is, is this true? Although, again, Mike Caulfield’s version of this is, “is this what you think it is?” is a great better question. Is this what you think it is, especially now that you can’t tell because it has six fingers. There’s no more “Oh, it has six fingers. I’m going to show you how to identify AI pictures.” Well, that doesn’t work anymore. So understanding what’s the source? What are the kinds of claims? Do other people say this too? So that’s probably a separate skill from expertise, just finding out what’s true. And again, all of us are doing it on a daily basis, because the news is full of stuff that it’s like, “Is that really true?” And “What’s the source for that?” and “is that picture really real?” So we’re all having to do that. So that should be easy, but the expertise is the harder part. How do I get you to be an expert if the junior level jobs and internships are going away. And so I think that that is a primary question that we should be thinking about. And my rudimentary answer is we need a lot more talk about motivation, a lot more talk about process that “yes, you will have a calculator, but the process of learning to add and to multiply is valuable, and here is why.” And so here’s what we’re going to do. And then I promise you, we will introduce the calculator. It is coming, but you will not be as good a user of that if you don’t first do this. And my analogy to that, so I teach music, and so people wanted to learn how to play jazz pian, and so you would give them these timed tests. I’d say, “okay, E flat minor, what key are we in?” And they’d have to write that. And so those were, like, those old multiplication, you know, 60 seconds, and you’ve got to, like, scribble as fast as you can. “I want to play the piano, why am I…” I said because, if you have to think about what key E flat minor is in, and how many flats that is, the band is already… that time has to be immediately in your fingers. And so in order to do that, I’m going to give you what feels like really rote memorization. Five flats, by the way, is the answer. And unless you see E flat minor and immediately go three, two, and you have that hand ready to go, you can’t converse with me in jazz. So I had to explain to students why I was making them do these, what appeared to be really dumb, like rote third grade math tests, because that will allow you to speak with me. When you see a piece of paper and we’re playing in real time and it’s going, click, click, click, click, E flat minor, F minor, you can change key. You know how to do that because it’s automatic, and so I have to make it automatic for you. So notice how much time I spent convincing you that you had to know how many flats. So that explanation is not something we’re always good at. So that’s not a great answer, but I think it’s the right question.

Rebecca: What you’re talking about is reminding me of a conversation I had with my graduate student a couple weeks ago about changing disciplines and fields and learning the language of a discipline, and how much that actually allowed him to communicate about what he wanted to communicate about this whole time. And so he’s worked on a really great presentation about this [LAUGHTER] exact topic that kind of becoming fluent in the language of the discipline allows you to communicate things that you couldn’t communicate before.

José: And that you can do with AI. There’s a great tool from Stanford called Storm that actually has a little button that allows you to change the disciplinary focus. And so one of my assignments is,”Okay, so pick a topic, pick a question, and go to Storm and then look at it through three disciplinary lenses, and then write an essay about what is the difference? What did you learn? What are the differences in how anthropologists and historians think about this question? So I do think that’s an important piece. And again, it’s often not explicit. We should make it more explicit. In the same way we should make the critical thinking discussion more explicit. For years, we’ve just said, “Oh, they’re learning that. They’re learning critical thinking.” It’s like, “ah?” We did a great study four years ago, and the students were like, “Critical thinking, what’s that?” You just spent four years doing it. It’s like, “but you never use those words.” And so adding the words, “This is critical thinking,” turned out to really help students. So we’ve got to be more explicit about that disciplinary knowledge and why it’s useful and how we’re learning it. And I remember the light bulb in graduate school. I remember sitting in that talk when it was a guest speaker, and all my professors were there trying to outdo each other, and it’s so “Civilization and culture.” And the other guy said “Apollonian and Dionysian.” And it’s like, “Oh, they just want to use their own terms.” It’s like, “Oh, my God, they’re not really arguing. They’re just like, trying to see who’s top dog. And so I remember [LAUGHTER] that light bulb going, “Ah, please.” [LAUGHTER]

Rebecca: Can students use of AI tools help them reduce the equity gaps that students have faced in their prior learning experiences. This kind of goes back to John’s earlier question a little bit.

José: Yeah, so I think the answer is yes, but we have to be careful. So the first is that AI has bias. We have to make sure that everybody knows this, but it’s also the case that humans have bias, and if we were ignoring human bias, we weren’t doing anybody a favor either. So it turns out that AI bias is a little easier to fix if you know what you’re looking for. If I know that AI tells me I should be earning less money because I’m a woman, I need to tell it to respond to my resume like a man. [LAUGHTER] How much should a man make in this job? And then that’s a useful tool. So understanding the bias is even more important than it used to be, because now I can say, “Oh, well, how could I correct this?” My example is, if I said to you, “so rank the candidates for the new position of history professor that you’re searching for,” can you ignore the prestige of where they got their PhD? Uhhh, that’d be “no,” because you’re a human. It’s like, you know, don’t think about the cookies. You can’t not think about something I just mentioned. But in AI, I could say, just rank the candidates based solely on the quality of their teaching, review the faculty based solely on the quality of student feedback. I can do things as a check on equity. I can also do things like so I can make you a bot. So you didn’t take calculus that’s required for this course, or most people have had it. So I’ve created this bot that will help you figure out what you’re missing, or create a bot that will help me, I didn’t take biology before chemistry, and I could do individualized tools. And AI is pretty good at learning, “Well, you tend to make this mistake a lot. This is a grammatical mistake you always make, and so you made this in the last paper,” and one that people don’t think of, which is that most of us, well, all of us, because I do it too, and everybody does it. All of us provide students with way too much feedback at once. [LAUGHTER] Here are 37 things to fix about your paper. And the truth is, you can only do one, one at a time. After you fix that. I’ll give you something else. This hit me when I was trying to relearn tennis a few summers ago, and it was like, move your feet, hold the hand, the grip. It’s like, “Shut up. Just give me some more balls. One thing at a time, let me just put my feet right, and then… So I use AI to take my feedback and what is the most important one thing the students should fix for next time, and then save the rest, because I overdo it. And so I can tell the AI not to give the student too much feedback at once, and it can do it. And I have a… all of us, I think… have a really hard time doing that. So the potential for AI to close equity gaps, to help us see bias, to make more ethical decisions, is there, but it was not going to happen automatically. We have to be very, very careful and very aware of the kinds of things AI gets wrong and the kinds of biases that are built into it, in order to make sure that doesn’t happen. But as a check on human bias, it’s really a pretty amazing new tool that we didn’t used to have.

John: In terms of that issue of feedback, when students are given 20 or 30 different things to correct or to improve in a paper and assignment, they will often choose whatever is the easiest, which is not always what is the most important. And that idea of focusing on the most critical or most important type of feedback makes a lot of sense.

José: Well, I can give an example. So this is a real example from a PhD student who’s like, comes back to me and says, “Oh my god, I handed to my dissertation to my three readers, and I got three different types of feedback. My God, they don’t agree. Is this normal for you people?” Uh, yes, [LAUGHTER] you got contradictory feedback. [LAUGHTER] Congratulations. Welcome to the club. And she’s like, “Well, what do I do? One of them wants me to cut chapter two. The other one wants more. What do I do?” I said, “So, you put the whole thing into AI with the three letters, and you say, ‘so, make a table list all of the things they want me to fix, from easiest to fix to hardest to fix, and then list: Does everybody agree that should be fixed?’ Check, and then another column, if they disagree. What’s the easiest way to make them both a little happy?” Graduation. So easiest to fix, the hardest to fix. So fix the easy stuff first. That’s fine, but an AI here could keep you on track to make sure you actually prioritize. So yeah, here are the three easy things you should fix, because that spelling mistake was terrible. But here is the real issue, the one real thing you should focus on. And I would say students leave a conference with me, and I didn’t know this until AI, but now they go to the AI bot and they say, “Okay, what’s the one thing I really need to fix?” Oh, my God, I didn’t make that clear. And I know that because I can see them talking to my bot about the conversation [LAUGHTER] they had with me.

John: With all the changes that are occurring in terms of the development of AI, we’re beginning to see some shifts in labor markets that are not that well defined yet, but it’s very likely that many of the jobs that students were preparing for in the past may not be there or may not be there in the same way going forward, and jobs are very likely to be changing in the future. I think you’ve already partly addressed this, but how can we prepare students to thrive in a world where the nature of the tasks that they’ll be doing might be changing fairly dramatically in response to more widespread use of AI in many occupations.

José: So, you know, I’ve been saying this for a while, but the future has been unknown for a long time. And so first you said, “Look, your major has never mattered less. I don’t know what the jobs are going to be in five years.” In fact, when I wrote Teaching Naked, my daughter was still in college, and then she became a director of social media, which was not a joke I put in the book because it hadn’t happened yet, but there’s like, oh, wait a second, you could be a director… this didn’t exist five years ago, and now you can direct it. So this isn’t new, that new jobs are changing and being created all the time. The internet destroyed jobs and created lots of IT jobs, but it’s impossible to guess what those are going to be. So in some ways, that’s great news for those of us in the humanities and liberal arts, because it means that the thinking is going to matter. Good use of AI is creative thinking, asking better questions, evaluating answers. Computer Science, it’s hard to know which exactly to do. But that was also true 20 years ago, your degree in the iPhone I was not going to be useful forever, unless you could figure out they’re going to be new iPhones. And so adaptability, flexibility, has always mattered. It’s just it was slower, and now we’re seeing it’s happening more quickly. It is very hard to know what the job market is going to look like, whether it’s going to be we all work three day weeks, whether certain classes of jobs go away. Is AI really taking jobs? It’s clearly going to take some jobs, but the research says that every job is going to change. There’s at least one task. And so for faculty, my first thing is find out what you no longer need to do, whether it’s making a makeup test, grading, drafts of accreditation reports, assessment of student for accreditation, assessing student learning. There’s got to be something that you hate to do that AI could already do better now. My doctor no longer sits at the computer and types, my doctor now gives me eye contact, which is a little creepy, [LAUGHTER] but the evidence says that doctors who use AI transcribers to do the nurse’s notes, the insurance notes, whatever, that they have more cognitive load to focus on patients, and they’re happier, there’s less burnout. So this is a good thing, right? Because no doctor went to insurance school, they went to medical school, so insurance notes are not part of the job I love. So could AI help with that? And my doctor says the AI is now better, the first six months, he checked it and checked it, but it turned out he was looking stuff up, because the AI is better at it. So what could faculty do that AI is already better at? The department schedule, chairs? Because that’s what’s going to happen, or the bibliography, the literature review, keeping track of what are the latest trends. So that should give you time to do something else. And for faculty that will learn a bit about AI, because you’ve got a few hours back, but I think for students, the question is, how can you be constantly thinking about what else you could do, how your job is changing. I’ll give you one example. So in the old days, if I had a new product or I needed to know how to price my product, I would send a group of interns out to various cities to do focus groups. Go find 12 people at the supermarket and give them a taste, see if they would pay $4.99 for 12 ounces of this. Now I can do that with AI. You’re a 35 year old suburban soccer mom and you live in Oklahoma, and it can give you the synthetic personas to do marketing. So on the one hand you say, “Oh, I’m going to lose my job.” Well, no, because you’re going to go back to your boss and say, Okay, boss, so instead of doing 10 focus groups. I did 1000. I did them in every state, and I did them by year. I did them for every year between 30 and 40, and I did them for rural. And so that’s actually happening. So somebody is setting up synthetic personas to do marketing focus groups for their products, and they’re now able to do work of much higher quality than they were. So there’s still a job, but the job is now different than it was, and so I would classify that as creative thinking that’s raising standards but also how could I do this, not just more efficiently, how could I do this better? What do I want? I want to know how much to charge for my product. What could I do to get better data? And the answer is, “Oh, I could create 1000 synthetic AI personas to do this market research and go back to my boss with way more data than she was expecting.” Those are the assignments I like, too, right? So I might give students an assignment and say, “Okay, so you need to figure out a way to use AI to do something you couldn’t do by yourself. How do you make this better? What else could you now do that you couldn’t do before?” And that’s thinking, that’s the critical thinking, part. And so the process becomes important. It’s not just did you produce this thing, but how did you think about it? How did you make sure it was right? How did you validate? Those are going to be really important questions in the future. Did you validate? Did you make it better? We should be able to do that. We love that stuff.

Rebecca: One of the things that we’ve talked a bit about today is the rapid amount of change, change in what jobs look like, changes in tools. It’s hard to keep up…

José: Impossible.

Rebecca: Right. So what are some low effort and high impact AI practices that faculty might explore if they want to get started, but haven’t quite gotten there yet.

José: Well, the first thing I think you need to do is to try it in your own life. I actually think that just try an AI browser, try shopping, trying to see if AI could help you buy a new subwoofer, buy appropriate gifts for your kids or whatever. I have a really hard relative, Uncle Larry, plays the ukulele, and he likes whiskey, but I got him that last year. Give me 10 ideas for new gifts. Link me to the website. Shopping is really changing, so just finding a way to use it. So we have this website: weteachwithai.com. We have lots of models, lots of tools. Try a literature search with Undermind. That will change your life. Try FutureHouse, if you’ve not tried that, if you’re a STEM person. Try some of these tools. And start to think, “Aha.” Because if you try a disciplinary tool, right? We talked about disciplines earlier, you’re think, “Aha, this is going to change how experiments are run. This is going to change how literature search is done. This is going to change how verification is done.” And now you have the basis for rethinking your assignments. But the easiest thing for faculty to do is to take your syllabus, and if you’re nervous about putting your whole syllabus in, then start with your academic integrity policy, take your cheating policy, put it into AI and say, evaluate this as if you were… and then put the description of the students in your class, right, 18-year olds at Nebraska City College, whatever, put in the description of your students and then say what might be insensitive or unclear. How could I make this policy more motivating or more meaningful to and then describe your students. That works for every assignment, it works for your syllabus. In fact, one thing you can do is build a course profile. So I build a course profile. So I have 64 students. I have three who have this accommodation. I have this student who didn’t take calculus. I put in the details. Then maybe even I asked them what they like. I’ll come back to that. And then I say, so is there anything about this reading list or this assignment that is not going to feel inclusive to my students. I’m teaching engineering, and I didn’t used to have a lot of women. Now I have more women, but I’m using the same syllabus from 1975, could I change some of the examples so they might be more inclusive of a more international or gender diverse group of students? So get feedback from your students. It’s a focus group, essentially, right? Because you don’t know… you’ve been here too long. You like school so much, you’re still here. You’re weird, all of those things. So if I want to know how an 18-year old would respond to my assignment, I need to find an 18-year old. So those of us who had kids go through college found that very useful. It’s like, hey, what do you think of this? But once they graduate, it’s like, “Oh, my focus group is gone.” So now I could use AI to get feedback. So a course profile is a great way to do that, and that’s low-hanging fruit.

John: So you’ve suggested a few ways in which faculty can use AI to improve their teaching. Are there other things they can do to update their courses, to make them more inclusive, etc?

José: Yeah, I have a prompt on the weteachwithAI website that says. “So here’s my syllabus. I want to update my syllabus. I’m really skeptical about AI, but I know students are using AI. So what are some things I need to know? How do I update my policies? You give me 10 ideas for how to create new assignments that might prepare students for this new world. And by the way, you’re an expert.” Learning to prompt, right, recognizing that you have to tell it it’s an expert, is useful. So I think that’s a pretty easy thing to do. Another thing you could do is customize your assignments. Ask AI, “I need an idea for a 10-minute activity to break up this lecture. I need an idea for new assignments, or I have this assignment, but I have 50 students. I know they have different interests. Not all of them are majors, some of them are pre-med. Could you create different versions of this, one version of the problem set for pre-meds, one for this, one for that?” Because nobody really cares about Train A leaving the station and going 20 miles an hour faster than train…. Right? Nobody cares. So why not? The Republicans are registering 20 voters an hour and the Democrats are registering… It’s the same math, but the research says the students were going to do that problem more because it feels like a poli sci problem, or the wide receiver you’re covering runs 30 miles an hour and you’re only…, that feels more like a football problem. And so it turns out that matters. So you could even go to the extent of customizing every assignment for every student. Do a survey before class or the first day of class. So give me an alias. I don’t have to put your name in the AI, but give me an alias and then tell me what motivates you to do your best work. What do you want to be when you grow up? What are your favorite colors? What’s your favorite sports team? What are your favorite hobbies? And then I feed my generic assignment with that and the spreadsheet, I say, now create an individualized assignment with a unique link that you will then email to all of the students individually, that’s a customized version of this assignment. You can also do this in class. So an example: “So write a five page paper on Hamlet.” Okay, so students are like, “Oh, really?” So first what I want you to do is, I want you to tell the AI a bit about what really motivates you. Tell it about your passions for baseball or dancing or whatever it is, and then ask it, how could you customize this assignment so that you could write a paper about Hamlet that overlaps with what really motivates you? Could you write a paper on Hamlet and baseball or Hamlet, using analogies for whatever, and so AI is actually really good at that creative back and forth, because it hallucinates. It makes stuff up. It’s creative. So that’s the kind of way where I can get students to increase their motivation, more customization of their assignment, because they’re using an AI to create and again, I couldn’t do that at scale with 600 students, but with AI, I can customize 600 assignments just as fast as I can customize three.

Rebecca: You’re starting to hint at this, but if faculty implement some of the strategies you talked about today and are in your book, what types of changes would you expect to see in your classroom and in the student learning outcomes?

José: Well, I think that we have two problems. One is that we charge a lot of money for something that’s now out of date. If you’re still delivering content in a lecture, there are cheaper ways to do that. If you’re giving a credential, there are cheaper ways to get the credential. But if what we’re teaching is an experience of critical thinking, of being with other people, of learning about diversity, we should double down on that part, and we can outsource again, not unlike I talked about in Teaching Naked and the internet, but it didn’t quite happen with the internet, right? But we could outsource some of the content acquisition. They could do things outside of class to get first contact with content. And so I do think that classrooms will probably physically change. We do not need 600-seat lecture halls anymore. We need smaller classrooms. We may not need to meet as often with students, but we do want to meet them in smaller groups. So I think that’s going to change, but I think more face-to-face human reactions. If you have an asynchronous course that was mostly online or asynchronous, most of those students are within a few 100 miles. They are fairly local. I was a consultant for one state, and they were having this problem because the students, they’re spread out. And I said, “Well, why are you moving the students around? Move the faculty around. You have seven campuses in your state. The faculty could go to a different campus a day and meet with students, and then the rest of it is online already.” So we have to think differently about what our classrooms look like, what we do in those classrooms. But I think that’s going to be the harder change. Certainly, AI could do grading of papers. There’s a whole lot of caveats to that, but AI could do content delivery. Instead of having to read the chapter, work with my bot that’s interactive, an interactive textbook instead of a midterm. Maybe I have you talk to Einstein, and when Einstein thinks you know enough about thermodynamics, the chat bot puts a grade in the grade book. So those things I think could be automated if we do them carefully. The bit we can’t automate is the “I believe you can do it” piece. “I believe in you, and I’m a human. I believe in you. You can do this.” That bit we can’t outsource. And so the question is, how do we make sure we have time for that. My example is advising. So we think we’re advising. The student comes into our office for a 20-minute meeting, and we go, you don’t have a prerequisite, but you want to study abroad next spring, so you have to take this course now. Oh, you don’t like vampires, and you want to switch? Well, let’s find another sec, right? And we spend 20 minutes adjusting their schedule, and it’s like, “Oh, bye.” [LAUGHTER] And we never got to ask them, “How are ya? How do you feel? Do you have friends?” That’s the mentoring piece. And what we do often is we do administration stuff. Is your schedule right? Do you have the right prerequisite? I have to sign this. That’s administration. So if we separate administration from real mentoring, perhaps I now have a 20-minute mentoring conversation with you and say, “You know so AI is going to make sure you have the right prerequisites, and it’s going to adjust your schedule. And so I’m not going to do that. How are you?” Not everybody can do that, by the way. We have some faculty who are not good at relationships. You used to be able to skate by with, “I’m a content expert. I got my three Nobel Prizes. I’m just going to lecture.” I think that’s going to be harder. So I think that we’re going to see changes to everything about the campus, how it looks physically, how we divide up our time, what people do with students. And I do finally think the stand and deliver lecture is probably going to finally, maybe, go away. Maybe.

John: Turning just a little bit to institutional policies. Are there any sort of policies you’d recommend for institutions in terms of institutional policies with AI. Many campuses have introduced syllabus requirements, that there be an AI statement or something to that effect. Some campuses have gone a little bit further. Are there any guidelines that you’d suggest for campuses that would involve best practices, currently, in terms of AI use?

José: Yeah. So the good news is, the AAC&U has been doing this institute. This is, I think, the third year, and so we have 180 campuses, so we’re seeing a lot of what people are doing. So the first is, last year, everybody was worried about policy. So I would worry less about policy, because policy is changing. It’s not a bad idea to make sure everybody has a syllabus statement, but lots of people will just put “No AI,” and then they think they’re done. So I actually think the first place to start is with control. So I call this CLEAR. So C is for controls. Do I have a safe AI? Do I know which AI i can use? Can I put student data in? Can I not? Do I have to use CoPilot? Or if I’m not going to use CoPilot, what can I use? And what data can I put in there? Because people are confused and they’re told, just use CoPilot… But I want to use Nano Banana. Well, that’s okay, as long as you’re not putting student data in. So understanding what the controls are. L, is for learning. You’ve got to give them places to play. E is for examples. How are you using it in your course? And then A is for adoption. So we need some mechanism for Okay, let’s run a pilot. Let’s do that. Let’s try that. And then R is for review. Did this help or did it hurt? Oh, that was bad. But,there’s going to be failure. This is not all going to work, but in order to figure out what does work, just like anybody who builds… I think it’s Adidas who has a failure funeral when shoes don’t work, they acknowledge that not every pair of shoes is going to be a best seller, not everything we do is going to work. So we have to be able to experiment and try and forgive each other, and be able to have a climate where we know what’s safe. We have some examples, we’re all learning, and then we review to say, “Oh, we tried it twice, because you can’t try it once.” I mean, some things you can, but a lot of the times where we think, Oh, I tried this new thing in class and it didn’t work. So yeah, that’s because the first time you did it, you were crap at it. You’ve been lecturing for 20 years. And so you tried this new active learning thing, and it didn’t work because you didn’t introduce it right, you didn’t set it up right? You aren’t good at it. Try it again. [LAUGHTER] You’ll be better at it next time. And so I think that for teaching, we often think that teaching is just content. And I discovered when I started doing corporate work and I had to do the same 3-hour presentation 35 times in a row, [LAUGHTER] that it got better by the 34th or the 35th time, that actually practicing… I wasn’t changing the content. I was just getting better at the delivery and the management of the client, all those sorts of things. And that rarely happens in higher ed, where we do the same thing over and over again. We get feedback, and we try to make the activity and the learning better, and not just, “Oh, well, I’ll do that next year and try some new thing.” So I would worry a little less about policy and a little more about building the right culture where people are having the right conversations or experimenting. I do think we’re starting to have the curriculum conversation, which is, where does it go? I think people now recognize, okay, I can’t graduate students without any AI. That’s got to be someplace. So the first place to start is the senior seminar, because they’re about to go look for jobs, and so they should know how to do a practice mock interview with an AI before they go: give me a 30-minute mock interview in the format that Microsoft uses for my interview next Tuesday. Here’s the job description. That’s an easy thing to do with an AI and have students practice. But it’s also, read between the lines of this job description to tell me what they’re really looking for, so I can write a better cover letter. AI can read between the lines. So that’s cool. And also, how is the discipline changing? Think about it, you just spent four years as an accounting major, and then Excel spreadsheet gets introduced when you’re a junior, but the curriculum can’t catch up because, like, well, we don’t teach that, because it didn’t exist when you started. Four years from now, maybe. So the seniors’ capstone is the place to say, well, you know what, this new thing just happened. It’s called a spreadsheet, and I’m going to introduce you to the spreadsheet, because you’re an accountant. So that’s a good place to say, Okay, I still need to learn how to do all those other skills by hand, but before you graduate, and that means you’ve got at least one person in every department who is having to keep up with what is going to happen when our students graduate, and what are they no longer going to be asked to do, and what new thing will they be asked to do that we might need to prepare them for. So, senior seminar is a great thing to say, “Let’s put some AI there.” Probably at the freshman year too, about… again, I would redefine cheating. I would think about it as what’s misuse and what’s really fraud. Why does integrity matter? Why do credits matter? It’s the same thing with the bibliography. That’s not about academic integrity. You go to the movies and what do they do at the end? They show you the credits. Is that academic integrity? No, that’s the movies, right? But telling people who gets credit for this matters, and students get that, right? They really care about that. If you steal somebody else’s TikTok, that’s not academic integrity, but it matters. So there are probably other places you need to do this, but I don’t think you can have one policy for everybody. I am noticing that some universities are now having a policy that you can no longer ban the use of AI for specific things. In other words, you can ban AI, but you can’t ban it for feedback, or you can’t ban it for brainstorming. So they’re limiting, because students were going from all AI to no AI, and it’s like, “I’m confused.” So nobody can ban AI for everything, in the same way that you can’t ban a spell checker, right? We don’t do that anymore. We used to, we used to ban pencils and erasers too. We didn’t like the typewriter either. But we’ve had a long history of this. It’s going to change everything. Yep, and it did.

Rebecca: So going back to your idea of controls, rather than really thinking about policies, but giving some boundaries on things, you mentioned, identifying specific tools that maybe people are allowed to use, what kind of data they can put into such tools. Paid services often have more services or more capabilities, and so in the interest of equity, should institutions adopt some sort of standardized tool that everyone then has access to?

José: Ah… it’s a lot of money. So I do think that we need to think carefully about those issues. It may be the case. The problem is that Microsoft CoPilot is expensive and you buy one tool. So I think the first thing is for people to understand what happens to data when you put it in if you’re already using Microsoft on your computer, and you wrote your dissertation on Microsoft and you use iCloud, or you used Microsoft 365, OneDrive, or you used Google Docs to write your dissertation, it’s already in the AI. You have much bigger privacy concerns when you talk in your car. There are other things that we need to talk about for privacy. So I think the first step is helping people understand what does happen to your data when you put it in an AI and which AI. And by the way, there are these controls. If you open your settings, you can turn some things off… not sure I trust them… but you need to know that the default in Claude is off, but the default in ChatGPT is on, and that will probably change tomorrow. So you need to be checking your settings, and you need to be thinking about what kind of data is it ethical and legal to put in here? And do I want to do that? If I do that, then there are a lot of free tools I could use. So I think step one is not saying, “Do whatever you want, but only do it in this tool,” because then everybody’s an idiot. Much better to educate people, that’s what we do, and to say so let me educate you as to what the tools do and how they do it, and what are the parameters. And then you can make informed decisions about whether or not you want to use free Gemini twice a day, which is all [LAUGHTER] you can use in the best model, but at least you have that. And then you can switch to ChatGPT for your three times a day or et cetera. And we’re going to have this model, and you can always use this model for this kind of thing. But I think if you don’t also educate people as to how this works, then having one tool for everybody is just a waste of money, because not everybody’s going to use it. People are going to use their favorite tool anyway, and they’re going to use it poorly and with risk. So the first question, the way to reduce your liability is to educate, not to restrict. And so I would start with that, I like BoodleBox because I think the pay structure is reasonable. It’s a used-based model. It gives you access to lots of different models, it’s not just only ChatGPT or Gemini, because one of the things we now know is that Claude is better at Excel than CoPilot, and that it used to be the best writer, but now ChatGPT is a better writer, and Claude is a little verbose, and Gemini does other more…. So they are distinguishing themselves. They’re doing different things. And of course, we have all the millions of APIs that really do different things, and so I don’t think, for the moment, anybody can get by with just using one model. I mean, yeah, you should try making a picture in Gemini and writing text. That’s good. But we’re in the AskJeeves, AltaVista phase. [LAUGHTER] I don’t know where we’re going to end up. And so for the moment, it’s useful to try a couple of things if it doesn’t work in one try another, but to understand how data and privacy work.

John: And that reminds me of Dogpile, I think that was the name of it, where if you put a query in there, it would search all of the major tools, [LAUGHTER] which sounds like perhaps the best way of approaching this now.

José: Yeah, and there are tools that do that. POE will do that, but I also think that it’s not just about getting in it because you have to follow up. Iteration is a part of good usage. And so picking the right tool is important, but mostly following up. I was doing some shopping the other day, and I had a really odd problem. I have this old set of speakers and an amp from the 1990s so I needed a new piece of equipment. So I tried Gemini, ChatGPT. I was using Comet. I tried different things, and I didn’t know what the answer should be, but I got pretty good advice. But let me just try this over here. Let me try changing the prompt a little bit, so I think that sort of skill is going to be necessary for everybody to manage, in the same way that digital literacy, not everything you see on the internet is true.

Rebecca: What?

José: I know, I know. It’s hard.

John: Well, we always end with the question: “What’s next?” …which is an especially important question in addressing AI.

José: Oh gosh, yeah, nap would be the right thing. Well, here’s what I think. So I think what’s next is that we’re about to see lots of new competition from places you didn’t expect. This one. I do know, we’re going to see a $6,000 eMBA from an accredited university rolled out in the next couple of weeks, the whole degree. So it’s the professors have been turned into AI avatars, so it’s scalable. So it’s cheaper than your MBA [LAUGHTER] or even your eMBA, and they’re hoping to scale it. So we’re going to see degrees that are totally AI, that are going to be cheap and scalable. And this is something new, because traditionally in strategy, you can either be customized and expensive or low cost. Can’t be both. But AI allows you to be low cost and customized. So your new competition is an entirely new class of things. On the other end, we’re going to see boutique institutions. You’re going to see Harvard. They don’t have to change because they’re Harvard. So we have in-person classes. We do this, but they’re expensive. So the question is, for most of us, where in that spectrum are you going to be? Most of us are not Harvard, so we can’t charge what they charge, and then have boutique. So are we organic vegan boutique, or are we, you know, mass? So most of us are going to be neither extreme, but those two things are coming at us, and so the market is about to differentiate or spread. And so for most places, thinking about what kind of institution should we be? Where are we going to plant our flag? And it’s not whether or not we’re going to have AI for most of us, it’s how much and where. Harvard could ban AI and get away with it and give you a four-year degree with no AI because they’re selling a credential, but most of us cannot do that.

Rebecca: Well, thank you so much. This has been a great conversation. We appreciate you joining us today.

José: I just make stuff up as I go along. [LAUGHTER]

Rebecca: So does AI. [LAUGHTER]

José: And that’s why we’re both creative people [LAUGHTER], or creative entities, I guess.

Rebecca: Beings?

José: I don’t know. [LAUGHTER]

Rebecca: I don’t know. [LAUGHTER]

John: And by the time we release this, the second edition of your book will be out. In fact, my copy is arriving in my mail tomorrow. I’m looking forward to the second edition.

José: Yeah, December 2 is the drop date, so Amazon should switch over everything, it should switch to the second edition tomorrow, and there’s a lot more. We didn’t think it was going to be 40% new, but it is. We thought, oh, we’ll make some updates and we’ll fix this. It’s like, “Oh God, scrap that, whole new chapter.” So it’s a thicker book with a lot more than we thought it was going to be.

John: As I think has happened with AI in general, it’s a lot more than many people thought it was going to be when it first appeared.

José: There you go.

John: Thank you.

Rebecca: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Ryan Metzler.

[MUSIC]

423. Using AI for Project-Based Learning

Students are motivated to learn when they have autonomy and see the purpose in what they are learning. In this episode, Tara Chklovski joins us to discuss a curriculum in which students use AI tools to solve challenging problems in their communities. Tara is the founder and CEO of Technovation, a nonprofit developer of a curriculum used in over 160 countries and reaching over 400,000 students, to prepare young women for careers in technology.

Transcript

John: Students are motivated to learn when they have autonomy and see the purpose in what they are learning. In this episode, we examine a curriculum in which students use AI tools to solve challenging problems in their communities.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Tara Chklovski, the founder and CEO of Technovation, a nonprofit developer of a curriculum used in over 160 countries and reaching over 400,000 students, to prepare young women for careers in technology. The current focus of Technovation is on teaching students to use AI to solve real-world problems in their communities. Welcome, Tara.

Tara: Thank you, Rebecca, thank you for having me.

John: Our teas today are… Tara, are you drinking tea?

Tara: Yes, I am. [LAUGHTER]

John: What kind of tea are you drinking?

Tara: It’s Costco’s green tea. I love it.

John: Very good.

Rebecca: Excellent. I have a Twinings English breakfast that I actually got when I was in London.

John: I have Twinings English breakfast that I got delivered to my house by Amazon. [LAUGHTER]

Rebecca: Actually, I made a whole pot this time, John, like it’s the good stuff. So we’ve invited you here today to discuss Technovation. But before we start with that, can you tell us a little bit about your journey to this position.

Tara: Yeah, I grew up in India, and I always wanted to be an aerospace engineer, and I came to the U.S. to do my Masters, and then my PhD. And it was just interesting to me that I did an internship in India at Hindustan Aeronautics Limited, and I was one of the only women on the research and development team, and then when I came to the US, and I was doing an internship at a drone company, I was also one of the only women on the research and development team. And it was just very surprising to me that the most powerful country in the world had a similar kind of talent makeup and distribution as a very low-income country. India was much lower income than it is now, and it made me kind of think about it, and it was just surprising to me to run into people in the U.S. where, especially women, thought it was okay to say, “I’m just so bad at math,” and that is almost like such a shameful thing to say in India, because it’s kind of saying like, “Oh, I’m not intelligent or I’m not a good person.” And it was just kind of interesting to me to see the culture of even people in a wealthy country just assuming that certain paths are not open to certain people based on their gender or socio-economic status, and in the richest, most innovative country in the world, that’s also the most entrepreneurial, that was just such a shock to me that that is okay for us to say. And so I stepped out of my PhD program and actually started a nonprofit 20 years ago, and the goal was to bring cutting-edge technologies and an innovator’s mindset to groups that are not typically thought of as innovators, and that was the sort of path. And we were funded by the U.S. Navy in the early years, and we were very, very data driven, trying to figure out what type of model makes the most impact on students and their mindset as a innovator. And one model really rose to the front that had a durable impact on students’ identity as a problem solver, and that was a three-month long experience where you work in a team, find a problem in your community, and actually build a technology solution and present it to your community. And that experience is unforgettable. And 10 years after that experience, students would choose STEM careers because of that very, very formative, collaborative learning experience. And when we saw that, it was like, “Okay, this is it, and we need to scale this up.” And we brought it to 120 countries, and it has run primarily as an after-school program, and now we have adapted a new version of the same curriculum, but making it easier for educators to use in the classroom. And that’s the AI in Action curriculum.

John: Can you tell us about this program?

Tara: Yeah, as I was sort of mapping out the journey, it is really our Technovation curriculum and our Technovation model of supporting students to go out into their community, talk to nonprofit organizations, actually any organization, find out what are some big problems that are impacting people, and then developing a technology solution and testing it with those users. So we started doing AI nine years ago, before anybody was talking about AI. Nvidia and Google and the Patrick J McGowan foundation were the three sort of leaders who recognized what was coming and wanted to make sure that there was a lot of education support built in when AI really became mainstream. And guess what? It took about nine years for it to become what it is today. And so we are entering the space with a lot of evidence-based curriculum, a lot of incredible stories of students actually using and building AI solutions for good. And so, as I had mentioned, this curriculum has run primarily in after school, and so we adapted it to be a shorter version that educators can adapt for in school, and you go through the same sequence of how do you find a problem that’s a meaningful problem, impacts a lot of people, and you need to have no prior technical background. How do you actually build an AI prototype? How do you build your own data set? How do you train a model? How do you make sure that it is not going to hurt anyone? And then actually test it with users, iterate, and then present it to the broader community on kind of like a demo day.

Rebecca: You mentioned that it was after school, now you’ve shortened it so it can be embedded into curriculum at schools. How do educators come about the program and actually bring it to their classrooms?

Tara: Yeah, so it’s completely free. Just go to technovation.org, and you can go onto the curriculum. And so there are two tracks. Basically, one is a competition track, and that is primarily for girls, and so that has typically been what we’ve been running after school. And then we have the AI in Action track, which is for educators, for use in the classroom, again, completely free, and you can access it from our website. You can also sign up for our educator guide that will provide you. Again, you know, you need no prior AI background, and the exciting thing is, we want you to become part of our global educator community and get connected to other communities of practice, learn from one another, and especially for students, its super inspiring to see what other students around the world are using and building to improve the world.

Rebecca: Can you talk a little bit about the competition track, why the focus has been historically on young women?

Tara: Yeah, we don’t have, like, I started like, the problem statement. I think every one of the 190 countries legally discriminates against women. They have a very big ladder to climb. And I just think that a country really loses out on innovation when you just have one type of person that is in the leadership role or in the sort of innovator’s seat. And, I think, if a country is really to make progress, you want everybody to have these kind of future-ready skills and these future-ready mindsets. And so I don’t see this as from an equity lens, I really see it as this is what a country needs to innovate and to stay thriving. So that, especially in the age of AI, where so many of your traditional white-collar prestigious jobs are being automated, there’s a real question of like, what are the big problems that are left for humans to solve? And these are these very complex human-related problems where so many people don’t have access to healthcare, education, there’s hunger across the US. And so who’s solving these problems? These are very, very difficult problems, and we are not empowering students to tackle these very complex problems. And so I think that this kind of future-ready skilling and mindset is really important to bring, especially to groups that have not been part of that.

John: Can you tell us a little bit more about the competition track? How is it structured?

Tara: Yeah, absolutely. So there’s just so much evidence to show that having a deadline motivates humans to try doing big things in a short amount of time. And so, yeah, it is a challenge and the challenge is find a problem in your community with a team. And so we invite girls to form teams sometime around now, anyone can go onto our website and register and start basically going through all the steps. As an educator, you can think about if you want to invite or do a club, or an after school club, or bring some students together, and then have the students self select into teams… anywhere between one to five… and then the most exciting part is for them to find a problem that they all care about, that they want to solve. And this is the highest learning opportunity, because they have to sort of think about their own value systems, who they are as people, what do they care about, and then they have to compromise as a team. And so the collaboration that happens there is very, very deep, and the conflicts that happen and the negotiation that happens. Once you register on the Technovation platform, we try to connect every team with mentors from industry, depending on your zip code or whether you’re virtual, and so educators are always connected to more help, so that the teams are not on their own. And then the team of girls and mentors walk through the 12-week curriculum. There are three subtracts divided by age. So beginners, eight to 12, junior is 13 to 15, senior is 16 to 18. And now we have an advanced accelerator track for 19 to 24. And so the curriculum is appropriately scaffolded for these age groups. And then, yeah, you spend 12 weeks actually prototyping, testing your product with users, and then doing that final demo day. And depending on how effective your solution is and how big your problem is, you may make it to the different rankings, so quarter final, final, and then we have the big world summit, where we apply the finalists from around the world. Last year we had it at Nvidia, and that experience is absolutely life changing for these girls, because they make friends with other innovators from around the world, and they stay in touch with them, and that’s their cohort of fellow inventors. So yeah, that’s the whole journey. This past year, we had about 33,000 girls compete.

Rebecca: That’s wonderful. I’m thinking, as an educator, it sounds really enticing and really exciting, but also maybe a little overwhelming to like, take a group of students on a journey like this. You mentioned that maybe the educators don’t necessarily have to have a background in AI, but what are some things that they would need to be prepared to do to lead a project? And is there training for the educators?

Tara: Absolutely, there’s a training for educators, and you can sign up, and now is the time, and you get connected with my incredible team of colleagues who absolutely love to support educators on this professional development journey and this journey of sort of bringing a sense of purpose to their students. It’s so, so fulfilling, because it’s not like your average class project, because you’re actually going out into the community. You have students listen to people and hear about their problems and then frame a solution. And students are very self motivated, because when you sort of promise somebody else that I’m going to make a solution for you, the students have an increased sense of accountability and responsibility, which grades can never do. And so you really unlock this deep sense of purpose and a problem solver’s identity in students, and that’s very fulfilling for educators. So Rebecca, you’re absolutely right. It feels overwhelming, but I think the thing to remember is like, there’s no right or wrong, because even a very lightweight experience for students is very powerful, and like with FIRST Robotics or any kind of competition framework that lasts for a long period of time, every year you do it, you get more students, you understand things a bit better, things become familiar, and you just get better at facilitating. But you’re not alone, and I think if you keep in mind that you don’t need to be the expert, but you can basically use AI to solve so many of these questions that you may have, it’s not going to be that overwhelming, because if there’s something that you don’t know, you just say, “I don’t know, let’s go find out.” And that part is just fun. I find that it’s so exhilarating to work with young people, because they’re so unafraid, like I’m afraid to try new things and break things, they just go bang at the keys, break things, and nothing happens. And it’s very inspiring to me, and that kind of reverse mentoring is something I’m like, “Yeah, I’m a facilitator, but I’m learning so much here too.”

John: You mentioned that the AI in Action curriculum was free. Who bears the costs of this? I know you’re a nonprofit organization.

Tara: Yeah, that’s a great question, John. So technology companies and our corporate partners who are really interested in building a talent pipeline, a talent network, because if we are to increase resilience in the United States, there is a real question of, where are the people coming from? And so the industry is very, very interested in addressing the skilling problem. AI skilling is a very important part of the full AI supply chain, and that’s what they have been supporting, the development the build out of the curriculum, as well as engaging their employees as mentors.

Rebecca: I want to pick up on something you were just talking about, which is there’s a lot of opportunity for educators to learn from students about lack of fear. A lot of college students, for example, are more comfortable with AI than maybe some of the faculty. Is this also true in elementary and secondary education, and if we’re going to venture into an AI space that we might be unfamiliar with, what are some basic things to get started?

Tara: I honestly think the Technovation curriculum and experience is a very powerful one, because it actually unlocks and opens up the black box of how a large language model works. I’ll give you one example of what a student builds, because it’ll help you immediately understand what it means to build a dedicated AI solution for a problem. So a girl was looking to clean a local lake. This was in India, and bringing volunteers together, and she wanted to sort of give some data back to the volunteers about how the lake was getting better, and she didn’t have access to a water sensor and her idea was she would train an AI model to detect the sounds of birds, and more birds would come as the water became cleaner. And so when she was looking online for an AI model that would automatically detect bird song, she only came across North American models. And so she built her own data set of Indian birds so that it could actually recognize the Indian bird songs. And these are examples of students around the world building new data sets that don’t exist currently for problems that are not being addressed by the AI models and solutions out there. And so educators can really be on the forefront of that if they were to partner with young people and students. And that’s where the innovation lies, and the only thing holding us back is fear. And I think we’re all super busy, and so the other thing holding us back is that we probably feel too overwhelmed and uncomfortable to make time and to prioritize this. But here’s the thing like, this is unlike previous technologies. This is unlike the internet. This is unlike mobile. This is unlike a lot of the other kind of technology waves, because it’s never threatened what humans did so closely. And the kinds of conversations we are having today are very different from what we used to have before, because sort of what it means to be human is being questioned when an AI, sort of chatbot responds sympathetically to you, and what if you didn’t know it was an AI chatbot, like, what if your friend was not a true friend and responded in a similar sort of robotic way? It still makes us feel the same way, right? And so that is a very scary piece, because that means that I don’t know what it means to be uniquely human. And so I think that over the next five years, people, especially our age, adults, if we are not fully grasping our hands around this crazy, scary thing, we’ll just get more and more buried by our own fears. And so the most we can do is to really embrace it and experiment and young people do it the best. So I think the Technovation model of adults co-learning with young people is such a powerful and a fun way. Otherwise, online learning is so boring. [LAUGHTER]

Rebecca: One thing that you’re highlighting is having a goal, [LAUGHTER] or like a problem that you’re interested in and connected to and that you want to solve. And I think that’s always a motivating way to pick up some new skills.

Tara: Absolutely, and I think grounding it in your community, whatever your community would be, it could be physical, geographical, online.

John: Now, I know at the college level, students are often racing a bit ahead of their professors in terms of adopting AI tools, and in terms of that willingness, as you said, students are much more willing to try things and not be too worried about the risks of things breaking, but I know a lot of college professors are a little reluctant to try it. Are you seeing the same sort of things in K-12 education? The response of many school districts initially was to try to ban the use of any AI, but that broke down pretty quickly, but are you still seeing that same sort of resistance to introducing AI tools in K-12 education?

Tara: Absolutely, you can see it everywhere. It’s the same kind of percentage of adoption, fear, hesitation, almost in every workplace, and especially so in the education space, because it takes so many years to build curriculum, build evaluation methodologies, set those systems in place, align teacher… like all the incentives and performance around that. And so I think we’re going to have this period of intense chaos where the evaluation methods have not caught up with the tools that students are using. And so I think the place is really to have open conversations with your students, because this is such an interesting time. You can bury your head in the sand and say, “This is above my pay grade,” or “This is not my job,” or you can really engage your students in some fascinating questions about how they are using AI tools to do their homework at home, and how do they see their brains changing as they are using AI for a lot of their assignments. And I think that will be very, very interesting. I don’t know if schools can quickly adapt and ask these questions, of like, what are the skills that are important to teach? Because I think we’re all grappling with that question. I think when people talk about empathy, being uniquely human, I was just kind of saying, I don’t know what that means when a chatbot can mimic empathy much better than even a human could. So I think that this is a time for parents and educators to really open their minds and actually ask young people and their children, no matter how young, because guess what? They’re all using AI. What do you think is happening in your brain? I have two daughters, and my older one was saying that she can feel her brain going soft in some places, but being exercised in different ways. And so that’s very interesting. Like, I totally feel my brain going soft in some ways and stretching in other ways. And so, yeah, what is happening here?

John: People who already have become experts or have developed some expertise in an area are able to critically evaluate the output of AI systems. But there’s a lot of concerns that beginners might do some cognitive offloading and not develop those critical thinking skills needed to critically evaluate the output. And we know those hallucinations do occur because of the nature of these models. What can teachers do, at all levels, to help develop the critical thinking skills needed to be able to critically evaluate the output?

Tara: I think that’s such a great question, John. And I’ll come back to really encouraging students to tackle the big problems, because complex systems thinking is not a linear problem, like where you follow a bunch of steps. I’ll give you an example. A very common problem that students try to tackle is hunger, especially in the United States. And the problem is so complex, because many people in the middle class families see the amount of food being wasted in restaurants, at your home, and so you immediately think, “Oh, what if it was a better food distribution method that would bring this food that’s being wasted to the people that are hungry?” And many people will just stop at that solution, and that is not a solution. The hard part is, really, this is perishable, these are perishable items. They’re dependent on preferences. It’s dependent on where you are, and the cost of logistics and transportations are enormous. And how would you do that at scale? Everything can be done for one little thing, but you’ll break down when you do it at scale. And so that’s a complex system, and that kind of mental mapping and problem solving is completely different from the little project that you would outline to a student. And so those are the kinds of skills that are really the ones that are what we need. And I think teachers have… there is time in the classroom. And if you align a lot of, I think, reading, literacy, math standards, into a rich project like this, I think you are preparing students very well for the future. And then they’re completely understanding they’re using AI at every step of the game, at every step of that way, and they’re seeing exactly what you were mentioning because they’re so invested in the end outcome that they’re checking and they’re like, “Wait, that doesn’t line up.” And so they’re motivated, they’re invested in their learning journey. I see students short cutting when they don’t care. We’ve traditionally sidestepped that question when students ask, “why should I care about this? Why does this matter to me?” And we typically say, “This is important for your brain, this is important for your future.” And I think we can’t hide behind that for much longer.

Rebecca: You’ve mentioned that the competition track at least is 12 weeks and I know the one for the AI in Action is a little bit shorter, but there’s a structure to it, because it goes week by week. Can you talk a little bit about what that structure looks like?

Tara: Yeah, absolutely. And so the very first part is problem identification, kind of like what we were talking about, and not very broadly, like, “Oh, I’m going to solve hunger,” but really narrow it down into what exactly is the problem, and then how can AI technologies actually address it? And so if you were to go back to our hunger example, it could literally be, how do we use AI to predict supply and demand? And how could we make the distribution of unused food as efficient as possible with minimal use of gas and energy and things like that? And so that problem identification, a meaningful problem, takes a few weeks, and we provide a lot of exercises, worksheets, for students to be able to do this, and for educators to facilitate that. The next phase would be prototyping, where you actually prototype on paper, and then you go out into the community and get feedback from actual people on would this actually help you? Where are you getting stuck? And then kind of building out the user interface for that. Third stage would be to actually start to code it, and this is where they’re learning whether they’re going to build it as a mobile app, or have it as a website or whatever. And then you start to think about, once it’s actually working, you test it again with people, and then you actually think about, how can you bring it to a lot of people? And so there’s a marketing element to it. There’s a business element to it. You look at competitors, you look at how much money is needed to actually scale it. And so that’s where that entrepreneurial learning happens a lot. And then the last phase is again, after iterating your solution a bunch of times, you actually develop a pitch video or a pitch where you tell your story and you practice explaining your solution to a larger audience so that more people adopt it and give you feedback. So that’s the full spectrum, and it’s a spectrum of starting a startup. I think for educators, and Rebecca, you were talking about like it’s a very real thing, like when you’re busy on your day to day, it feels overwhelming to try something new, to learn something new, and to restructure your life to make room for that. And I think that, as I was saying, that this is a very unusual time, and you may think that, “Oh, I’ll be fine,” but the option is, there’s a very real risk of jobs being completely eradicated. I think educators are not going anywhere, because we learn best from humans, but you can really be a superstar if you learn how to leverage these tools to support you, rather than have that be pushed down on you when you’re not ready. And so I think taking ownership of your own learning journey and your career and your professional development, I think it’s a very exciting sort of juncture for adults today where you can choose to really be a world class superstar because you have the world class superstar tools at your disposal, and the only thing holding you back is your mindset.

Rebecca: So we always wrap up by asking, what’s next?

Tara: Yeah, I think for us, it’s really about trying to support and get educators to help their students get ready to solve these big problems. Because I sort of see an extreme case where corporations are making so much money with AI automation, maybe, like, I don’t know whether universal basic income is a real thing, but what if it is? Then the big problems that corporations will not be solving are the ones that still cause human suffering, healthcare for all, high quality education for all, no hunger, and those are not incentivized by industry. And so whose job is it to solve those? It’s our job. And I think that really encouraging students to have that sense of empathy and empathize with another human’s suffering and to try to do something about it. I think that’s the core of what makes us human. And so that’s why we want to bring this and work with educators to bring this to many, many more students.

John: So if people would like to find out more about the program, we’ll include links to your website so that people can learn about the AI in Action Program and the other programs that you run. And thank you for all of your work on this.

Tara: Absolutely. Thank you for having me.

Rebecca: Yeah, thank you for sharing this exciting work. And it’s even better that it’s free.

Tara: Yes, absolutely!

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

422. Negative Examples

To understand what a concept is, we must also understand what it is not. In this episode, Bill Goffe joins us to discuss his study of the effect of the use of negative examples on student learning outcomes.

Bill is a Teaching Professor in the Economics Department at Penn State.  He had previously been one of our colleagues here at SUNY Oswego. Bill is very well known in the profession for his Resources for Economists on the Internet, which was one of the very first internet guides available for economists, and it’s now hosted and sponsored by the American Economic Association. He has served as a member of the American Economic Association’s Committee on Economic Education, the Secretary-Treasurer for the Society of Computational Economics, an Associate Editor for Computational Economics and the online section of the Journal of Economic Education. Bill is currently co-chair of the Liberal Arts Teaching Group, a faculty learning community at Penn State.

Show Notes

Transcript

John: To understand what a concept is, we must also understand what it is not. In this episode, we discuss a study of the effect of the use of negative examples on student learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Bill Goffe. Bill is a Teaching Professor in the Economics Department at Penn State. He had previously been one of our colleagues here at SUNY Oswego. Bill is very well known in the profession for his Resources for Economists on the Internet, which was one of the very first internet guides available for economists, and it’s now hosted and sponsored by the American Economic Association. He has served as a member of the American Economic Association’s Committee on Economic Education, the Secretary-Treasurer for the Society of Computational Economics, an Associate Editor for Computational Economics and the online section of the Journal of Economic Education. Bill is currently co-chair of the Liberal Arts Teaching Group, a faculty learning community at Penn State. You can also find Bill on many listservs and forums devoted to teaching and learning. Welcome back, Bill.

Bill: Thanks, John. Appreciate that.

Rebecca: Today’s teas are:… Bill, are you drinking any tea with us today?

Bill: I am. I’m drinking Twinings green tea, decaf.

Rebecca: Nice, good choice. How about you, John?

John: I am drinking a peppermint tea from the same vendor.

Rebecca: Nice. I have hot cinnamon spice today because my daughter requested a pot of it. So we’re sharing a pot of hot cinnamon spice this afternoon.

John: Very nice as we approach the holiday season here when we’re recording. So we’ve invited you here today to discuss your recent working paper with Austin Boyle entitled “Negative Examples in Lecture Improve Student Learning. First, can you tell us what you mean by negative examples?

Bill: Sure. So you’re describing a concept to someone, let’s say a young child, and describing what a dog is, and you talk about different types of dogs: Dalmatians, Pomeranians, whatever. And unless you describe how a cat is not a dog, they’re probably not going to have a great idea of what a dog actually is. And this might be the case, say, for a second language learner, where you might have different terms for different types of mammals, also related to something called contrasting cases, where experts are very good at delineating different types of phenomena or different types of facts, if you will, say a physician or variation theory used in math. So these are all ways of getting students a rich understanding of something.

Rebecca: Can you talk a little bit about what prompted your interest in negative examples?

Bill: Sure. So I’ve used clickers to teach for a long, long time. You currently use iClicker Cloud. Love it. And I tried to write a paper on just in general, using clickers to teach economics, and the paper didn’t really work very well. There’s reasons it’s never been written, and the reason was I use clickers in many different ways, so a little bit more focused approach to having a paper. And a few years ago, I came across the book How to Teach Economics by Yousuf Hamid. He was a high school teacher in Great Britain, and I had to read that. Well, to be honest, I read it during graduation. You have to go to graduation every year, but, you know, it always kind of work. And he talked a lot about negative examples. And I thought, “Oh, that’s pretty interesting.” And it also followed work by Kirschner, Sweller, and Clark on the idea of direct instruction lowercase “d” and “i”, and I thought, “Well, that’d be an interesting to try, and it’s fairly focused,” and that’s the genesis of the paper, and pulled Austin into it as well.

John: We’ll include citations to those resources in the show notes. So, many people, when they’re teaching, tend to use primarily positive examples. What’s the problem with just using positive examples of a concept? I think you’ve already addressed that a little bit.

Bill: One example, the original work on this, at least formal original work… the idea goes back a long, long time… it was by Engelmann and Carnine, a book on direct instruction. And they give an example of a glem, a G- L-E,-M. Could you define that just with one example? Well, you could not. And they give an example of a pencil. Well, is it wood, or is the pencil horizontal, whatever? But it helps you focus in better on an idea when you understand what it’s not. For example, in economics, we talk about capital quite a bit. Unless you tell students, it’s not stock, they probably won’t pick up on that. Or you give examples of money, how money is very specific to economics. To economists, they’re probably going to pick up on it’s not some general sense of wealth.

John: And I can affirm that the concept of capital is one tht students have always had a lot of trouble about because the way we use it in economics differs from how it’s used in everyday use. And just for our listeners, could you explain the difference between the everyday use of capital and the way we use it in economics?

Rebecca: I’m an everyday listener.

Bill: Great. We’ll see if this works. So, to an economist, capital is something owned by business, manufactured and used to make things, so, a commercial mowing machine, a commercial oven, a jet liner, construction equipment. In everyday usage, it might also include financial capital, stocks, for example, might be considered capital. I would add one thing economists really haven’t done, the Bureau of Economic Analysis actually redefined capital recently, includes intellectual capital, and I just don’t teach that. It would be confusing, but there’s actually about as much intellectual capital these days as physical capital in the United States. But maybe in an intermediate course. I would do that.

Rebecca: Seems like an easy thing to get confused. Can you talk a little bit about how you pick which concepts to use negative examples for? I can imagine that some concepts are more ripe for this strategy than others.

Bill: Yes, we wanted something fairly limited, not a long concept of how to manipulate a graph or something, that could be a further work, but just a definitional sort of thing, but the students have misconceptions about this, as we described there with capital. And so we picked three different ones, money, capital, and technology. Technology in everyday use, our phones, for economists, it’s basically efficiency in the production process. You know, it is rather unfortunate, there’s what, 50,000 words in the English language, and we use words that are used in other ways just to make it more complicated. And I joke to students, I wish I could use different words, but that would hinder their further education, I fear..

John: And this has been a problem for economists for well over a century in those very same issues, and it would be nice if we didn’t have to do that, because it introduces that confusion right when students are just starting to learn the vocabulary of economics, and the way I talk to them about I say, “Hey, part of this is you’re just learning the language of economics, which may look similar to how we use these words in everyday use, but it’s a bit different, and you need to focus on these things because of the ways in which we’re using the term.”

Bill: I would add that I’ve seen some STEM education research papers. They teach concepts first and jargon second, and they find that actually improves learning because you’re trying to remember the terms and they’re using the terms, it overloads your working memory. And there you teach jargon later, they find improvements.

Rebecca: We do that sometimes in design, where we separate the technical skill from the concept, because learning them all at the same time can be complicated. I was thinking of a term that I run into all the time where there’s confusion, where this strategy might work well, which is digital accessibility, where people confuse the idea of meeting these standards to help people with disabilities gain access to content and material with the idea of “just available,” like it’s on the internet.

Bill: I learned that distinction just recently myself. I would add on choosing these terms, we found even after instruction, students got it right between 60 and 71% of the time. So we seem to have picked a nice set of terms and definitions to use.

John: Which is actually much better than they would tend to if you had only used positive examples. And we’ll talk about those results a bit in just a minute.

Rebecca: Can you tell us a little bit about your study? How you had it set up? How many students were involved?

Bill: Yes, we had about 1200 students. I teach large sections, about three sections every semester, about 340 students that did this over two semesters, basically just repeated things, get more power for our tests, and did a so-called crossover study. Crossover study, have your participants that are both controls and the intervention side of things. We did a 3×3 one, which is a little bit hard to describe on a podcast, where you can’t visualize things, but there’s a fairly famous one for ultra processed diets. You might have heard of by Hall and company in 2019. They put people into hospitals, they control directly what they ate, and one group got ultra processed foods for two weeks, the other group got more wholesome foods, I guess you could say, and then they switched who got the wholesome foods, who got the ultra processed foods. So thus you could deal with the fact that people might have responded differently, that different foods have different effects on people. And we did the same thing here in our study, except we had three concepts and three sections and three teaching methods.

John: So what were the three teaching methods that you used in the study?

Bill: So we just use a positive example, where I would list a half dozen examples of capital or money, or not quite of money, because there are quite six examples of that. Then with lecture, we had both positive and negative examples, where these things are money, this is not. This is capital. This is not. And a third type of teaching method was with a cliicker question, where I gave a student’s list of things. Some are positive, some are negative examples, and they just choosed which was which.

Rebecca: So what did you find? [LAUGHTER]

Bill: We found that including negative examples improved learning in the following class by about 21 percentage points. Let’s say the class got 50% correct just with positive examples, the class that got positive and negative examples got a 71% correct. So it is very statistically significant, it was robust with lot different statistical methods we used. So we’re pretty pleased with that, it was a clean result. Sample size helps, and I think probably doing it just with precise definitions, not long drawn out concepts or something like that.

John: Based on these results, would you suggest that perhaps we should all focus a bit more on using negative examples as well as positive examples in our teaching?

Bill: Yes, most certainly it’s an easy intervention. For example, you might know Jim Lang’s book Small Teaching, where he talks about easy interventions you can do, and this one’s easy, as long as you remember to do it. But you’re making up your notes, you think of things that aren’t whatever you’re talking about. You’ve probably seen those on exams, overall. Interesting result we had too that the clicker questions, the active learning, was no different than the lecture. So we’d like to do further work on that at some point, most active learning studies have been semester long, active learning not very well defined, and we didn’t find any difference here, so presumably, at some point, active learning make a difference be interesting to find out what that difference might actually be.

John: That’s an interesting result, and that’s probably worth some follow up.

Bill: I think partly on if you do see similar results with the flipped class and active learning and such, people don’t really have a framework on what they actually ought to be doing before class and what active learning should look like. I’ve got another paper on this, I mentioned, that uses so-called deliberate practice, That’s the 10,000 hour rule. It’s really not 10,000 hours, but there’s some key parts of active learning people don’t do that much. One is to focus on sub skills, for us in economics, things like change in demand versus change in quantity demanded. We generally don’t teach that specifically, is my thinking. And then also, you want to make connections between things. How are supply curves and demand curves separate? How are they similar? Because we remember things in terms of frameworks or schemas. The negative examples help develop schemas. You understand what things are not, kind of triangulating, if you will. And good active learning does the same thing, but if they’re just filling in a worksheet with numbers, they plug into the formula, that’s active learning, I suppose, but not much learning there. So we ought to focus more on exactly what active learning is doing for people.

Rebecca: Following your study, what are some different ways that you’ve incorporated this practice into your own teaching practice?

Bill: So I definitely try to do the negative examples, capturing, oftentimes what I see on these exams and when I’m thinking of a question, I always try to have it connected to something else. Oftentimes one concept’s related to another one, and we make that connection explicitly in clicker questions so that, “Oh, if you remember this, this is a variation of that.” For example, how is consumer price index similar and different than the GDP deflator would be one thing we do, and why are they fairly similar? To understand better how those things work. Oftentimes, I explain things based upon what they’ve done before. Another thing I do is we have very nice technology at Penn State, I have a main screen in the classroom. I have a side screen. Side screens, I put up an outline includes both the outline for the day and our bigger topics. So I’m constantly referring to that outline saying “what we’re doing here is a variation of this, is different than that,” and so forth. And if I do surveys as students, they find that very useful. A better study would be to not have the outline there someday and see how learning goes. I just now thought of that as an idea to do a study on.

John: We know that students come in with a weaker set of connections than faculty, and sometimes we assume that they’re able to make these connections and see the relationships, but they haven’t yet developed that skill, and sometimes it’s generally believed that that type of scaffolding, by putting up an outline of the main concepts you’re trying to get across, can help. But I don’t know if I’ve ever seen any tests of that, certainly not in economics, but I’m not sure if I’ve seen it in other areas either.

Bill: I have not. I can’t say I’ve looked very hard. I would add, to do my literature searches. Now I oftentimes start with ChatGPT, but that’s another whole podcast, [LAUGHTER] I suspect.

John: Yes, and there’s some other good tools out there, too. There’s research rabbit and other things that will actually, even in some cases, will provide more accurate results. But ChatGPT is getting a lot better all the time for these things.

Rebecca: Other than in the test results, have students responded? Or have you asked students about using the negative questions and how that impacts their learning?

Bill: I have not actually, that’s a good point. That’d be another thing to add. It’d be interesting see if they actually can pick up on that. Oftentimes, they’re not that great on understanding their own learning. Something we’ve all seen as instructors, someone thought they did much better on the exam than they actually did, which is something you want to avoid. That’s a tough place to be in and you have to be sympathetic as instructor when that occurs.

John: Now, one thing to keep Rebecca happy, we should also ask a little bit about the methodology that you used. In the study, you used both pooled data as well as panel data, and you also used ordinary least squares and binary dependent variable models. So could you just talk a little bit about why you used multiple approaches and how consistent were the results across those.

Bill: Yeah, the results are very consistent. So sometimes you don’t quite know what econometric approach you take, you can argue this is better, that’s better. And another approach, and it’s pretty common these days, I think it’s by far the most common, is you take multiple approaches and see if the results are robust, to see if you get similar results. Indeed, we did. So it’s a very happy occurrence when that occurs.

John: So were you surprised at the magnitude and strength of these results? I know when I read the paper, I was surprised at just how large that effect was, because it’s very rare to see effect sizes that large in a study of alternative teaching approaches.

Bill: I don’t think I had many preconceptions here. That’s odd. I certainly was pleased, but I can’t say I was quite surprised. It certainly makes me want to take this approach again with where we do the crossover studies, you can do many things with that. For example, we’re told we should teach about things students find interesting. Well, is that really true? I did an example of that this semester where Penn State students, they do a large fundraising project called Dance Marathon, or THON, and last year it raised $17 million, about half the student body is involved. And so I do a question about real prices, and the question was, did THON actually raise more money when you correct for inflation, you know, something they’re invested in a lot, and informally looking at the data, that didn’t seem to have any impact on that segment, on calculating real prices, I need to do more work on that, but I hope more people take this methodological approach, the crossover studies, where you could study a lot of things, the outlines, is it interesting or not, and so forth?

John: Because with the crossover study, you’re ruling out the possibility that there was just some systematic difference between the sections or between the other conditions, right?

Bill: Yes, exactly. We actually found there were differences between the sections. Some did much better on the questions than others, but by effectively pooling these different interventions and controls together, we can adjust for that. We can correct for that. I don’t see it that much in economics. John, have you seen it very much, the crossover studies?

John: I haven’t seen very many, but one that stands out, and I think it was done by economists, was back when the initial studies came out that suggested that when people took notes by hand, they remembered more than when they used a computer. I do remember a study, and I’ll include a citation to this in the show notes. What they did is they had all the students use mobile devices for note taking for a portion of the semester, they broke up the group into two halves, and half of them, for the first half of the semester, took all the notes using mobile devices, and the other half took notes by hand. And in the second half of the semester, they switched, and what they found was there was no difference between how students did when they took notes by hand and how those same students did when they took notes on the computer. However, those students who regularly chose to take notes on a computer, on average, did less well regardless of whether they were taking notes by hand or on a computer, which suggests that the students who choose to take notes by hand may be perhaps a little bit more focused on the material no matter how they’re taking the notes, and those students who may bring in mobile devices to use, perhaps may do it because they prefer having the availability of distractions. So it’s not quite the same thing, but it’s similar in that you’re controlling for the individual effects and separating it out so that you are getting a better comparison, correcting for any self selection in the people who are involved, which is really, I think, what crossover studies are trying to remove, the effect of any systematic selection between the groups.

Bill: I would add, I’ve surveyed my students, and generally my students aren’t performing quite as wel do choose to use laptops, and higher performing students do not, so consistent with your more serious study, for sure.

John: And another thing I’ve been noticing a lot is that for quite a while, I’ve been walking around quite a bit, but I’ve also noted that the students in the front third or so of the class tend to be very actively engaged in answering questions, and they tend to do really well, and they tend to ask more questions. But when I’m in the back of the room, I see a lot of laptops where there perhaps may be a video playing, there may be some online shopping going on, or there may be some sort of video game that they’re working on on the mobile device and their laptop or one or the other. And I think we often do see that sort of sorting in terms of the way in which students choose to work. I’m not sure how I could get rid of that effect, other than moving them all up to the front.

Rebecca: it does seem like at least the multi taskers, in quotes, are at least removing themselves, and because they’re probably distracting to other people, they’re just moving themselves to the back so they’re not a distraction to the people who are trying to focus.

John: Except they’re surrounded by other people, and there may be a bit of a contagion effect going on back there.

Bill: I’ve seen papers on the contagion effect, yes. There was a Carl Wieman paper. They did look at seating in large classrooms and found a very definite effect. They even switched seats halfway through the semester to bring the people in the back up to the front, and they still did worse.

John: Some of our students are just much more interested in the material and much more engaged than others, and there’s just so much we can do with that.

Bill: It’s tough, but I would be a little bit sympathetic to students. It’s been a long time since I’ve had to sit through a typical undergraduate today, and you know, you’re three, maybe four courses a day, a lot of sitting. That would be tough, a challenge. There’s a book called My Freshman Year, and there’s a field anthropologist who went to college for a year to understand her students better, and she found it was harder than she thought.

John: And she found some dramatic differences, too, in the way in which students connected with each other from when she had been a student, that they had much tighter networks with the people they knew before they came to college, because it’s less costly to connect with them over social media and so forth.

Bill: Yeah, friendships have changed with the rise of social media.

John: It used to be costly. I remember having to bring all these quarters to put in pay phones back when I was in school.

Rebecca: You mean, in those 1900s phones that my daughter refers to, John?

John: Yes, I think that would be the case.

Bill: What’s a quarter?

Rebecca: Yeah. [LAUGHTER] Well, I think we still know what quarters are. We just don’t know what pennies are anymore. [LAUGHTER]

Bill: Well, that could be, yes. [LAUGHTER]

John: Are there any other things you’d like to share about this study?

Bill: Well, I think that’s pretty much it. I think I would emphasize this little bit more about, again, developing student schemas. But we talked about that earlier today, this broad network, and it’s through connections, and this helps them make connections more focused on whatever the topic might be at hand. So I think negative examples could be used for teaching most anything.

Rebecca: Including our colleagues when we’re doing professional development and all kinds of other contexts.

Bill: Yes. [LAUGHTER]

John: And it’s easy for us to lose focus, because we already can distinguish these things more easily, and students have not yet developed that capability.

Bill: The curse of knowledge really reigns here, where you know it well. It’s hard to understand how other people don’t. I was doing a think aloud exercise the end of last semester where I interviewed students half the semester in how they’d interpret different things. And I spent a lot of time on aggregate supply, aggregate demand model in my course, standard staple things course, asked about certain phenomena, no one actually used the model. I was only interviewing four students, so, I mean, it’s not much power there, but it was kind of crushing, but it gave me insights on how they think about things.

John: In an earlier podcast with Doug McKee, he talked about having students come in and just work through the problems verbally, so he could see what sort of process they were using, and by seeing that, it was easier to help guide them in a more productive direction. Because if we don’t really understand how students are processing things, it’s hard to help them understand things more deeply.

Rebecca: Yeah, we use that talk-aloud method and design a lot too around troubleshooting, because troubleshooting strategies require, especially around technical skills, you need to think about problems in particular kinds of ways to be able to solve them correctly, or to be able to solve them at all, [LAUGHTER] and doing that kind of talk aloud strategy works really well for figuring out where something went awry in the troubleshooting.

Bill: And experts think a lot differently than students, I would suspect. You would never think someone thought that way.

John: And fMRI studies confirm that different areas of the brain are activated for experts and for novices when trying to solve a problem. So it’s easier when you become an expert, but we have to help students develop the skills to get there. While we’ve got you here, could you also tell us a little bit about how you teach classes so large with so many students, every semester.

Bill: I use Perusall as a way to have them read ahead. They come into class, they have some sense of things and then a lot of clicker questions, typically, as long as it’s not something new. For example, I was doing long-run growth. I do that a fair amount, and the book doesn’t really talk much about the institutions and the role of markets in that. So that was a day or two of me talking. I’m going to be doing the business cycle starting tomorrow, and that’s a fair amount of me talking. But other than that, it’s them pretty much asking questions, or me asking questions. Probably on Thursday, we’ll get to calculate employment rates like read about that in Perusall. I’ll ask them, “Who in this group of people is unemployed?” Then I’ll show them some data, what we compute from that data. So it’s a lot of using ideas. I do a lot of peer instruction where they first do votes on their own, and then I say, “Okay, there’s a lot of uncertainty here. Please convince your neighbors.” And I use an app by iClicker, so I can see their votes on my phone, and then they do a revote, and they usually improve 20, maybe 30 points, then I provide further instruction on the topic at hand. And there’s a couple papers I’ve seen. Let’s see Michelle Smith at Cornell, a computer science paper. They found that if you have students revote, that helps them; if you have a further explanation from the instructor that helps them; if you both, it helps them even more, and it’s probably enriching their schemas, they’re seeing several explanations that reinforce the connections between things. Also I do a series of quizzes, rather than midterms. I do seven quizzes. We’re very fortunate at Penn State, we have a testing center, so I send all my students there for testing. So I run the quizzes for two days, after the quiz they do a second stage of the quiz. In class they start, then they answer at home using any resource, and finally I release the answers, and then they reflect on and describe what they found most difficult on that quiz. So we do a two-week cycle. There’s a lot of feedback, even though the course is quite large, with about 1000 students a semester.

John: So having that testing center gives you a bit more security in the assessments than many of us have when we give quizzes online. Many of the assessments I use are done online, and I used to feel a whole lot better about them before we had ChatGPT and other AI tools appear, especially with some of the browser extensions, which will directly answer things, so you’re able to avoid some of that, which is really nice.

Bill: I really feel very fortunate, yes. I would add, it’s a good thing that the AI tools came after Covid, rather than before.

Rebecca: So one of the things that you mentioned when you’re describing your class was Perusall. Can you describe what that is for people that don’t know?

Bill: Oh, yes. An e-text goes inside perusal, and then students mark it up with comments, questions, and responses to other people’s questions. So it has them thinking bit about the text. It grades them automatically. They need to start early so there’s a little bit of spacing there. They need to make sure they document across the assignment, and they’re also rewarded for opening up multiple times, not just the evening before the exam. And it’s fairly straightforward. Most students get full credit on each assignment. So it’s a bit of encouragement there to read ahead. I used to use so called JITs for just-in-time teaching assignments where I’d ask some questions, including what they found confusing, but I like Perusall better. It’s less grading work, and also, I think it covers a little bit more topics or not just looking at the answers to my JIT questions.

John: Now, do you break them up into groups when they’re using Perusall?

Bill: Yes, Perusall actually does that. When I look at the marked up text from 1000 students, the page does not load basically, or has difficulty loading. And you can also download, I’ve not done this, you could download all their comments, and you can run through a chatbot or whatever to get a sense of what they have trouble with, or what they find confusing. I think textbook authors would find that really, really useful. You could say your textbook was designed with the input from 1000s of students, and so your students can read this more carefully. I’ve not found an author willing to do that, but I think that would be a big win for someone.

John: It could be, and from a marketing standpoint, that could be really helpful. When iClicker announced that they had AI tools being built in, I was really hoping that they would integrate that into the text responses, because right now, it’s just generating questions, which is not bad, and it doesn’t do a really bad job of that, but it would be so nice when you got maybe 1000 or more responses coming in, or a few 1000 responses, if it could very quickly generate a summary of the responses to free-response questions. That seems like something that wouldn’t be all that hard to do, and it would be a nice extension.

Bill: I agree. I think there’s some polling technologies that do that. For some reason iClicker doesn’t. I wonder if there’s a legal issue or some other constraint. I would love to see that as well so I’m not just asking five different possibilities.

Rebecca: So we always wrap up by asking: what’s next?

Bill: So I want to do more work on active learning, the ideas from deliberate practice. Does it make a difference if students have good sub skills. And does it make a difference if you teach by making connections between different topics? There’s many ways active learning are implemented. And it would be cool to do a study where you look at active learning in many different implementations, but some get those interventions, some do not, in a crossover study, of course. So that’s what I would like to work on next a bit.

John: That sounds like a very productive area of research that could really be helpful for all disciplines.

Bill: Yes, well thank you. That’s the idea, I think, a bit about marketing of papers, you know, who would find it interesting, and so forth.

Rebecca: Well, thank you so much for joining us, and we look forward to your future research.

Bill: Thank you so much. It was great fun.

John: It’s great talking to you, Bill, always. I’ve learned a lot from you over the years, and it’s great to still be able to see your work out there in various places.

Bill: Great. Okay. Thanks so much, John. It’s great to connect up again.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Ryan Metzler.

[MUSIC]

421. Why the Magic Matters

Disney is a common shared cultural experience. In this episode, Jill Peterfeso joins us to discuss how Disney’s pixie dust can hook students and provide opportunities for critical examination in a variety of disciplines. Jill is the Eli Franklin Craven and Minnie Phipps Craven Professor of Religious Studies at Guilford College. She is the author of Womanpriest: Tradition and Transgression in the Contemporary Roman Catholic Church and a co-editor of Why the Magic Matters: Discovering Disney as a Laboratory for Learning.

Show Notes

  • Peterfeso, J. (2020). Womanpriest: Tradition and Transgression in the Contemporary Roman Catholic Church (p. 285). Fordham University Press.
  • Franzese, A., & Peterfeso, J. (Eds.). (2025). Why the Magic Matters: Discovering Disney as a Laboratory for Learning. Bloomsbury Publishing USA.
  • Twinings Tea Caddy at EPCOT.
  • Jill Peterfeso (2021). Leveraging Disney Magic. Tea for Teaching podcast. Episode 207. September 29.
  • The Disney Dish podcast
  • Steele, Bruce (2019). One Day at Disney: Meet the People Who Make the Magic Across the Globe. Disney.
  • Giroux, H. A., & Pollock, G. (2010). The mouse that roared: Disney and the end of innocence. Bloomsbury Publishing PLC.
  • C, Eric Lincoln and (0awrence H. Mamiya (1990). The Black Church in the African American Experience. Duke University Press.
  • Snyder, T. (2021). On tyranny graphic edition: Twenty lessons from the twentieth century. Ten Speed Graphic.

Transcript

John: Disney is a common shared cultural experience. In this episode we discuss how Disney’s pixie dust can hook students and provide opportunities for critical examination in a variety of disciplines.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today Jill Peterfeso. Jill is the Eli Franklin Craven and Minnie Phipps Craven Professor of Religious Studies at Guilford College. She is the author of Womanpriest: Tradition and Transgression in the Contemporary Roman Catholic Church and a co-editor of Why the Magic Matters: Discovering Disney as a Laboratory for Learning. Welcome back, Jill.

Jill: Thank you. It’s so good to be back. I’m really happy to see you guys again.

John: It’s really nice to talk to you. It’s been too long

Jill: It has. How are you, Rebecca?

Rebecca: I’m doing well. It’s been far too long for sure. Today’s teas are:… Jill, are you drinking some tea with us today?

Jill: I am. I just got it ready. This is an apricot tea. It is a light black tea by Harney and Sons. It’s really delicious. What about you guys?

John: I’m drinking a black raspberry green tea today.

Rebecca: And I have English tea time today. Very good… from England again?

Rebecca: No, it’s not my super fancy stuff, it’s grocery store, too. Sorry.

John: Recently she’s been bringing you these tea she’s picked up at Scotland or in England and other places during her travels. My travels are usually to Wegmans or to Amazon.

Rebecca: …those far, far away places.

John: …although very soon I will be stopping at the Twinings Tea Caddy in Epcot.

Jill: I’m so excited.

John: …which brings us to the theme of today’s podcast, which is your work on Why the Magic Matters. In an earlier podcast, you had described how you had incorporated Disney into several of your classes, including a travel component, but this new book extends this work in some new directions. First, though, before we talk about the book, could you tell us a little bit about how you developed an interest in Disney as an area of academic interest.

Jill: I came to Disney as an area of academic interest through teaching. It was 2014-2015 and I was asked to develop a first-year seminar, which is this class that all first- year students take, introduces them to college, introduces them to the liberal arts. But sort of the catch is, we don’t want it to be too clearly in your area of expertise. So it couldn’t be about American religious history, which is my area of expertise. It couldn’t be about Catholicism and women, which I’ve written a book on. It needed to be something that was going to be an area of discovery for me, as well as my students. And as a lifelong Disney fan, I thought Disney would be a really fun class to teach to first years, and I think it would be a good hook. I think they’d want to take that class. I developed that class. I taught it for the first time in fall 2015, and it really was an amazing opportunity to discover something in a back- door way. It wasn’t like, “Okay, I have this whole huge foundation that I developed through grad school and years of research and training from the experts. This is really like me doing a deep dive into something that I’ve always been curious about, and inviting students in on the journey.” That’s how I came about it. I taught that first class, and I taught that FYS, gosh, I think four or five times since then, I’ve also gone on to teach a couple other classes. I taught a class on Disney narrative magic that was a cross-listed English religion class. And then I’ve taught two study away classes on Disney that involved a trip component. I’ve gotten to do a lot of things with Disney, which is really exciting. What I didn’t expect when I first started teaching Disney was that it was going to turn into an area of academic interest and research. And I really happened, probably, I think about my second time teaching that first-year seminar, I found that we were having conversations in the classroom about these issues that I talk about in my American religious history classes and other things that I research and study and share with students. And so it was becoming clear to me that there was an alignment of Disney with my other research on Americanness and what people believe and why and how American history gives rise to culture. And so really, since about 2019 as I was finishing up Womanpriest, my first book, I’ve been thinking about ways to turn Disney into something academic. Teaching blended in with a lot of my own personal biography, like I was one of those kids who had The VHS tapes as a kid. We had the records, the little mini records in the books, and they were Disney stories. And I got to go to Disney World when I was eight, and it changed my life, like that that place existed was really mind blowing to me. So it’s been just like a really happy accident of all these things coming together, and it really makes sense as an area of academic focus. I don’t think many of us who do Disney studies do that in grad school. I think a lot of our grad advisors would be like, “Really?” but it’s something that we find once we’re a little more established and trying to find cool things that grab student attention. Disney works for that really well.

Rebecca: Can you talk a little bit about the origins of this particular book project, Why the Magic Matters?

Jill: Credit goes to a podcast, [LAUGHTER] actually. I was listening to a podcast I listen to religiously. It’s called the Disney Dish with Jim Hill and Len Testa. And I was driving home, I have quite a commute to and from school, and I was driving home from school and listening to the weekly podcast. And they were talking about this class at Elon University on Te aching Disney, and it involved a trip to Walt Disney World. The class was called, like, The Science of Happiness. And I’m like, That’s Elon University. I’m literally driving by Elon University on my way back home. So Elon is very close to Guilford College, where I teach. I did some research. I’m like, I got to figure out who teaches this class. I did some diving into their website, and found that the teacher of this class is a sociologist at Elon named Alexis Franzese. I reached out to her over email. We met up, and I was thinking at the time, this was early 2023 and I was prepping to teach a Disney away class later that spring. And I was like, let’s team up. We could write an article about teaching Disney, especially in difficult political times, because this was when Disney was in a very public fight with Florida Governor Ron DeSantis about the don’t say gay bill, etc. And Alexis said, well, let’s do an edited volume. And so that was sort of the birth of the edited volume piece of this.

John: So how did you select the contributors for this book?

Jill: Once we had agreed on a premise, which is like educators of Disney courses and content sharing their research with a wider audience, we wanted to be sure that we had a range of academic disciplines represented. We wanted to surprise and delight the reader, the reader of Disney fans and also students, to find how many ways that Disney speaks to academic disciplines, and how many academic disciplines can speak to Disney. So early on, I will admit to some fan girling. I’m like, okay, who’s doing the scholarship on Disney that I really love? And I reached out to them. Thankfully, they were in agreement and they wanted to participate. Alexis did some of the same. We talked to people who we’d gotten in and said, “Who else can you recommend?” So we really reached out to people, and then got people sort of through that snowball approach. We didn’t do a call for papers. We didn’t need to. We were thinking we might need to, but it was exciting how quickly professors, faculty, researchers doing this work, wanted to be involved. And I think that speaks to the energy and enthusiasm around Disney and teaching and sharing Disney. I think for almost everyone… I can’t speak for everybody… but I think there’s, like a passion piece to scholarship on Disney and sharing that with students and wider audiences. We really wanted it to be widely represented. I wish we had the hard sciences in the book. It’s one of the only areas that I feel is lacking. But, I have a list, so if you’ll allow me, this is our very impressive array of disciplines: we have curriculum studies, art history, business and marketing, data science, film history, indigenous studies, Asian studies, literature, education studies, leadership studies, history, media studies, religious studies, psychology, sociology and disability studies. We’re really happy and really proud of that array of voices.

Rebecca: So one of the things that you mentioned earlier was that Disney is a good hook to get students involved. And in your first chapter, you talk about how Disney can relate to any academic subject, [LAUGHTER] and you’ve just outlined the wide array of those that are in your book. Can you talk about some examples of how Disney can be relevant in these multiple disciplines?

Jill: There is this Disney+ show, and it is also a book that is produced by like Disney Publishing. It’s called One Day at Disney, and it looks at a huge range of careers that are available through the Disney company, like what is happening 24/7 around the globe with Disney, and I talk about that a little bit in the introduction. With that as an example, it starts to open your eyes to all the ways Disney demands investment and involvement from all these different… we could call them academic disciplines… but areas of expertise for their cast members and their employees. In the book, some examples like we have two people in our book talking about data science. So we have Len Testa, who wrote our forward, he of The Disney Dish, and Lucy D’Agostino McGowan. They talk about data science at Disney. A day in a Disney park is full of data, and what Lucy does in her chapter is to look at all this data for one ride at Epcot, Spaceship Earth, and looks at what Disney can do with that data, and also what she does in the classroom with that data and teaching her students how data analysis works, what Disney can do, this is how Disney decides where to place cast members, how Disney obtains and posts these up to minute wait times for rides and attractions, and then what guests can do with that data if they use an app that sort of is tracking wait times and busyness et cetera. Looking at data helps to understand how one plans out a day at a Disney park. That’s one chapter, another chapter that’s rooted in American history and film studies is from historian Sarah Nilsen, and she looks at Walt Disney’s biography and focuses on his love for animals, and how his love for animals got baked into what he was creating in his films, and the characters and the lives that film goers were invited to see, and she makes an argument that Walt became one of America’s first 20th century environmentalists because his passion for animals was then put into films, and later into parks, and that inspired Americans to think differently about the environment. Then one other example is my co-editor, Alexis, teamed up with one of her former students who had taken one of her Disney travel courses, which I think is awesome that that partnership even exists in the book, and they use sociology and psychology tools and frameworks to analyze what Disney expects and demands from parkgoers to enjoy their time at Disney. You don’t just show up at Disney… anybody who’s gone to a Disney park knows you don’t just show up and everything happens seamlessly. You have to do a lot of labor. And in this chapter, where they’re using psychology and sociology, they look at concepts like emotional labor, emotion work, impression management to look at how Disney recruits people, especially parents, to help create happiness, to make magic happen on Disney trips. As I think of it, all of our authors are peeling back the curtain of how Disney works and inviting them to see what Disney is doing that is not just surface fun entertainment, but there’s a lot to it, and how that works, and how we can think about it as critical educated thinkers.

John: One of the things that we’ve talked about quite a bit in the past year or so on the podcast is that there’s a lot of evidence that students feel less connected than in past decades to what they’re learning in college, that they feel that it doesn’t really connect to their own lives or their own experiences. Given Disney’s large role and very long-term role in our culture, does that help provide that sort of shared connection to students, getting them more engaged? You’ve already alluded to this a little bit, but how has this worked in terms of students making connections and becoming engaged with the course.

Jill: Definitely Disney is a common shared cultural experience. And even international students know of Disney, and that is a great foundation for students in a first-year seminar, for instance, because they’re already going to be connected in some personal way to the material. We have a shared recognition, if not a shared love of Disney, certainly not every student who takes the Disney class loves Disney. Honestly, a lot of them think this is going to be really easy, [LAUGHTER] and that’s why they take it. I’ve had them admit that to me, they’re often surprised by how deeply you can go into Disney, but they are hooked by the familiarity. I’m a big believer in Disney content, especially for early students, or discovering students, because Disney is something that they know well enough that they can feel smart about it, like a lot of my students are first generation, and Disney is not intimidating subject matter. So I like to think that students comfort with knowing some parts of Disney means that I can really push them on their critical thinking because they’re grounded, or they’re rooted in something familiar: magic, happy endings, characters, positive stories about the world, histories of America, like they’re grounded in something. Now, all of this said, I have had a couple students who take my Disney classes who complain like, this doesn’t mean anything to my life, and that’s where I really try to drive home the liberal arts-ness, like, “Okay, but this is like a training ground for your academic and intellectual muscles.” Like, when you read the introduction of Why the Magic Matters, you see a lot of my views on this creeping into that introduction where I’m like, “Okay, it’s not that you need to know how Disney works. Like, I think you should, because it’s a huge cultural artifact, but you need to know how to think deeply about these cultural artifacts.” The only other thing I’ll say as I think about the shared cultural understanding, I am going to be curious to see what happens with Disney and the shared culture in the next 5 to 10 to 20 years, because Disney is becoming prohibitively expensive… at least visits to the park are. This has been well tracked. Disney seems to aim for an income bracket of like in the top 10, top 5% and most of us are not in that. So I increasingly now see students who have never been to Disney parks, like that middle class pilgrimage idea that I know I grew up with is increasingly unreachable. So I’m going to be curious to see what that does to those of us who try to teach this content in a range of different academic settings. Some people teach Disney classes in very affluent schools with very affluent students, and then some of us don’t. I wonder what Disney’s shifting economics are going to mean for lower SES students. Yes, shared culture now, I don’t know if that’s always going to be the case, so that’s something for us to watch.

Rebecca: You mentioned earlier that Disney’s biography kind of finds its way in your book. Can you talk a little bit about how Disney argued that animation and film would be useful for educational tools, and there’s certainly evidence of this in the theme parks and other things. However, it’s a very curated approach. How does that impact our understanding of history, or the way that we might continue to perpetuate a version of history in our culture?

Jill: This is where I get really excited as a cultural historian. This is where I really like to hang out with my students, because culture, as we know ,is not just a mirror that we hold up to ourselves, and it reflects our values and our interests and our history. Culture is an actor. Culture creates and upholds and strengthens those values and histories and interests. A challenge I really have in teaching Disney classes… it’s a challenge I embrace… is to help students see that Disney is not just something they consume. It’s not just fun, simple entertainment for kids. Rather, Disney is doing consumptive work on us. Disney is commanding our attention. It is directing our attention towards certain narratives and stories while obscuring others. Disney’s upholding certain norms about gender and sexuality and body size and attractiveness and ethnohistories and race-based stories, et cetera. And of course, Disney loves to gobble up all our money while doing that. [LAUGHTER] Any college-level exploration of Disney must look critically at all of that. And this is not to say that Disney is bad. I don’t subscribe to any simple villainization of Disney, any more than I subscribe to a simple villainization of religion and religious systems as a religious study scholar. Nothing is all bad and just brainwashing. When we learn about humans and human culture and relationships, what can we learn about the appeal of Disney or the appeal of religion? There’s something there that’s worth looking at. We shouldn’t just dismiss it. I was teaching this FYS first-year seminar on Disney in 2016 when Trump’s Make America Great again slogan was becoming ubiquitous, and I remember talking to students at that time like, what does MAGA mean? What time and place does it refer to, do you think? And wasn’t hard for them to connect what they thought MAGA meant to some of the simple and sanitized and incomplete histories that Disney puts forward, that you might find walking down Disney’s Main Street USA, or in Frontierland, or in some of Disney films. Walt Disney was a patriot, no question. He loved America. He was desperate to serve in World War One, and he lied about his age in order to do so. He was fixated on Abraham Lincoln. The studio made anti-Nazi, pro-allies propaganda during World War Two. So he was really very much a patriot, and he was really concerned about telling an American story, but as scholars and as students, we get to question those stories as perhaps being more complicated than what’s been presented to us. And so the book, Why the Magic Matters, looks at this very question. So we have a couple of chapters that explore this. We have a historian named Alex Hoffman in the book. He’s out of U of Chicago. He teaches a course and his chapter, they’re both titled “Theme Park America,” and in Why the Magic Matters, he sort of takes readers on a walking tour of 1955 Disneyland, when Disneyland first opened. And he looks at some of these historical sections of the park, Main Street, Adventure Land, Frontier land, and he explains the choices Disney made in designing those lands, as Disney did. But he also looks at the American history beneath Disney’s depiction, and in so doing, he unearths racism, the Cold War, and like anti communist anxieties in the country in the 1950s the impact of settler-colonialism, and all of this as he helps us to read Disneyland in a historically authentic way, as opposed to the Disney sanitized way. A couple other chapters deal with this, but another one that springs to mind is one by Lucy Buck, who is an indigenous man. And in his chapter, which is called “Urban Indians in the House of Mouse,” he walks readers through his emotional development and his intellectual awakening as an indigenous person who grew up in Southern California, really close to Disneyland. And he talks about how his time in college, studying ethnomusicology and indigenous studies led him to see Disney’s films really differently, and more optimistically, it allowed him to try to envision a world where native creativity and culture could be represented and celebrated. iI my classes, there’s always a point where students are like, “You are ruining our childhood.” I have heard that so many times in so many forms, but thankfully, it’s a whole semester, and not just like a lecture, which would leave them kind of defeated, but they have a whole semester to gnaw on this material. And my goal is really to help them feel empowered in developing richer analyses of the world around them, because I really believe that once you can critically examine something, we can enjoy it without guilt or without fear of manipulation or without threat of loss of critical thought. I really hope that the class is an invitation to see Disney differently, but in so doing, these things in the world that we take for granted as being one way but can actually be something else, and the book tries to model that in a number of its chapters.

John: So the version of the world that Disney was creating was one that was very similar to the type of world that was presented in other forms of media in the 1950s and 60s, but it’s been moving towards a much more inclusive depiction of the world in recent years, but that’s led to quite a bit of controversy, as you mentioned, with the governor and also with our President. Was this type of culture war, perhaps inevitable for Disney to go through as we move towards a more diverse population and moving towards a white minority population?

Jill: This is such an important question. We talk about this in my classes. Here’s a mess of thoughts. I don’t think that conflict was inevitable because of changing demographics. I think Disney was always going to, of course, try to include more types of people as the American demographic shifted because, well, the money motive is obvious. It’s somewhat cynical, but it’s obvious. If Disney can create more fans and more people invested in its stories, that’s great for Disney as a company. But another thing that I like to think about is who creates Disney films and characters and rides and attractions and merchandise, like complex, interesting, rich identity, human people. Many of these creative types have a really wide range of identities. So it goes beyond this Disney’s bottom line to include consideration of the artists, the animators, the Imagineers who want to see themselves and their loved ones in Disney stories. So I think that was going to happen. Was the conflict inevitable? And perhaps I’m too naive, but I don’t think including more people in the story is going to lead to automatic resentment without certain cultural forces and political forces stoking those fires. The anger directed at Disney has to be put into a context of social media algorithms [LAUGHTER] and exploiting politicians. As soon as anyone with any sort of power and authority start to draw lines in the sand, companies like Disney are going to fall on one part of that line or the other, and whoever’s not on the side with Disney is going to be angry at Disney. They’re going to take it really personally. An obvious recent example is ABC Disney’s indefinite cancelation of Jimmy Kimmel Live, which at the time of taping, because we’re taping this, it is back on, It has been on back on for a few weeks, but Disney was in a damned if you do and damned if you don’t situation there. I’m in no way defending what Disney did, but in the minds of the public, Disney was going to infuriate huge groups of people no matter what they did. I don’t know, as a Disney scholar and professor, I’m always more interested in thinking about what the fights that Disney get embroiled in say about Disney as a cultural powerhouse, because Disney is shared American culture. Americans do believe they have some sort of stake in the ownership of Disney, even if they’re not really fans or consumers. They think that Disney is American, and thus Disney is theirs, because Disney is so intertwined with American culture and American values and American’s perception of history, if Disney does something somebody doesn’t like, many critics aren’t going to just see that as this huge corporation making a business decision. They’re going to be like, “this is an assault on my values and America’s story.” It’s going to hit emotionally, because Disney is all about emotion. It generates emotion. We do have some chapters in the book that try to look at this complicated question. So there’s one chapter called “Only Correct,” which is a play on Ian Forrester’s line, “Only Connect.” This is by Bonnie Rudner, who teaches literature up in Boston, and she’s looking at what she calls the corrections that Disney makes to its popular IP to its popular intellectual property. So specifically, she’s looking at Disney Renaissance films: Little Mermaid, Beauty and the Beast, Aladdin, Pocahontas. And she looks at first, how did Disney change it from its original form of a folk tale or a historic story to put it on the screen? But then, how is Disney changing it again, as it becomes live action or staged on Broadway? And she’s offering us a way to think about these corrections that goes beyond like a knee jerk “I don’t like the black mermaid because the mermaid should be white.” We know that that’s simplistic. What other opportunities are there for analysis that are rich but not just reactive. So we have other chapters like that. We have a chapter by a museum specialist, Bethanee Bemis, who is looking at Main Street, just an experience walking down Main Street USA as something that is deeply rooted to our senses and to embodiment. And what does that bake into us, in some ways about the American experience, even though it’s completely ahistorical like the actual Main Street USA, would probably have been filthy and a little bit scary, stinky, [LAUGHTER] what would have actually been in the street of a Main Street USA. Also, some of main streets by the turn of the 20th century were struggling economically. But what she looks at is how Disney’s creation of this main street imprints on us sensorily. And then she starts to look at some of the changes Disney has made to add more inclusion into its parks, and the implications of that, both in shifting our histories, but also inviting more people into the American story. You all are asking really good questions, and it’s things that our experts in the book are thinking about as well.

Rebecca: Giroux’s book, The Mouse that Roared: Disney and the End of Innocence, argues that Disney’s influence on the culture was dangerous. Do the authors of Why the Magic Matters share this concern.

Jill: The contributors don’t speak with one voice. There are 22 contributors in the book, but I think some of our contributors would say that there are elements of Disney that could be considered, quote, unquote, dangerous. Some of Disney’s stories, its depictions of race or ethnicity or body type, just capitalism, like all of these could be read as dangerous or potentially dangerous if misused. But we also have contributors who, I think, kind of unabashedly love Disney, and in fact, their careers have centered around celebrating the company and its history for what it can offer. Two chapters that come to mind here: we have this delightful chapter by Christopher Tremblay. Was a scholar and a teacher, and every summer, he takes students on this experiential travel course called “Walt’s pilgrimage.” He takes students basically along Walt Disney’s biography. So like Chicago, Illinois to Marceline, Missouri to Kansas City, out to California, ending at Disneyland and Forest Lawn Cemetery, where Walt is interred. They learn Walt’s life story as well as all these inevitable lessons about business and pushing through hardship, about hope in the face of adversity, and also about placed-based learning and ethnographic discovery. And so that chapter treats some of Walt’s complexity, but it’s more like, what can Walt teach us? Another chapter that’s like that is Jeff Barnes has a chapter, a class that he teaches at Disneyland. So in the chapter, he’s like walking us through Disneyland, and stops at these landmarks to talk about what students can learn, about leadership, about how to treat people in a working environment, about collaboration, all of that. So there are really a range of perspectives in the book, some that are more critical and sort of begging Disney to do and be better. And then we have some chapters that are like, Disney is what it is. It’s hugely influential. This is how it can make sense for you, and how you can sort of deploy some of its techniques. Like Disney’s been a remarkably successful company for much of its 100 years. What are they doing well, and what can we learn? So there’s a real range of voices in the book, and this is a nice moment to be doing Disney studies. I would say Disney studies, like when I was first reading it, like Disney studies in the 1990s and the early 2000s tended to be more critical and even angry. It was sort of fun to tear down Disney, and not entirely difficult to do[LAUGHTER] if you came at it from a certain perspective. But I have never found like a complete dangerous tear down as something that students respond well to. It also lacks the kind of nuanced thinking that I want students to practice directed at any subject. I think there are things about Disney that could be dangerous, like if speaking for myself, anything that is as culturally prominent as Disney must be critically examined and understood. We can’t just take it at face value, because doing so, that could become dangerous, in a way that ignorance in the face of power is always dangerous. So I wouldn’t say Disney’s dangerous. I’d say ignorance and uncritical reflection about Disney can be dangerous. So that’s one of my big motives in teaching the class.

John: Speaking of a nuanced approach, the book is organized into sections that represent dialectical pairs. Can you explain this approach and describe the sections that you’ve included in the book? I thought that was a really interesting way of organizing the structure of this.

Jill: Yeah, thanks. So the book is organized into these four sections with a pairing of terms that may seem contradictory, but in fact, like when we look deeply at Disney, we see that these ideas that seem contradictory actually go hand in hand. I describe it this way, like the book is organized around tensions, and thinking about Disney holding in tension a range of ideas, helps to negate any sort of black and white thinking about Disney, it’s not pure celebration, it’s not pure criticism. It’s about this nuance. For instance, one of these dialectical pairings in the first section is magic and strategy. I wrote the intro to that first section, and in that introduction, I lay out how magic is like one of those go to buzzwords that everybody associates with Disney nowadays, Disney equals magic, and yet, so much of what Disney does is meant to look magical and feel magical, but it is, in fact, deeply strategic. It is done by very talented artists and business people and cast members who are creating magic. Disney’s a multi-billion dollar company that knows how to press on your motions and win your brand loyalty. The chapters in that magic and strategy section look at how Disney does both things. It creates magic and operates strategically to create magic. The dialectic approaches, it’s a great one for teaching. I love to use dialectic in the theory portions of my courses, because they really push students out of black and white thinking. When I teach, for instance, African American religion, there’s this great book that explores the dialectical model of the black church, that’s Lincoln and Mamiya have developed that theory, and it’s great, because it’s like nothing is just one thing. It is a mess, a beautiful mess of contradictory things. My first book was called Womanpriest: Tradition and Transgression in the Contemporary Roman Catholic Church. So even there, I was playing with tension with these two concepts that seem contradictory, but are, in fact, essential to the thing itself. I know Alexis has experience with dialectics in a therapeutic form, because she also has a background in psychology, and I use them in some of my teaching as well. And so the humanities professor in me and the social scientist and her like, both resonated with this dialectic approach, this theory of tensions. So we have magic and strategy. We have leisure and labor. We have authenticity and simulation, and we have nostalgia and innovation. Those are our four sets of pairs, and each of those words seems like that they want to fight each other, but in Disney, they coexist and create what Disney is.

Rebecca: Who is the intended audience of your book? You’ve kind of hinted at some potential audiences. Can you talk a little bit about who your intended audience is and maybe who might also benefit from your book?

Jill: We would love the book to find wide readership among Disney fans who are interested in a more academic, dare I say, nerdy, take on Disney. And we also envision this as a book for college students taking college courses on Disney. We learned in the process of doing this book that there are a whole lot of Disney courses out there taught from a range of disciplinary perspectives. It was one of the things that got the publisher’s attention that this book, targeted at undergrads, could actually find a home in courses. xc, who I mentioned earlier, has written an academic article… it was probably like seven or eight years ago now… but he looked at all of these Disney courses that he could find, like on college websites and things, and he found 70 happening in various colleges and universities. And I know there are more. So we definitely think the book could be for college students, but I’ve been recommending the book to like friends who have teenagers who love Disney, like, “hey, if they love this, they’re going to love getting to learn about this, and they may actually find some interesting academic disciplines that they hadn’t thought about via Disney.” I think the book could be really for a wide range of people. We made sure that it was accessible, that the language was not something that would be too intimidating. This is not like just aimed at a very small number of academics. We want this to be something that people might pick up and read a chapter here and there. It should be something that is Invitational to educated readers who are curious.

John: So given all those courses out there, have you considered coming up with a second edition or an expanded edition of this?

Jill: We were told by the publisher, it depends how well this one does. So if you all like this book, buy it for your friends. It’s great for Christmas gifts. Yeah, they said there’s, I forget the number, but there’s a number of books that they like to see published before they talk about another edition. That’s what they told us. Hopefully it will go over really well, and that there will be a demand for more, because I know there are many more scholars out there doing really awesome things in the classroom around Disney, and I think the more we share that, the more we can think about Disney in fun ways, and also the more we can make a case for higher ed and education being something that really can operate in one sort of day to day life, or one’s experience of life, if Disney is just entertainment, we can think about those things richly. We’ll see John, hopefully.

Rebecca: So we always wrap up by asking, as you know, what’s next?

Jill: So research wise, I’m hoping to get back to my solo Disney book project. I sort of tabled that when Why the Magic Matters started to take off about two and a half, three years ago. So I’m hoping to get back to it. That project is called Faith, Trust, and Pixie Dust: Enchanting and Evangelizing America. That’s currently the title, and it looks at Disney alongside American religious history. I don’t want to say it’s Disney is a religion, but how the ways we can analyze Disney can intertwine with the ways we can understand religion, and in so doing, understand religion more richly and how it operates in the world, and understand Disney more richly and how it operates in the world. So I’m eager to get back to that. In addition, even though my Womanpriest book came out five years ago, now I’m still writing and talking a lot about women’s ordination and the Catholic Church. I’m, in fact, headed to Venice, Italy in a couple of weeks for a conference at the end of October where I’m going to talk about women’s ordination and different types of Catholic feminisms, like comparing North American Catholic feminisms in European. So I’m grateful that there’s still a lot for me to talk about with that. In terms of teaching. I’m not sure when I’m going to teach Disney again, maybe next fall. We’ll see how scheduling works out, but I would love to use this book in the classroom to see how that works. I am designing currently a new course for spring 2026 which is called The Holocaust and Contemporary Issues. It was Holocaust studies that got me into academia, that got me into religious studies in the first place. It was transformative to me as a 19-year old to understand the magnitude of what happened with the Holocaust and the implications of that. I’ve often taught an upper-level Holocaust course, but I’m developing a class that’s for first years or sophomore, students who haven’t a lot of experience in religious studies. And we’re going to spend about a month on Holocaust history, and then we’re going to look at what’s emerged since that’s related. So we’re going to do a unit on the Rwandan genocide. We’re going to look at some of Timothy Snyder’s work on rising authoritarianism in the 21st Century. We’re going to look at the situation in Gaza, nothing controversial at all. So basically, we’re going to ask what faith-filled resistance looks like. What does it mean to stand up against oppression and injustice in today’s world? I don’t know what will happen with that, but I’m really excited. I feel a call. I feel an obligation to dive into those hard issues. So that’s what I’m working on in the classroom.

John: Well, those are challenging issues, but very much needed, I think, in terms of education today, but it’s kind of nice also to take a break and reflect on magic and bringing some magic into lives where so much of the news that we see is much less magical right now.

Jill: I’ve had students joke that I’m the Holocaust professor and the Disney professor and like that… [LAUGHTER] I don’t know… is that a dialectic tension?

Rebecca: Is that a way to balance it all out?

Jill: Exactly. So I’m like, “Yeah, you’re right, those seem antithetical, and yet they coexist in my teaching and my research interest.”

Rebecca: Super comfortable in that gray zone, that’s for sure.

Jill: Yes.

Rebecca: Well, thank you so much for joining us, and we look forward to talking to you in the future about some of your other projects that you’re working on.

Jill: Thank you both so much. You all do so much good work with this podcast. I’m happy to be part of it.

John: Well, thanks, Jill, and I hope I’ll see you one of these days back in Durham, I’m still coming down there in the summers, or at least I have been so far.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing Assistance provided by Madison Lee.

[MUSIC]

420. Teaching from the Same Side

Antagonistic relationships with students are not uncommon—there is a history of faculty distrust of students. In this episode Michelle Miller joins us to discuss a same-side approach where faculty and students work together in support of student learning.

Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology, Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide to Learning Students’ Names: Why You Should, Why It’s Hard, How You Can. Michelle is also a frequent contributor of articles on teaching and learning in higher education to a variety of publications including The Chronicle of Higher Ed and is the co-editor with James Lang of the Teaching, Engaging, and Thriving in Higher Ed Series at Oklahoma University Press.

Show Notes

Transcript

John: Antagonistic relationships with students are not uncommon—there is a history of faculty distrust of students. In this episode we discuss a same-side approach where faculty and students work together in support of student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology, Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World and A Teacher’s Guide to Learning Students’ Names: Why You Should, Why It’s Hard, How You Can. Michelle is also a frequent contributor of articles on teaching and learning in higher education to a variety of publications including The Chronicle of Higher Ed and is the co-editor with James Lang of the Teaching, Engaging, and Thriving in Higher Ed Series at Oklahoma University Press. Welcome back. Michelle.

Michelle: Hi, thanks. It’s great to be here. Thanks for the chance to speak with you today.

John: It’s great to talk to you. It’s been too long. Today’s teas are:… Michelle, are you drinking tea?

Michelle: I am not. I had all great intentions, and one thing led to another, and our time was upon us, and so I will be drinking a nice, tall glass of water

Rebecca: Again, the foundation of tea. I have Lady Grey today, John.

John: And I have ginger peach green tea today.

Rebecca: Nice. So we invited you here today to discuss a workshop that you’ve been offering recently on teaching from the same side. What do you mean by teaching from the same side.

Michelle: So I want to just say right off the top here, it’s not like a formal framework or a theory or something like that, it’s really one resonant phrase that kept coming to the top of my mind to bring together a number of themes and ideas and even philosophies that I’m seeing converging out there in the world of higher education, pedagogy and course design. And so when I look at all the roots, I think of things like Pedagogy of Kindness, Cate Denial’s wonderful book and framework. I think about the idea of relentless welcome that was talked about in Peter Felton and Leo Lambert’s book, even some themes that come up from ungrading and alternative grading, all of which are feeding into this idea of: “Are there strategies? Are there practices that are essentially more adversarial than they need to be in our relationships with students and what might we try instead, and what can be some of the outcomes?” So those are some of the things that pulled together and have started again to come to the forefront of my mind as I’ve talked about this with faculty. So, teaching from the same side, itself, just the phrase, it’s also something that I wrote about in a newsletter issue from a couple of years back, and it keeps coming back to my mind, and I think it continues to resonate with readers of that newsletter. I think I got a lot more traction with readers that I have with some other ideas or themes or discussions that I’ve put out there in the past. That’s what got me thinking a little bit more about some fundamental realignments that can take place in our ideas about why students are taking our classes, our ability to meld those goals and find common ground with the goals that we have for them, and to even interrogate and question some practices that we may take for granted, even things like how we frame policies in our syllabus, what those policies even are, and how we think about drawing students into the concepts and material that we want to share with them as excited, engaged teachers ourselves.

John: It makes so much sense that we should share the same goals as our students, that we want our students to learn, and they want to learn. Yet a lot of faculty come into teaching with this adversarial approach. And Cate Denial talked about that in her book at the beginning, how she was encouraged to not trust students so much. And I remember getting a very similar induction into academia from colleagues. Why is an adversarial teaching approach or teaching relationship so common?

Michelle: Well, I really like how you talked about your formative experiences as a college instructor. And I think even for those of us like myself, where it’s: “ah, that was decades ago. How could that still be affecting me today?” But that’s what formative experiences do. They set us on a track, and they put us in a mindset that may be with us for a lot longer than we realize. So I think it’s really productive to think back, and I encourage faculty to do that as well. When I do get to talk to folks about this and work with them, I say, “Well, what were some of your early experiences, both as a student and as a budding, upcoming instructor?” And yes, like so many of us, I was told explicitly and implicitly at every turn, “Well, you got to keep an eye on those students. Your job is to choose the goals and the content and make the demands on the students.” We are the experts in the room. It makes sense that we would be leaders in that way. But then that turns into the sense of and you can absolutely expect that students don’t want that. I’ve heard different variations on: students just want the easy points, they want to check a box, they want to get a credential, go on to have a job and not intellectually engage. And so, hear that over and over and over, and you’re encouraged, and maybe it’s even valorized in certain cultures, certain departments, certain institutions, that the more you can push that onto students, the better instructor you are. And so I think many of us are questioning that, but we may not have some better ideas or anything concrete to replace it with, or we’re concerned what would happen if I changed this policy or took a different approach? And I’ll say that I also had an experience I’m just realizing now I’ve been thinking back to this experience. It was actually the very first time I ever taught a course as the instructor of record, and it was fraught with all kinds of stresses, as you might imagine [LAUGHTER], and there were definitely times when I felt that what I wanted was mismatched with what students wanted, and that we were on different sides, but it was long enough ago that I’m comfortable even sharing the particular conflict that happened. So I had a teaching assistant in this class, and the way it played out, the teaching assistant was about my same age, which was already kind of fraught enough, so I’m trying to navigate this relationship with him, and he is quite naturally trying to step up and be helpful and tell me what to do and what he thought I was frankly doing wrong in the course. And we had had a lot of student feedback coming in over and over that students, they weren’t happy with the course, but there was one thing in particular that they kind of wanted, and I looked, it was like, “Well, that’s an easy fix.” It just had to do with how many points and how fine grained of feedback they were getting on this one recurring assignment. I was like, “Well, we could do that if we change this midstream and it works better for everybody. Why not?” And my teaching assistant and I had a confrontation in my office. I mean, I remember it quite well. It w as the sort of where you’re both standing up on other sides of the desk and raising your voices at each other. And he said, flat out: “If you do this, regardless of what the change is, if you let those students know that they can ask you for something and that you’ll give in…” his words… “then you’ve lost it. The students are going to be in charge from here on out, and that’s going to be it for this semester. That is what will happen if you do this.” And I don’t see a good reason not to do it, and I’m going to try it, and we tried it. And what do you know? Students were happier and satisfied. We quit getting those same emails every single week, and it was fine. If anything, the level of respect and willingness to put forth productive effort improved. It didn’t go away. So I had the benefit of having that, but it’s something that I still had to tuck away and return to all these many years later and occasionally still do to remind myself that sometimes it’s a matter of just trying it. So even if we’re kind of dissatisfied with an older, adversarial philosophy, there are ways to experiment and find better approaches that meet our needs and our goals and our aspirations as the experts in the room, but also meet students’ needs a lot better.

Rebecca: We see a lot of examples where this us versus them relationship comes up. We see it on campuses, between faculty and administration, between students and faculty in a lot of different contexts. What’s the value of trying something different and not having this us versus them mindset.

Michelle: Well, the other really indelible experience and tipping point that I had around this does it have to be us versus them thought process was during remote emergency instruction. So during the pandemic itself, where that case, I think, like for so many of us, it wasn’t even a matter of like, well, maybe I could take something good and make it even better, that we could accrue benefits. I think we were in survival mode. I know I was. I had a handle on the technology. I think I was probably pretty well positioned, but I still had this experience, as many of us did in spring of 2020, going, “Oh, everything that I planned out and I was going to do and all the boxes I was going to check, because that’s what I always do, we can’t do that anymore.” And I just felt the ground fall out from underneath of me. I’m kind of going, Well, how do I choose? [LAUGHTER] And if we can only do some of these things and not a lot of the others, how can I make the right decisions, and part of it is realizing…[LAUGHTER] giving yourself a little bit of grace, and saying, “Well, you do the best you can, and not all the decisions are going to be right.” But I kept returning to that guiding beacon of, “why did students take this class in the first place?” None of us anticipated how it was going to play out when we all signed on for this. But what were the most important takeaways that they needed and wanted to get? And I just was like, “look, whatever the activity or learning objective was, if it aligns better with that, what brought them to the course in the first place? That’s what we’ll do, and other things are going to take the back burner or go away.” So I think that was one real benefit of when we know we need to revise something and not everything can happen, it can be a prioritization tool. But, what I sense is drawing in some of my fellow faculty to this is the idea of that better and more welcoming atmosphere and the possibility of taking more of our contact and conversations with students out of the realm of like the negotiation over points. “If you do this, I will give you that.” I don’t think any of us are really in teaching and getting inspired by conversations like that. sometimes they have to happen, but to have fewer of those and more of “I am struggling with the material. I’m worried about how this is turning out. I don’t know what to do next,” and me being able to step in and say, “Ah, I can help you,” because we are fundamentally on the same side. And even up to yesterday, when a student was asking me something kind of involved about a test grade, and could we do this? I had to take a breath, step back and say, “Okay, what moves her closer to the common goal of her learning about and getting excited about psychology, which is what we’re doing in intro to psych, and what would move her further away from that?” And yeah, more of a same-side solution, without getting too far into the weeds of what it was, I think we hit on something, and I saw a little bit of a light come up for her as well. Now, I think non-adversarial teaching approaches of different kinds are starting to make it into scholarship and teaching and learning and other sorts of literature. But here too, I want to be upfront and say, Hey, I don’t think that we can point to 20 research studies and five meta-analyses showing, well, if you tweak this one thing about your mindset or philosophy, that students are going to attain X, Y and Z. I hope it happens. But right now, I’m in that, to me, exciting and sometimes intimidating state of like, I think there’s something to this, but we just have to keep working working with it.

Rebecca: Minimally, maybe you’ll just be happier. [LAUGHTER]

Michelle: …minimally happier as an instructor. Yes, that’s not about just saying anything goes or let’s always be buddy buddy, and I never want to have that conversation about your grades. It’s not that so much. I think it’s a matter of balance and realizing how even subtly telegraphing or communicating to students like, yeah, no, I said it was all about the learning, I mean that and the points, all that will shake out [LAUGHTER] in the end, and I’m actually willing to act accordingly and keep on that talking point. That really does start to make them step back and say, like, “Oh, actually, this is different, and I think I like it.”

John: Earlier you mentioned that so many of the discussions among faculty are with perceptions of declining student engagement, less student motivation, and students seeing education primarily as a means to a degree and checking off that box so they can move on to the next stage. And there are some surveys that indicate that students today see less connection between what they’re learning and their future lives and careers. Might the same-side approach help support an environment where it’s easier, perhaps, to motivate students,

Michelle: Right, so student motivation. And here too, it’s sometimes, even to this day, still like a little uncomfortable to me as a cognitive psychologist and one who comes from that tradition and sort of side of our psychology. But, as I frequently share about Minds Online. I realized just in the course of writing that first book, which is very cognitively oriented, we talk about memory, we talk about transfer of thinking skills and all this stuff. But I was like, “Oh, wait, if we don’t also talk about why students are doing all this great evidence-based [LAUGHTER] coursework in the first place, then why are we here?” So yeah, just acknowledging that, and reminding myself pretty much every day as an instructor that yes, I do have to think about this. So there is that perception. And I’m not, first of all going to necessarily just totally turn away from somebody else’s authentic experience and say, “This is what my students are telling me, but I respectfully also submit that when I really started reflecting on it, my experience, even recently with students, has not fit that mold of like, “Well, Professor, can’t you just not take up my time, give me my points and let me go on my merry way to a job.” I have students expressing a very diverse range of goals. Some have not formulated goals. They’re not trying to tick a box to get a job. They don’t know what the job is yet. They’re not even sure what credential they want to get. So I think in the right environment, if you know what to look for, and you’re primed to look for it, that you will see that students they do want to learn. Now do they always act in accordance with high intellectual ideals that we start the semester with? No, and I would submit that none of us do. It’s just not how human beings work. But there is a core and a well of authentic intrinsic engagement. But actions need to follow those ideals. So we talk about motivation. I do keep coming back to and frequently sharing with my fellow faculty two old good solid aspects of motivation theory that just seem to work time and again. And one of them is actually getting a lot of new air time as well, and sort of a revival of interest. So I can share those with you, and I’ll hit the real high spots of these. First of all, self-determination theory, our old friend… and all of the motivation theories, by the way too… they also really bring home an important point, which is not to think about motivation in terms of “you’re a motivated person,” or “you aren’t.” Motivation theory just really looks at the conditions of the environment and the setting, which is great news for us teachers, because students don’t have to walk in, just motivated people. They can respond to the cues and the incentives we give them. So self-determination theory is we’re more motivated when we have autonomy and choice in what we do, we have some sense of mastery, or that we’re getting better at what we’re learning, and there’s relatedness or social components, so somebody else out there cares [LAUGHTER] that I’m getting better at whatever this thing is. And I’ve just seen informally, but it’s a very strong impression that I have over the last few years of like scholarship of teaching and learning and learning and learning sciences research, lots of folks are discovering or rediscovering these principles and say, “Well, what happens if we actually bake these into, say, an online class? What happens to student achievement, especially around the autonomy and choice part?” And that’s one that too, that used to be really tough to do. I tried building all kinds of choices and saying, “Well, maybe students can be graded more on tests or weight their assignments more heavily.” I tried that many years ago, and it was so hard to do and track, and now, with better technology tools, we can say, “Why didn’t you pick this? This is your strength and you want to pursue it. Do that, or here’s a fork in the road, pick A or B.” So those things can really elicit just some basic, more motivated behavior. So there’s that, and then there’s always intrinsic and extrinsic motivation. Lots of us say, like, you don’t have to just rely on intrinsic motivation and hope that it will be there. There’s ways to kind of promote that, and it doesn’t mean that you have to get rid of all grade or other kinds of incentives. So I’ve been playing around with different ways to spark that intrinsic motivation, some sneaky ones that psychologists know about, and also some ones that I think lots of us hit on through our teaching experience, things like introducing surprise or asking students to predict, rather than always just telling them what the outcome is, so that can help build that over time.

Rebecca: Were there some sneaky ones you were going to share?

Michelle: Well, see how I did that. [LAUGHTER]

John: Is there a secret handshake or something needed for these?

Michelle: Oh, it is. It is, it walks a fine line between psychology and perhaps a little bit of manipulation, but people can decide if this is in their comfort zone or not. It has a terrible name, which is the “hypocrisy effect,” but has wonderful outcomes. So essentially, it’s a way to try to induce attitude change by getting people to act in accordance to what we’ve said or done before. So we all kind of naturally like to be consistent. If we say it, we tend to start to believe it over time. So there’s ways of doing this, essentially getting students to endorse or identify for themselves the value of either course features or things they’re learning about. Like I have a colleague here at NAU who, years back, was teaching a first-year seminar and it’s a pop culture topic. I won’t get too deep into what it was, but you might look at that and say, “Oh, sure, it’s inherently interesting.” Then again, students may look at it and say, “Well, maybe I should spend more time actually studying in this required chemistry class” or something. And maybe their families are even saying, like, “Why are you taking that?” So, right from the beginning, what he did is he asked students to write a letter home. I don’t know whether they sent it or not, but this is a classic technique to tap into this. So that was the assignment: write to your family and tell them why you’re taking a class in this thing that, on the face of it, might seem a little frivolous, and talk about the skills you’re going to acquire and how that’s going to help you in everything you’re going to do to make the most of your college education from here on out. So, that could be a way to spark authentic buy-in to: “Yeah, this is why I am actually going to put effort into this class,” or in my case, I hit on this with an assignment that does double duty, trying to get more investment in using class time and staking more of the grade on that without doing very traditional participation grading. So they do a reflection on what they’ve been learning over the last two weeks. And one of the prompts that’s in there does ask them like, “What’s your favorite learning?” and yeah, I’m leaning on them pretty heavily.[LAUGHTER] It’s a little heavy handed to say, “Tell me what you like.” But what does that do? You come up with something. I mean, students do have the option. If they’re like, “this is not doing it for me,” [LAUGHTER] I won’t take that personally, but most do find something. And it’s a good fit for intro to psych, because you do something totally different every week. There’s something for everybody [LAUGHTER] in this course. And so they’re the ones saying, “Well, here’s why learning about Pavlovian conditioning can really help you in your life. Here’s something I didn’t know about our sensory systems that was really insightful.” So instead of me always telling them, they’re telling me, and it should spark more engagement over time.

Rebecca: Good. I want you to get off the hook without the sneaky… [LAUGHTER]

Michelle: Yes, knowing me as you do.

John: In our past discussions and much of your work, you usually focus on the cognitive side of learning, as you’ve mentioned, things like attention, memory, critical thinking. How might teaching from the same side support all of that?

Michelle: Well, I think that that’s just such an exciting perspective on this as well, and that’s something that I’ve really been challenging myself on lately too. Say, are these two separate things? So building in things like more retrieval practice, that active, effortful recall that we know is so much better than just the standard playbook of like, okay, you make a PowerPoint deck and explain it to me, and then I’ll take it home and reread it, which we know does not work really well from the knowledge perspective. Distributed spaced studying is sometimes harder to do as schedules get busy, and transfer and practice, this is hard stuff to actually get in there and keep practicing the skills, including with really messy, challenging problems. All that stuff is evidence-based. You know it works. I know it works. Our students, on the other hand, just they see the effort, or even some other facets of this that dissuade them from it. So the way I’ve been thinking of it lately is like, okay, so we finally have these evidence-based practices going mainstream. Like, really over the last 10 years, it’s this incredible success story with more people, podcast listeners, there’s all this great interest. So that’s wonderful, but maybe the next level is to put more into getting students to internalize those on their own. Because, after all, I mean, these are techniques that even if they’re not built into a class, if I know about them, I can probably bring them in as a student, or even if I’m not in a formal class, I’m in professional learning situation years after I’ve graduated, what better skill to have available to me than knowing how to break something down and say, “Oh yeah, I’m going to use some of these same principles.” And there’s some suggestions from the research literature that when students do have those good experiences and see it in action, that they actually do internalize it, even without anybody standing over them and saying, let’s do this. So I’m challenging myself to say, “Well, can I change my mindset from evidence-based practices, cognitive learning principles…?” These are not things just to do two students, say, “All right, I’ve applied this to you because I know it will help,” but also to say, “Hey, this is why it is the way it is, and isn’t it exciting? Let me bring you in.” And that might seem simple, and some of it does come down to transparency, this concept that many of your listeners may be familiar with, where we just quickly say, “Okay, here’s the assignment, but also here’s why it’s here, why it’s set up the way it is, and how it’s supposed to help you.” There is some research showing that that’s beneficial. Students will sit back and go, “Oh, okay, now I get it, and I’m more invested, and I’m on your side now as well to say I’m going to do this hard project and do it the way that you suggested.” So yeah. There is that. But I think also what I’ve facing up to myself is it’s not always just like, “Well, let me just explain to you a better way to do it, and then why won’t the students do it?“ And perhaps coming away is like, “Oh, they’re lazy. They don’t want to try something new, or they just aren’t listening to me.” Well, think of it in terms of risk. Students, by the time they’ve gotten to our classrooms, they are, of course, not blank slates, especially when it comes to their study techniques. If you talk to students for any length of time, you know that they’ve developed what they believe oftentimes are successful study approaches. And it’s a risk. So it’s really a risk-benefit calculation that our students may be engaging in to say, “Well, not only are you telling me to do this thing that’s hard, to close my book and quiz myself, but that’s going to take away from or replace something that’s worked before. Just because you’re telling me to do this, you want me to put my grades on the line [LAUGHTER] and throw out the thing that got me to here.” So when you see it in that way, you realize that there’s perhaps some different tactics and approaches. And here, some really esteemed cognitive psychologists have picked this up as really a challenge. So Mark McDaniel and his longtime collaborator, Gil Einstein, they’re known for a ton of different applied research findings and discoveries all over. And they stepped back and said, “Oh, wait a minute. This is an example of a dynamic that has been with us for some time in all branches of applied psychology, which is just knowing doesn’t transform behavior. Maybe folks out there are all eating their eight servings of vegetables and 30 minutes of exercise, doing everything we know we’re supposed to do, we don’t do it.” And there’s lots of reasons why, but Einstein and McDaniel said it’s still my sense is a work in progress, but they’ve put together a pretty detailed step-by-step program. They call it KBCP. So knowledge, belief, commitment, planning, I think, is what it stands for. But basically, you can get the sense here that it’s about not just “do your retrieval practice,” but also, “let me tell you about it, sure, inform you. But I’m not going to stop there. Let’s also do some demonstrations. Let’s try it with and without, and we can all see it in action.” So building that really authentic buy-in, and then encouraging students to get out there and try it in higher stake settings. So okay, what do you plan to do for that big exam you have in that other class? And let’s lay out a plan for that, and then come back and reflect and say, Well, did it work the way that I predicted it would [LAUGHTER] as the expert? And now, what do you want to do? So again, it’s something that I think is emerging, and their program is pretty detailed. So I don’t know that everybody’s going to go through that step by step, but the spirit of it is very much with me as a cognitively oriented instructor and researcher to say, “Okay, but how do you really reach people so that they are willing and excited to take it forward” and realizing that students, under the right circumstances, may do that, and that’s a much greater gift to them than just what you’re doing by designing a single class.

Rebecca: One of the key things of that model seems to be the idea, or from my experience, is that ability to achieve something and achieve that success and see it working, so that you then might be motivated to try it again. Thinking about any parent who has a kid that doesn’t like to practice stuff, but if they have some success in seeing that the practice helps them achieve something that they want to achieve, then they’re more likely to do it again.[LAUGHTER]

Michelle: It’s a competence piece. Now you’re putting it that way. Now I’m kind of seeing that connection. It’s the competence piece of SDT, and, yeah, as inspiration to stick with it and do the hard things. And when we’re on the same side, we have our techniques as parents who are also in the motivation business, but I have to feel like we are in a better position to do that when we can help celebrate and say, remember, we are in the same side. And here’s where it was difficult, but here’s where, yeah, you tried it and it worked, and let’s celebrate that win together from the same side.

Rebecca: You’ve mentioned a wide variety of cognitive practices. Can we think about what that looks like in practice?

Michelle: Yeah, and here’s where we can tap a little bit more into my other love within the field of psycholinguistics, but some of this is just basic communication principles as well. So as a piecing together and pulling from lots of different areas, how do we communicate that we are on the same side and establish that atmosphere of, frankly, trust and a non-adversarial approach to these things. Some of the things that have struck me as just, first of all, a lot of this work, a lot of the folks that I would consider to be my models, and the folks I’m looking up to here, they’ve talked about the syllabus as a starting point. So there’s lots and lots and lots of work out there. Sometimes, like, as the work builds up, it’s hard to synthesize, but a few threads that I’ve been pulling out. So anybody who has not read Kevin Gannon’s classic, I think, “How to Create a Syllabus” guide, it came out in the Chronicle of Higher Education a few years back. It was really, I think, one of the first clear, unambiguous, like “we need to do this and not that” in our syllabi, including calling into question, in a good way, a lot of the things that we have taken for granted, and I know I just copy pasted [LAUGHTER] semester after semester, including things like using all caps. It might come from a good intention of like, “this is important, I want you to see it.” But as Gannon so accurately points out, this is how we scream at each other online. And why do that? [LAUGHTER] and it’s just saying is there some other way to highlight what’s important? Or even, I had a faculty member say to me recently, she’s like, “Yeah, for all the complaining about, oh, students don’t read it, students don’t get it. Well, think about the policies we deal with. When do we look at them? …was when we need them.” So yes, absolutely, students should read the thing from top to bottom. This faculty member, she even sends out, pieces out, like relevant parts of it as reminders or announcements at the right times of the semester. So there’s other ways to emphasize besides just the all caps and Gannon also talks about is like, is there some way that your syllabus is kind of getting across that, not only here’s the things I don’t want you to do, but there’s no reasons for it other than I say so and it gets across this idea that “and I expect you to do all these bad things.” So playing around with ways of dialing that down. So syllabus warmth is another major concept, some interesting findings about how sensitive students are to that. And then lastly, I ran across, just in the last two years, a great article came out, maybe we put in the show notes. It was focused on STEM syllabi. And here they just took a very open approach, and they surveyed a variety of syllabi to say which of these seems more supportive and inviting and warm to students? And they have a great list that they laid out, including things like just phrasing things as we: we will, we are going to engage in these difficult topics, and here’s how we are going to do it, instead of students must… [LAUGHTER] which is what used to be in all of my syllabi, talking about challenge in positive ways. And here’s the resources that we want. And again, sometimes, yeah, it’s a good thing to have, like, “well, here’s the tutoring center.” But to say, “hey, successful students go to the tutoring center, and I promise you, [LAUGHTER] they will be supportive,” …some other way to do that and encouraging, very explicitly, that we do want students to come talk to us. So yeah, it’s like, well, I said what my office hours were, and now I’m sitting here, and why won’t students come talk to me? [LAUGHTER] And now I have several other alternative ways to schedule appointments, and I do give a little rah rah, but I also back up that verbiage with “and here’s even more access to me and all the ways to do it and in the circumstances under which you should do it.” So that’s one big starting point, is the syllabus. We can definitely get into a few others. But if you start there, I think a lot of other things will fall into line.

John: And we should mention we have a few past podcasts that address some of that. We’ll include some of the links in the show notes related to effective practices in creating that same side environment in the class using the syllabus.

Rebecca: One of the things that I think I’m hearing you say is that to be on the same side doesn’t mean we don’t hold boundaries, it’s just how we communicate those boundaries.

Michelle: Yeah, absolutely. And to pick up on that, run with that a little bit, that is now a really important part of my practice as well. While I’m going through the syllabus and cleaning it up and fixing it up, as far as tone, I’m also questioning the policies, so things like I’ve written about being pretty critical, and I don’t do anymore, the 10% per day deduction for deadlines. I handle deadlines very differently now, which is not to say I don’t have any or it’s anything goes, but I think it’s more the process of questioning it, of saying, “Well, which deadlines in this course are actually fixed, and have I explained that to students or mentioned it in any way to differentiate those where there is a firm deadline, like you really do have to get your discussion post in by Sunday night,” because, Haven’t we all been in an online discussion and somebody answers you three weeks later and we don’t actually have a conversation? Didn’t take forever, but that’s just a little bit of saying, “Yeah, it’s not just because I say so.” So yes, as we get into the substance of it, that is where we do have to think pretty deeply about what appropriate boundaries are. And just for the record, too, it doesn’t mean being buddy buddy, and we’re all friends with students at all times. I’m pretty formal in my interactions with students. That’s appropriate for my style and my persona. But I would challenge listeners, go back over a syllabus, as you’re planning for next semester, identify at least one thing to say, “Is there some way that this could be less harsh?” Perhaps even, as I mentioned, I have my little heuristics for saying, “Well, do I say yes or no?” Does it take the student closer to the learning goal or away from the learning goal? If it’s an assignment that’s intended to develop their skills, so it’s a formative assignment somewhere in the middle of the semester and it’s late, I mean, if the goal of that is to engage them in the learning and that’s why I worked so hard building in the first place. Well, if they’re five minutes past the deadline, don’t I still want them to do it, maybe I still want them to do it a week after the deadline. So with that learning and the shared goals and those shared ideals as the reorientation point, that’s what we can go through and do. So yes, it doesn’t mean throwing out grades, necessarily, unless you want to bring in ungrading for other reasons. It doesn’t mean that now we’re all completely informal, and it doesn’t mean that there aren’t any hard and fast boundaries in the course. But I’ve thought those through and not simply just transferred them over, because that’s what we always do, and if we don’t, students will take advantage of us.

John: One other thing that we’ve talked to you about in the past is the importance of learning students’ names. Could you just talk about that a little bit, and we’ll include a link to your book as well as the earlier podcast on the topic.

Michelle: Yeah. So I mean, I guess if we trace the whole evolution of how to get on the same side or begin that in our instruction. It starts with me thinking about my past and what do I believe is true about students, and then looking in at the overall policies and the objectives of the course, then getting into the communications, and then I’m in the classroom with them, and if it is a face-to-face class, one of the very best, often surprising to students, and effective ways to make a personalized classroom atmosphere is to know and use students’ names. And so that’s also something folks who’ve followed my writing or things I speak about know, that’s something that I’ve really been on a mission to do in my own practice, and to help others know how to do it as well. Now, that said, it’s very, very difficult. That’s one of the themes of the book. If folks see it right off the top, it’s like, and if you’re one of the many people who sidle up to me in various settings and go, “Oh my gosh, I have to tell you something. I’m not good at learning names,” like, okay, most of us are pretty bad at it. It takes effort, and that can be an effort that we model. I mean, even as I circulate around in my classroom, if it’s a big class, we’ve got little table tents with names, which is a great starting point, but I want to eventually know the names, [LAUGHTER] so I go around to the little groups during small group discussions, “Okay, turn your table tent over. I’m going to practice.” And they see me making mistakes. They see me putting effort into it, and see me struggling a little bit, which maybe also establishes a little bit of common ground there as well. But if we put in that effort, I guarantee you that not every student, not every time, but over time, you will see a definite trend towards students saying, “You know what? I think I will listen to what she’s saying about the value of this course. I will follow her advice when she says, ‘study it this way, not that way,’ even though my way is harder, I’m going to stick with it when times are tough. And yeah, I’ll do what she says and try to focus on the ultimate goal of learning and attaining what this class is all about, and not just about taking off points and making sure that I can just put in less effort.” So I think that that is definitely of a piece with the rest. And I hope that folks, whether it’s through the advice in my book, or a lot of other good advice out there, or their own techniques, will keep working to get a little bit better every single semester.

Rebecca: I think we have lots of opportunities now to think about how to work more from the same side, but we always end by asking: “What’s next?”

Michelle: What’s next? Well, I’m going to keep working with this idea and seeing how I can bring it into the classroom and keep an eye out for the great research that’s out there, and continue to share that in the newsletter that I put out to faculty and instructional designers and others who care about learning. I will be speaking about this or offering me an opportunity to be part of a workshop… upcoming POD conference. So for those for whom POD is part of their professional life, they might want to check that out. And I will be going into a new role that I’m excited to take on at my institution. It’s a co-directorship position where we’re going to be looking at AI in the curriculum and bringing some of the developments and robust debate in that side of instruction right now, it’s part of, I think, all of our lives, bringing that into my professional life at NAU as well.

John: Well, that’s an area where there’s so many interesting challenges and such a rapid pace of change. That should be an exciting thing to be working on.

Michelle: Yep, all the more reason to get on the same side, don’t you think?

Rebecca: Definitely, and you won’t be bored, that’s for sure. [LAUGHTER] Lots of motivation to get involved in that work.

John: And I wish I could make it to the POD thing, but I’m going to be at the OLC Conference, which, unfortunately, is at the same time this year.

Michelle: That’s how it shakes out some years. We make our choices the best we can.

Rebecca: Well, thanks again for joining us, and hopefully we’ll get to talk to you again soon. Thank you.

John: Thank you, Michelle. Again, it’s always great talking to you, and I also hope we get to talk very soon.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Madison Lee.

[MUSIC]